Multivariate Polynomial Regression

[MervynLarrierJr]

The tutorial is great, but here's where I'm at a loss: if I have multiple independent variables that I want to consider in the regression, how can I do that in a way that's similar to sklearns PolynomialFeatures: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html#sklearn.preprocessing.PolynomialFeatures
I've been using MLJLinearModels.jl where applicable, but would it be better to just sub-in the SKLearnMLJ interface?
If there's a way to do it that easily builds off of the tutorial and I've just missed that, pointing me in the right direction is also appreciated.

[ablaom]

Perhaps the InteractionTransformer is what you want. It takes a table of features and expands it to include monomials in the given features of some given order.

[ablaom]

And you can use a pipeline to chain the transformer with your favourite linear model.

[ablaom]

A non-MLJ alternative would be pre-process using an R-style formula, which is provided in Julia by StatsModels.jl. Unfortunately, there is no integration of that transformer with MLJ. I think you can use these formulas in GLM.jl, which is another provider of linear models.

[MervynLarrierJr]

Thank you. I've used the interaction transformer, but since it doesn't provide the square terms I'd been trying to construct some transformation matrix based on some of the footballs i found to little success.
Out of curiosity, is there not a specified transformer because for univariate polynomial regression it's easy to create the polynomial? Or are polynomial models not really considered useful? I'm very new to machine learning so I'm just trying to build my intuition.

[ablaom]

You are right! InteractionTransformer includes only monomials where degree of a given feature is 0 or 1. I had no idea. I've added an issue, as that seems unnecessarily limiting to me.

Out of curiosity, is there not a specified transformer because for univariate polynomial regression it's easy to create the polynomial?

Yeah, I guess most users are just creating these by hand (or use @formula from StatsModels.jl).

Or are polynomial models not really considered useful?

Certainly useful.