applying transformations on formula arguments

[mkborregaard]

Not sure whether this belongs here or on GLM?
A normal use case is to provide exponents (and sometimes log-transforms) directly on model objects. E.g.

using RDatasets, GLM
lm(@formula(SR ~ Pop75 + Pop75^2), LifeCycleSavings) #the key here is the exponent

However, adding the exponent seems not to be supported. The current workaround appears to be to create a LifeCycleSavings[:Pop75_2] = LifeCycleSavings[:Pop75] .^2.

There are use cases where this won't be ideal, because the model does not realize that it's the same input argument that has just been squared. Importantly, for predict you'd need to also do that transformation on the new DataFrame (newX). This in particular means that you cannot plot(newX, predict(mymodel, newX)) (for a single input variable), which is a particularly useful functionality IMHO.

[nalimilan]

Yes, this is the right place. Some previous discussion happened in JuliaData/DataFrames.jl#19 (one of the oldest issues) and JuliaData/DataFrames.jl#867.

[kleinschmidt]

I'm working on a pretty major overhaul of the model matrix setup that can handle this (among other things); see https://github.com/kleinschmidt/StreamModels.jl for a prototype.

[mkborregaard]

That package looks awesome!

[kleinschmidt]

Thanks! It's pretty rough at that point so I'd appreciate any bug reports or optimization suggestions if you do try it out.

[Nosferican]

Could some of the transformations be extended to take an index or indices? For example, lag / lead / first-difference.

[xiaodaigh]

Just to add on: currently @formula(target~a+b) works well. But @formula can't handle arbitrary transformation just yet like I can in R. e.g. @formula(fn(target)~fn1(a) + fn2(b)) doesn't work.

Support for R style literal computation I(a*b) would be nice too! But may use a different syntax

[kleinschmidt]

This is handled now by #71

[mkborregaard]

Awesome! Still too early to close this?

[kleinschmidt]

Yes I think so, let's wait until that (or something like it) is merged.

[kleinschmidt]

Closed by #71 (but feel free to re-open if that doesn't suitably address your needs!)

[mkborregaard]

Wuuuut! 🎉 :1000: