Shrinkage with crossed grouping factors

[dmbates]

The shrinkage function, currently in https://github.com/palday/MixedModelsMakie.jl but being considered for incorporation here, compares the within-group least squares estimates with the conditional modes of the random effects plus the relevant fixed-effects coefficients.

This works and makes sense for a single grouping factor, say for longitudinal data such as sleepstudy. However, it is awkward to consider what should be the comparison values when there are multiple grouping factors for the random effects, especially crossed grouping factors like subject/item.

An alternative is to compare two sets of conditional modes, those at the estimated values of the parameters and those at very large values of θ. The initial value of θ is chosen so that the λ's are identity matrices. I believe that if we multiply θ₀ by, say 1000 or 10000 then we are essentially fitting a model the treats all the random effects as fixed-effects with just the tiniest bit of regularization or damping to make them all estimable.
The regularization comes from the I in Λ'Z'ZΛ + I so by making Λ very large we make the regularization very small.

I can code this up if it seems like a reasonable approach.

[palday]

This is somewhat related to olsranef in MixedModelsPermutations.jl . @jaromilfrossard has also worked out something nice (and better than what's current in the trunk branch there), which I haven't yet gotten around to integrating there.

[dmbates]

The approach of setting θ so that Λ is a very large multiple of the identity and extracting the conditional modes of the random effects for comparison with those at the estimates is now in the MixedModelsMakie code for shrinkageplot. I think it is rather clean and am grateful that we can drop some of the earlier contortions on fitting within-group models.

I think we can consider this discussion closed now.

[kliegl]

Thank you very much, @dmbates. I think this is very elegant and useful.