Generalize storage of factorizations.

[maleadt]

Our current Factorization structures often hard-code the storage of indices, coefficients, ... resulting in downstream packages like CUDA.jl or StaticArrays.jl having to re-implement these structures for compatibility. For example, https://github.com/JuliaArrays/StaticArrays.jl/blob/559134c9edda3b8ee9a71938386d2904338d7cc9/src/qr.jl#L1-L6 or https://github.com/JuliaGPU/CUDA.jl/blob/43e0f48b135325b97415cd1c8802e47732de85c6/lib/cusolver/linalg.jl#L77-L81. This is unfortunate, since it requires duplicating a lot of functionality, and requires special-casing in packages like ChainRules.jl.

This PR explores adding additional type parameters to a couple of factorization structures (more to be added) to explore the fallout such a change. It's unlikely that we can do this in Julia 1.x, but maybe PkgEval tells otherwise...

cc @andreasnoack, ref JuliaGPU/CUDA.jl#1194 (comment).

[timholy]

This will also be nice for the day when we finally get around to implementing arbitrary-axis support for appropriate parts of linear algebra.

[maleadt]

CI seems OK with this, so let's try PkgEval:

@nanosoldier runbenchmarks(ALL, vs=":master")

Note that I haven't updated all factorizations, so this may result in too positive of a report.

[maleadt]

@nanosoldier runtests(ALL, vs = ":master")

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

[andreasnoack]

None of the test errors are related to this change so it might actually be fine go move forward with this in 1.x

[maleadt]

None of the test errors are related to this change so it might actually be fine go move forward with this in 1.x

That's encouraging. I've updated the PR to include the remaining factorizations; I'll do another PkgEval run when CI passes.

[dkarrasch]

I wouldn't have expected errors, actually. I think the one major concern about adding type parameters is that when these are types of fields, the formely concrete types become unsufficiently parametrized and hence abstract, which may affect performance. But I'm not proficient enough with our package search engine to find out if that's a common pattern in the ecosystem.

[maleadt]

@nanosoldier runtests(ALL, vs = ":master")

[fredrikekre]

I wouldn't have expected errors, actually. I think the one major concern about adding type parameters is that when these are types of fields, the formely concrete types become unsufficiently parametrized and hence abstract, which may affect performance. But I'm not proficient enough with our package search engine to find out if that's a common pattern in the ecosystem.

Yes, I think we have avoided adding extra type parameters for this reason in the past.

[maleadt]

I wouldn't expect there to be many structs capturing factorizations, but there exist some at least. From a quick look at JuliaHub, filtering out instances that are currently abstract already:

• https://github.com/JuliaApproximation/ApproxFunBase.jl/blob/master/src/Spaces/QuotientSpace.jl#L8
• https://github.com/chriscoey/Hypatia.jl/blob/master/src/Solvers/systemsolvers/naive.jl#L148

I'd be happy to take a closer look and submit PRs to packages if that's an argument in favor.

[andreasnoack]

I think this change is worth it even if some structs become weakly typed. This is something that was requested before 1.0 but we didn't get around to so if it can be done without breaking code then I think we should move forward.

[timholy]

We added a type-parameter to ReinterpretArray that allowed us to get a 20x performance boost in some benchmarks (the new reinterpret(reshape, ...) framework), and StepRangeLen recently added one for correctness to allow the length to be encoded by something other than Int. I know the first triggered updates in some packages but there were, if memory serves, relatively few vocal complaints.

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

[maleadt]

Rebased. Let's do another PkgEval run to see if there isn't any unexpected breakage since last time:

@nanosoldier runtests(ALL, vs = ":master")

With the 1.8 feature freeze soon, do we want to consider this? Comments above seemed positive, PkgEval fallout minimal, and for the few packages that do regress (in terms of functionality of performance) I can create PRs to fix them.

[vtjnash]

SGTM

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

[maleadt]

@nanosoldier runtests(["CompressedSensing", "Conductor", "CopEnt", "DiffEqOperators", "DiffEqParamEstim", "DiscreteValueIteration", "DynamicAxisWarping", "ECharts", "Evolutionary", "GaussianMixtureAlignment", "HomotopyContinuation", "HypercubeTransform", "ImmersedLayers", "MRIReco", "MatrixPencils", "NavAbilitySDK", "NumericalAlgorithms", "ParetoSmooth", "Pathfinder", "Polylogarithms", "PowerSystems", "ReactionMechanismSimulator", "SignalAnalysis", "SimpleWorkflows", "Spirograph", "StatsFuns", "StochasticGene", "Sundials", "SymFEL", "VoronoiGraph"], vs = ":master")

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

[maleadt]

Package test failures are related, but respectively in @allocated and @inferred tests, so it seems nonbreaking:

HomotopyContinuation:

┌ Warning: `LU{T, S}(factors::AbstractMatrix{T}, ipiv::AbstractVector{<:Integer}, info::BlasInt) where {T, S}` is deprecated, use `LU{T, S, typeof(ipiv)}(factors, ipiv, info)` instead.
│   caller = ip:0x0
└ @ Core :-1
ldiv: Test Failed at /home/pkgeval/.julia/packages/HomotopyContinuation/UFWkw/test/linear_algebra_test.jl:60
  Expression: #= /home/pkgeval/.julia/packages/HomotopyContinuation/UFWkw/test/linear_algebra_test.jl:60 =# @allocated(ldiv!(x, WS, b)) == 0
   Evaluated: 32 == 0

Pathfinder:

A diag, D diag: Error During Test at /home/pkgeval/.julia/packages/Pathfinder/7iiV9/test/woodbury.jl:32
  Got exception outside of a @test
  return type WoodburyPDMat{Float32, Diagonal{Float32, Vector{Float32}}, Matrix{Float32}, Diagonal{Float32, Vector{Float32}}, Diagonal{Float32, Vector{Float32}}, QRCompactWYQ{Float32, Matrix{Float32}, Matrix{Float32}}, UpperTriangular{Float32, Matrix{Float32}}} does not match inferred return type Union{WoodburyPDMat{Float32, Diagonal{Float32, Vector{Float32}}, Matrix{Float32}, Diagonal{Float32, Vector{Float32}}, Diagonal{Float32, Vector{Float32}}, QRCompactWYQ{Float32, Matrix{Float32}, Matrix{Float32}}, UpperTriangular{Float32, Matrix{Float32}}}, WoodburyPDMat{Float32, Diagonal{Float32, Vector{Float32}}, Matrix{Float32}, Diagonal{Float32, Vector{Float32}}, Diagonal{Float32, Vector{Float32}}, QRCompactWYQ{Float32, Matrix{Float32}, QRCompactWYQ{Float32, Matrix{Float32}, Matrix{Float32}}}, UpperTriangular{Float32, Matrix{Float32}}}}

Some packages generate warnings, but don't cause test failures:

WARNING: Method definition logabsdet(Union{LinearAlgebra.Cholesky{T, S} where S<:(AbstractArray{T, 2} where T) where T, LinearAlgebra.CholeskyPivoted{T, S, P} where P<:(AbstractArray{var"#s879", 1} where var"#s879"<:Integer) where S<:(AbstractArray{T, 2} where T) where T}) in module LinearAlgebra at /workspace/srcdir/usr/share/julia/stdlib/v1.8/LinearAlgebra/src/cholesky.jl:695 overwritten in module WoodburyFactorizations at /home/pkgeval/.julia/packages/WoodburyFactorizations/zmYkv/src/WoodburyFactorizations.jl:146.

So all in, this looks reasonable, and I think we can merge this for 1.8. I'll deal with the fallout, and once it's merged look at fixing or improving packages.

[dkarrasch]

The ambiguity in Pathfinder.jl is due to the fact that type inference believes there can be a QRCompactWYQ{Float32, Matrix{Float32}, QRCompactWYQ{Float32, Matrix{Float32}, Matrix{Float32}}}, which may indicate that there is some no-op constructor missing, but looks fixable, I agree.

[maleadt]

Another rebase to confirm CI failures are unrelated (tester_linux64 had a linalg failure, but the log was not clear).
Should be good to go once that passes.

[maleadt]

GC failure is known issue with ASAN after the recent effects PR, so this looks good to go.