WIP: Backports for Julia 1.5-beta-1

[KristofferC]

Backported PRs:

• Fix bad c-tor for WeakKeyDict. Fix #35782 #35783 - Fix bad c-tor for WeakKeyDict. Fix WeakKeyDict c-tor for Pairs is broken due to missing sizehint! #35782
• Have matrix square root account for Rii=Rjj=0 to prevent NaN values #35758 - Have matrix square root account for Rii=Rjj=0 to prevent NaN values
• Add a note to sharedarrays documentation. #35807 - Add a note to sharedarrays documentation.
• fix #35793, assertion failure with union of singletons #35812 - fix assertion failure in codegen with union of singletons #35793, assertion failure with union of singletons
• Fix evalpoly docstring #35821 - Fix evalpoly docstring
• Bump Statistics stdlib module #35869 - Bump Statistics stdlib module
• bump Pkg version #35874 - bump Pkg version
• Fix ImmutableDict(:a => 1, :a => 2)[:a] #35871 - Fix ImmutableDict(:a => 1, :a => 2)[:a]
• More robust installation of BugReporting package. #35802 - More robust installation of BugReporting package.
• Fix codegen for sizeof for arrays with non-power-of-2 types #35900 - Fix codegen for sizeof for arrays with non-power-of-2 types
• fix some 1.5 compatibility issues #35965 - fix some 1.5 compatibility issues
• fix #35937, serializing TaskFailedException in Distributed #35943 - fix Erroring when inside a thread in a distributed process causes MethodError #35937, serializing TaskFailedException in Distributed
• Bump Zlib BB release to v1.2.11+10 which enables -O3 optimisation #35979 - Bump Zlib BB release to v1.2.11+10 which enables -O3 optimisation
• Implement reduce_empty(op, Union{}) #35843 - Implement reduce_empty(op, Union{})
• Expose some additional LLVM APIs. #35957 - Expose some additional LLVM APIs.

[KristofferC]

@nanosoldier runtests(ALL, vs = ":release-1.4")

[KristofferC]

@nanosoldier runbenchmarks(ALL, vs = ":release-1.4")

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @ararslan

[KristofferC]

Some stuff to look at:

["array", "reductions", "(\"mean\", \"Float64\")"] | 1.08 (5%) ❌ | Inf (1%) ❌
["array", "reductions", "(\"mean\", \"Int64\")"] | 8.12 (5%) ❌ | Inf (1%) ❌

Likely due to the change to mean in Statistics

---

["io", "array_limit", "(\"display\", \"Array{Float64,2}(10000, 10000)\")"] | 3.13 (5%) ❌ | 0.99 (1%) ✅
["io", "array_limit", "(\"display\", \"Array{Float64,2}(100000000, 1)\")"] | 3.55 (5%) ❌ | 1.05 (1%) ❌

---

["io", "serialization", "(\"deserialize\", \"Vector{String}\")"] | 2.36 (5%) ❌ | 2.05 (1%) ❌
["io", "serialization", "(\"serialize\", \"Vector{String}\")"] | 3.08 (5%) ❌ | 2.23 (1%) ❌

Likely due to string de-duplication in serialization

---

["sparse", "index", "(\"spmat\", \"splogical\", 10)"] | 2.90 (30%) ❌ | 0.77 (1%) ✅
["sparse", "index", "(\"spmat\", \"splogical\", 100)"] | 97.14 (30%) ❌ | 0.77 (1%) ✅
["sparse", "index", "(\"spmat\", \"splogical\", 1000)"] | 2.88 (30%) ❌ | 0.87

["sparse", "matmul", "(\"A_mul_B\", \"dense 50x5, sparse 5x50 -> dense 50x50\")"] | 1.64 (30%) ❌ | 1.00 (1%)
["sparse", "matmul", "(\"A_mul_B\", \"dense 50x50, sparse 50x50 -> dense 50x50\")"] | 1.55 (30%) ❌ | 1.00 (1%)
["sparse", "matmul", "(\"A_mul_B\", \"dense 5x5, sparse 5x500 -> dense 5x500\")"] | 1.67 (30%) ❌ | 1.00 (1%)
["sparse", "matmul", "(\"A_mul_B\", \"dense 5x50, sparse 50x500 -> dense 5x500\")"] | 1.70 (30%) ❌ | 1.00 (1%)
["sparse", "matmul", "(\"A_mul_B\", \"dense 5x500, sparse 500x50 -> dense 5x50\")"] | 1.53 (30%) ❌ | 1.00 (1%)
["sparse", "matmul", "(\"A_mul_B\", \"dense 5x500, sparse 500x500 -> dense 5x500\")"] | 1.64 (30%) ❌ | 1.00 (1%)
["sparse", "matmul", "(\"A_mul_B\", \"sparse 50x500, dense 500x5 -> dense 50x5\")"] | 1.36 (30%) ❌ | 1.00 (1%)

["sparse", "sparse solves", "square system (lu), vector rhs"] | 1.87 (5%) ❌ | 1.71 (1%) ❌
["sparse", "transpose", "(\"adjoint!\", (20000, 10000))"] | 1.00 (30%) | 4.00 (1%) ❌
["sparse", "transpose", "(\"adjoint!\", (20000, 20000))"] | 1.00 (30%) | 4.00 (1%) ❌
["sparse", "transpose", "(\"adjoint!\", (600, 400))"] | 1.01 (30%) | 4.00 (1%) ❌
["sparse", "transpose", "(\"adjoint!\", (600, 600))"] | 1.00 (30%) | 4.00 (1%) ❌

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

[KristofferC]

I ran PkgEval with the wrong base branch so results here are not 100% accurate (although it should be pretty close because not much have happened on master since branching).

[maleadt]

Let's rerun to be sure (and stress-test some PkgEval changes):

@nanosoldier runtests(ALL, vs = ":release-1.4")

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

[KristofferC]

The PkgEval result is extermely noisy now due to all changes in the RNG streams (#32208, #33721, #35078, #29240).

This are the packages that failed the previous PkgEval but I filtered out all packages that failed in one of the PkgEvals for the RNG changes to at least be able start looking at stuff that are less likely to just be reliance on RNG streams in test.

@nanosoldier runtests(["AMRVW", "AdvancedHMC", "ApproxFun", "ApproxFunFourier", "ApproxFunRational", "ApproxFunSingularities", "ArrayLayouts", "AutoPreallocation", "AxisIndices", "AxisKeys", "BangBang", "BenchmarkCI", "BlockArrays", "Bridge", "BrkgaMpIpr", "CBindingGen", "ChainRulesCore", "Chemfiles", "ClinicalTrialUtilities", "ColorVectorSpace", "ComponentArrays", "ContinuumArrays", "CorticalSpectralTemporalResponses", "CoverageCore", "CoverageTools", "CurveProximityQueries", "DataAssim", "DataStructures", "DependencyWalker", "DiffEqBayes", "DiffEqParamEstim", "DimensionalData", "DistributedArrays", "Diversity", "Documenter", "DynamicAxisWarping", "DynamicGrids", "FIGlet", "FastIOBuffers", "FillArrays", "FunctionOperators", "FunctionalCollections", "FunctionalTables", "GEMPIC", "GeometryTypes", "Gtk", "GtkReactive", "HomotopyContinuation", "IMFData", "ImageInTerminal", "ImageSegmentation", "ImageView", "InfiniteArrays", "IntervalArithmetic", "JLD2", "JSONSchema", "JuliaVariables", "KrigingModel", "LazyArrays", "LazyStack", "Lerche", "LibGEOS", "LikelihoodProfiler", "LinearMapsAA", "LiveServer", "MIToS", "MicroCoverage", "MultiplesOfPi", "Nabla", "NiLangCore", "OceanGrids", "OpenEphysLoader", "OrthogonalPolynomialsQuasi", "OscillatoryIntegrals", "Pajarito", "PassiveTracerFlows", "PkgBenchmark", "PlotUtils", "Plots", "Pluto", "Polymake", "PowerDynSolve", "PredictMD", "PredictMDFull", "QuadraticToBinary", "QuantumOpticsBase", "QuasiArrays", "RecursiveArrayTools", "RiemannHilbert", "RigidBodyDynamics", "Simulate", "SingularIntegralEquations", "SingularSpectrumAnalysis", "SolverTraces", "SparseGaussianProcesses", "StaticArrays", "StaticNumbers", "StrBase", "StrRegex", "TaylorIntegration", "TensorKitManifolds", "ThreadTools", "ThreePhasePowerModels", "TikzQTrees", "Tracker", "Turf", "TypedPolynomials", "VarStructs", "Wrangling", "XLSXasJSON", "ArchGDAL", "AugmentedGaussianProcesses", "CUDAatomics", "CancerSeqSim", "CompScienceMeshes", "Crayons", "DynamicGridsGtk", "DynamicGridsInteract", "ExprRules", "FEMBase", "FEMBasis", "Fatou", "FieldMetadata", "GtkUtilities", "HybridArrays", "Immerse", "InspectDR", "KnetOnnx", "LegibleLambdas", "LocalRegistry", "MCHammer", "MLStyle", "Manifolds", "Manopt", "Maxvol", "Miletus", "NTFk", "NTNk", "NarrativeTest", "OhMyQSIM", "Persa", "PhysicsInformedML", "Poltergeist", "ProfileView", "RevealedPreferences", "RoMEPlotting", "SimpleGF2", "StructuralInheritance", "TrajectoryOptimization", "TuringModels", "TwoFAST", "VerTeX", "VideoIO", "Winston", "BloomFilters", "Hyperopt", "QuartetNetworkGoodnessFit", "ReinforcementLearningZoo", "DRIPs", "DirectGaussianSimulation", "Gridap", "ImageInpainting", "PowerModels", "StratiGraphics", "DataFlow", "EfficientGlobalOptimization", "ExtensibleUnions", "FastJet", "MbedTLS", "OptionalArgChecks"], vs=":release-1.4")

[KristofferC]

For people pushing commits to this branch (@ViralBShah), please make sure the original PR has a backport label, that you use cherry-pick -x and that you update the first post with the PR.

[ViralBShah]

I merged a PR and only later realized it went onto this branch. It would be fine to revert it and do the right thing.

However what is the right way to do this? Since this updated NEWS on master but here the update was to be made in HISTORY.

[KristofferC]

If it is a commit that wasn't merged into master then just mergint he PR into here (like what was done) makes sense, I didn't notice that was the case. Could perhaps just add a note to the top post so that everything that gets put in is accounted for.

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

[KristofferC]

There are so many PkgEval "regressions" here that I think it would be useful to merge this, and release a 1.5-beta to make it easier for the community in help testing / fixing packages.

[KristofferC]

@nanosoldier runtests(["AMRVW", "AdvancedHMC", "ApproxFun", "ApproxFunFourier", "ApproxFunRational", "ApproxFunSingularities", "ArchGDAL", "ArrayLayouts", "AugmentedGaussianProcesses", "AutoPreallocation", "AxisIndices", "AxisKeys", "BangBang", "BenchmarkCI", "BlockArrays", "BloomFilters", "Bridge", "CBindingGen", "CUDAatomics", "CancerSeqSim", "ChainRulesCore", "Chemfiles", "ClinicalTrialUtilities", "ColorVectorSpace", "CompScienceMeshes", "ComponentArrays", "ContinuumArrays", "CoverageCore", "CoverageTools", "CurveProximityQueries", "DRIPs", "DataAssim", "DataFlow", "DimensionalData", "DirectGaussianSimulation", "DistributedArrays", "Diversity", "Documenter", "DynamicGrids", "DynamicGridsGtk", "DynamicGridsInteract", "EfficientGlobalOptimization", "ExprRules", "ExtensibleUnions", "FEMBase", "FEMBasis", "FIGlet", "FastIOBuffers", "FastJet", "FieldMetadata", "FillArrays", "FunctionalCollections", "FunctionalTables", "GeometryTypes", "Gtk", "GtkReactive", "GtkUtilities", "HomotopyContinuation", "HybridArrays", "ImageInpainting", "ImageView", "Immerse", "InfiniteArrays", "InspectDR", "IntervalArithmetic", "JLD2", "JSONSchema", "JuliaVariables", "KnetOnnx", "KrigingModel", "LazyArrays", "LazyStack", "LegibleLambdas", "Lerche", "LibGEOS", "LinearMapsAA", "LocalRegistry", "MCHammer", "MIToS", "MLStyle", "Manifolds", "Manopt", "Maxvol", "MbedTLS", "MicroCoverage", "Miletus", "MultiplesOfPi", "NTFk", "NTNk", "Nabla", "NarrativeTest", "NiLangCore", "OceanGrids", "OhMyQSIM", "OpenEphysLoader", "OptionalArgChecks", "OrthogonalPolynomialsQuasi", "OscillatoryIntegrals", "Pajarito", "PassiveTracerFlows", "Persa", "PhysicsInformedML", "PkgBenchmark", "PlotUtils", "Plots", "Poltergeist", "Polymake", "PowerDynSolve", "PredictMD", "PredictMDFull", "ProfileView", "QuantumOpticsBase", "QuartetNetworkGoodnessFit", "QuasiArrays", "RecursiveArrayTools", "RevealedPreferences", "RiemannHilbert", "RigidBodyDynamics", "RoMEPlotting", "SimpleGF2", "SingularIntegralEquations", "SingularSpectrumAnalysis", "StaticArrays", "StaticNumbers", "StrBase", "StrRegex", "StratiGraphics", "StructuralInheritance", "TaylorIntegration", "TensorKitManifolds", "ThreadTools", "ThreePhasePowerModels", "TikzQTrees", "TrajectoryOptimization", "Turf", "TuringModels", "TwoFAST", "VarStructs", "VerTeX", "Winston", "Wrangling", "XLSXasJSON"], vs = ":release-1.4")

[nanosoldier]

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt