Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports for 1.6-RC2 #39614

Merged
merged 54 commits into from
Mar 10, 2021
Merged

Backports for 1.6-RC2 #39614

merged 54 commits into from
Mar 10, 2021

Conversation

KristofferC
Copy link
Sponsor Member

@KristofferC KristofferC commented Feb 11, 2021

Backported PRs:

Non-merged PRs with backport label:

Volker-Weissmann and others added 10 commits February 11, 2021 12:57
* improve first/last docstrings

improve `first`/`last` docstrings to indicate that two new methods have been added since v1.6 (#34868)

* Update abstractarray.jl

remove the trailing whitespaces

(cherry picked from commit bc2b854)
Because we now store these strings in `libjulia-internal`, we need to
call `stringreplace` on that library instead of `libjulia`.

(cherry picked from commit 4c5d62b)
…th certain shifted axes (#39522)

(cherry picked from commit 2155772)
Locating a package with known uuid means the uuid was either
found in a manifest in LOAD_PATH or a "package folder" in LOAD_PATH.
However, looking up the package entrypoint fails for stdlibs that
are not in the sysimage, even if they have been explicitly added
to project/manifest by the user. This patch allows looking in
Sys.STDLIB after stdlibs, fixes #39504.

(cherry picked from commit 5d7e13f)
@KristofferC
Copy link
Sponsor Member Author

Weird error on freebsd and aarch64:

est Failed at /buildworker/worker/tester_linuxaarch64/build/share/julia/test/precompile.jl:303
  Expression: Dict(modules) == merge(Dict((let m = Base.PkgId(s)
                m => Base.module_build_id(Base.root_module(m))
            end for s = ["Base", "Core", "Main", string(Foo2_module), string(FooBase_module)])), Dict((let m = Base.root_module(Base, s)
                Base.PkgId(m) => Base.module_build_id(m)
            end for s = [:ArgTools, :Artifacts, :Base64, :CRC32c, :Dates, :DelimitedFiles, :Distributed, :Downloads, :FileWatching, :Future, :InteractiveUtils, :LazyArtifacts, :LibCURL, :LibCURL_jll, :LibGit2, :Libdl, :LinearAlgebra, :Logging, :Markdown, :Mmap, :MozillaCACerts_jll, :NetworkOptions, :Pkg, :Printf, :Profile, :REPL, :Random, :SHA, :Serialization, :SharedArrays, :Sockets, :SparseArrays, :Statistics, :SuiteSparse, :TOML, :Tar, :Test, :UUIDs, :Unicode, :nghttp2_jll])))
   Evaluated: Dict{Base.PkgId, UInt64}(SparseArrays [2f01184e-e22b-5df5-ae63-d93ebab69eaf] => 0x0019aa864dc5cd0d, Base64 [2a0f44e3-6c83-55bd-87e4-b1978d98bd5f] => 0x0019aa8130b4e63e, Random [9a3f8284-a2c9-5f02-9a11-845980a1fd5c] => 0x0019aa848e2207b5, Libdl [8f399da3-3557-5675-b5ff-fb832c97cbdb] => 0x0019aa814155aed6, SuiteSparse [4607b0f0-06f3-5cda-b6b1-a6196a1729e9] => 0x0019aa890aa92039, FileWatching [7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee] => 0x0019aa813be86bc5, LibCURL_jll [deac9b47-8bc7-5906-a0fe-35ac56dc84c0] => 0x0019aa8955d7588d, REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb] => 0x0019aa8748038dfb, Future [9fa8497b-333b-5362-9e8d-4d0656e87820] => 0x0019aa85a8a759d6, Markdown [d6f4376e-aef5-505a-96c1-9c027394607a] => 0x0019aa842235211c���) == Dict{Base.PkgId, UInt64}(Random [9a3f8284-a2c9-5f02-9a11-845980a1fd5c] => 0x0019aa848e2207b5, Libdl [8f399da3-3557-5675-b5ff-fb832c97cbdb] => 0x0019aa814155aed6, SuiteSparse [4607b0f0-06f3-5cda-b6b1-a6196a1729e9] => 0x0019aa890aa92039, LibCURL_jll [deac9b47-8bc7-5906-a0fe-35ac56dc84c0] => 0x0019aa8955d7588d, REPL [3fa0cd96-eef1-5676-8a61-b3b8758bbffb] => 0x0019aa8748038dfb, Distributed [8ba89e20-285c-5b6f-9357-94700520ee1b] => 0x0019aa856c765753, LibGit2 [76f85450-5226-5b5a-8eaa-529ad045b433] => 0x0019aa85dbb3a6a8, DelimitedFiles [8bb1440f-4735-579b-a4ab-409b98df4dab] => 0x0019aa818c1e722d, Test [8dfed614-e22c-5e08-85e1-65c5234f0b40] => 0x0019aa8947716f74, SHA [ea8e919c-243c-51af-8825-aaa63cd721ce] => 0x0019aa814e2b1652���)
ERROR: LoadError: Test run finished with errors

@fredrikekre
Copy link
Member

That error happens on #39616 too

Add p7zip_jll to precompile tests since Pkg depends on it now.
@fredrikekre
Copy link
Member

fredrikekre commented Feb 11, 2021

Weird error on freebsd and aarch64:

Fixed by 352345c (which I squashed into the Pkg.jl bump commit in this PR).

@fingolfin
Copy link
Contributor

Could #39227 also be backported? It's just a C compiler warning, but I think there is some value in trying to be warning free.

@tknopp
Copy link
Contributor

tknopp commented Feb 14, 2021

Xref JuliaGraphics/Gtk.jl#562 since Gtk.jl does not pass tests on Julia 1.6.

@sostock
Copy link
Contributor

sostock commented Feb 16, 2021

Could #39671 be backported?

@KristofferC
Copy link
Sponsor Member Author

Xref JuliaGraphics/Gtk.jl#562 since Gtk.jl does not pass tests on Julia 1.6.

Need something to show that this is an issue with Julia and not the package. A bisection would be a good way to progress on that.

@tknopp
Copy link
Contributor

tknopp commented Feb 16, 2021

Need something to show that this is an issue with Julia and not the package. A bisection would be a good way to progress on that.

yes that would be helpful. My ping was not intended to say "there is a problem in Julia, fix that" but more giving you an extra data point of what issues are right now on Julia 1.6 in the package landscape.

I will not find time to track this down in the near future. If Jameson (the original author of the code in question) finds time this would be of course cool. He is also the one who can judge best if the issue is in Julia or if the issue is some assumptions that the package made, which are simply not valid anymore (but were never intended to be valid).

aviatesk and others added 10 commits February 16, 2021 12:10
All of them are detected by JET.jl's self-profiling.
The following code will print type-instabilities/type-errors for all
code paths reachable from `typeinf(::NativeInterpreter, ::InferenceState)`.
```julia
julia> using JET
julia> report_call(Core.Compiler.typeinf,
(Core.Compiler.NativeInterpreter, Core.Compiler.InferenceState);
annotate_types = true)
```

The remaining error reports (e.g. `variable Core.Compiler.string is not
defined`) are because of missing functionality on error paths.

(cherry picked from part of commit 1bc7f43)
This only would matter if we transition a call from T -> Union{}, which
is unlikely to happen, and even less likely to be observed, but it is
more correct this way.

(cherry picked from commit e67a3fd)
Previously, we might accidentally leave behind content in the fields
that should not be there. For example:

```
julia> code_typed(() -> (TypeVar(:x),), (), optimize=false)
1-element Vector{Any}:
 CodeInfo(
    @ REPL[1]:1 within `#3'
1 ─ %1 = Main.TypeVar(:x)::Core.Compiler.PartialTypeVar(x, true, true)
│   %2 = Core.tuple(%1)::Core.PartialStruct(Tuple{TypeVar}, Any[Core.Compiler.PartialTypeVar(x, true, true)])
└──      return %2
) => Tuple{TypeVar}

julia> ans[1][1].rettype
Core.PartialStruct(Tuple{TypeVar}, Any[Core.Compiler.PartialTypeVar(x, true, true)])
```

(cherry picked from commit 7c8114c)
Applications typically shouldn't use this function in
performance sensitive places, as it hints that their design is flawed,
but might as well make it faster anyways.

and optimize invokelatest kwcall too, while we are at it

(cherry picked from commit 2ff110b)
We can emulate this deprecated function, until we delete it in v2.
This in turn revealed some missing error checks, which we also add.

Fixes #39113

(cherry picked from commit 59eb9f9)
- unused jl_iterate_func
- cast type of realloc in jl_init_threading

(cherry picked from commit 83bee67)
@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runtests(["AlphaStableDistributions", "Alpine", "ArrayInterface", "AtlasRobot", "BundleMethod", "ControlSystemIdentification", "DataDrivenDiffEq", "DiffEqCallbacks", "DiffEqJump", "EliminateGraphs", "ExponentialAction", "FilesystemDatastructures", "GalacticOptim", "Gen", "GenParticleFilters", "Grassmann", "HybridArrays", "Jaynes", "KernelFunctions", "Kinetic", "KissMCMC", "KitML", "Leibniz", "LoopThrottle", "MechanismGeometries", "MeshCatMechanisms", "ModelingToolkit", "ParticleScattering", "PastaQ", "PointPatterns", "QML", "Quadrature", "RandomizedProgressiveHedging", "RigidBodyDynamics", "Ripserer", "SimplePosetAlgorithms", "Solaris", "SpatialJackknife", "ThreadedSparseArrays", "ValkyrieRobot", "VertexSafeGraphs"], vs = ":release-1.6")

@giordano
Copy link
Contributor

giordano commented Feb 24, 2021

A bisection would be a good way to progress on that.

#39811 bisected to #38180, which is in the release-1.6 branch (but I don't know whether the package was doing something unsafe which was accidentally brought up by that change)

kimikage and others added 9 commits March 2, 2021 08:20
…ystems (#39772)

This backports libgit2 patch to fix zlib stream handling.

(cherry picked from commit 503c63b)
* make copy correctly handle 0-dimensional SubArray

The current definition of `copy` for `SubArray` [here](https://github.com/JuliaLang/julia/blob/master/base/subarray.jl#L70) has the following consequence:
```
julia> x = [1]
1-element Array{Int64,1}:
 1

julia> y = @view x[1]
0-dimensional view(::Array{Int64,1}, 1) with eltype Int64:
1

julia> copy(y)
1
```

which is inconsistent with the contract for `copy` that promises to produce an array when array is copied, e.g.:
```
julia> x = fill(1)
0-dimensional Array{Int64,0}:
1

julia> copy(x)
0-dimensional Array{Int64,0}:
1
```

(cherry picked from commit a12d0ff)
This fixes the following:

```
julia> struct Symmetric{T,S<:AbstractMatrix{<:T}} <: AbstractMatrix{T}
           data::S
           uplo::Char
       end

julia> struct Symmetric{T,S<:AbstractMatrix{<:T}} <: AbstractMatrix{T}
           data::S
           uplo::Char
       end
ERROR: invalid redefinition of constant Symmetric
```

`Core._equiv_typedef` returns false which ends up triggering the error.

(cherry picked from commit 6968e47)
$ git log --pretty=oneline --abbrev=commit 2b4bed9..6bb8306
6bb83068bd796c4890baaeb39628ff79a4979374 Stop the grace timer iff adding first handle (fix #99) (#102)
af6864d8872247faf2a402d6b2baca5cb74ab96e fix ssh_key_pass bug (fix #91) (#100)

(cherry picked from commit fb500b0)
… the number of threads (#39854)

(cherry picked from commit c79309b)
When calling `make_interactive_report()`, we really need to pass in
`ARGS` as well, otherwise it always starts an interactive session.

This commit, in combination with `--` allows the user to run an `rr`
session while still passing arguments to the child `julia` process via:

```
julia --bug-report=rr -- --project test/runtests.jl
```

(cherry picked from commit 5e7aaa6)
@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runbenchmarks(ALL, vs = ":release-1.5")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @christopher-dG

@KristofferC

This comment has been minimized.

@KristofferC
Copy link
Sponsor Member Author

The various memory regressions for the Dict benchmarks are due to changes in the RNG. In 1.5, an integer < 256 was generated for these and these are special because they come pre-boxed in Julia and don't need an allocation. In 1.6, the integer is instead > 256 and thus an allocation is needed.

@nanosoldier

This comment has been minimized.

@KristofferC
Copy link
Sponsor Member Author

@nanosoldier runtests(["AlphaStableDistributions", "Alpine", "ArrayInterface", "AtlasRobot", "BundleMethod", "ControlSystemIdentification", "DataDrivenDiffEq", "DiffEqCallbacks", "DiffEqJump", "EliminateGraphs", "ExponentialAction", "FilesystemDatastructures", "GalacticOptim", "Gen", "GenParticleFilters", "Grassmann", "HybridArrays", "Jaynes", "KernelFunctions", "Kinetic", "KissMCMC", "KitML", "Leibniz", "LoopThrottle", "MechanismGeometries", "MeshCatMechanisms", "ModelingToolkit", "ParticleScattering", "PastaQ", "PointPatterns", "QML", "Quadrature", "RandomizedProgressiveHedging", "RigidBodyDynamics", "Ripserer", "SimplePosetAlgorithms", "Solaris", "SpatialJackknife", "ThreadedSparseArrays", "ValkyrieRobot", "VertexSafeGraphs"], vs = ":release-1.6")

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here. cc @maleadt

Turns out libunwind was basically ignoring value locations on the
platform, so that needed to be enabled. Should hopefully fix linux32 CI.

(cherry picked from commit da9e12e)
@KristofferC KristofferC merged commit fe2311a into release-1.6 Mar 10, 2021
@KristofferC KristofferC deleted the backports-release-1.6 branch March 10, 2021 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.