Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backports for 1.10.4 #54416

Merged
merged 13 commits into from
May 30, 2024
Merged

Backports for 1.10.4 #54416

merged 13 commits into from
May 30, 2024

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented May 8, 2024 •
edited by N5N3
Loading

Backported PRs:

Need manual backport:

Contains multiple commits, manual intervention needed:

Non-merged PRs with backport label:

@KristofferC KristofferC added the release Release management and versioning. label May 8, 2024
@IanButterworth
Copy link
Member

To get the heapsnapshot fixes in I think these are all needed

Which I tried but they're not very clean backports. I think someone familiar with GC would be better suited to handle them.

@fingolfin
Copy link
Member

It would be great to also get #53815 into 1.10.4. It requires (IMHO straightforward) manual resolution of the merge but it is easy enough, I can provide a PR if needed.

@KristofferC KristofferC force-pushed the backports-release-1.10 branch from fb457f2 to 6832cfb Compare May 25, 2024 17:36
goerz and others added 8 commits May 25, 2024 19:37
@goerz @KristofferC
Add inventory writing via DocumenterInventoryWritingBackport
b0f607a 
(cherry picked from commit 2d1442f)
@dkarrasch @KristofferC
Overload Base.literal_pow for AbstractQ (#54010)
525b3a2 
(cherry picked from commit b9aeafa)
@nalimilan @KristofferC
Fix make install from tarballs (#54143)
58cf81e 
This reverts part of 67b8ac0
(#47596 (comment)).
That change broke `make install` from tarballs due to building docs
again, which fails as there's no git repo (and also requires Internet
access to download UnicodeData.txt.

Fixes #54037.

(cherry picked from commit d6dda7c)
@jishnub @KristofferC
LinearAlgebra: Correct zero element in _generic_matvecmul! for bloc…
3577557 
…k adj/trans (#54151)

Fixes the following issue on master, where the zero element is computed
incorrectly (but subsequent terms are computed correctly):
```julia
julia> using LinearAlgebra

julia> x = [1 2 3; 4 5 6];

julia> A = reshape([x,2x,3x,4x],2,2);

julia> b = fill(x, 2);

julia> A' * b
ERROR: DimensionMismatch: matrix A has axes (Base.OneTo(2),Base.OneTo(3)), matrix B has axes (Base.OneTo(2),Base.OneTo(3))
Stacktrace:
  [1] _generic_matmatmul!(C::Matrix{Int64}, A::Matrix{Int64}, B::Matrix{Int64}, _add::LinearAlgebra.MulAddMul{true, true, Bool, Bool})
    @ LinearAlgebra ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:849
  [2] generic_matmatmul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:834 [inlined]
  [3] _mul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:287 [inlined]
  [4] mul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:285 [inlined]
  [5] mul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:253 [inlined]
  [6] *
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:114 [inlined]
  [7] _generic_matvecmul!(C::Vector{Matrix{…}}, tA::Char, A::Matrix{Matrix{…}}, B::Vector{Matrix{…}}, _add::LinearAlgebra.MulAddMul{true, true, Bool, Bool})
    @ LinearAlgebra ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:797
  [8] generic_matvecmul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:755 [inlined]
  [9] _mul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:73 [inlined]
 [10] mul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:70 [inlined]
 [11] mul!
    @ ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:253 [inlined]
 [12] *(A::Adjoint{Adjoint{Int64, Matrix{Int64}}, Matrix{Matrix{Int64}}}, x::Vector{Matrix{Int64}})
    @ LinearAlgebra ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.12/LinearAlgebra/src/matmul.jl:60
 [13] top-level scope
    @ REPL[10]:1
Some type information was truncated. Use `show(err)` to see complete types.
```
After this PR,
```julia
julia> A' * b
2-element Vector{Matrix{Int64}}:
 [51 66 81; 66 87 108; 81 108 135]
 [119 154 189; 154 203 252; 189 252 315]
```

(cherry picked from commit 2b878f0)
@vtjnash @KristofferC
set MAX_OS_WRITE on unix (#54233)
1de708b 
Fixes #54225

(cherry picked from commit b35d308)
@d-netto @KristofferC
fix typo in gc_mark_memory8 when chunking a large array (#54251)
f752249 
Should fix #54247.

- Before:
```
Julia Version 1.12.0-DEV.410
Commit 0735854* (2024-04-25 14:35 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin23.4.0)
  CPU: 12 × Apple M2 Max
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
Threads: 8 default, 0 interactive, 4 GC (on 8 virtual cores)
***** Vector *****
serial: 0.607365 seconds (3 allocations: 381.470 MiB, 0.22% gc time)
serial: 0.597241 seconds (3 allocations: 381.470 MiB, 1.18% gc time)
serial: 0.589679 seconds (3 allocations: 381.470 MiB, 1.21% gc time)
serial: 0.581607 seconds (3 allocations: 381.470 MiB, 0.97% gc time)
serial: 0.578147 seconds (3 allocations: 381.470 MiB, 0.96% gc time)
serial: 0.577399 seconds (3 allocations: 381.470 MiB, 0.94% gc time)
serial: 0.564742 seconds (3 allocations: 381.470 MiB, 0.93% gc time)
serial: 0.591733 seconds (3 allocations: 381.470 MiB, 0.90% gc time)
serial: 0.565088 seconds (3 allocations: 381.470 MiB, 0.97% gc time)
serial: 0.556800 seconds (3 allocations: 381.470 MiB, 0.94% gc time)
parallel: 0.563600 seconds (3.33 k allocations: 381.640 MiB, 0.10% gc time, 0.94% compilation time)
parallel: 0.564652 seconds (9 allocations: 381.470 MiB, 1.02% gc time)
parallel: 0.581996 seconds (9 allocations: 381.470 MiB, 0.95% gc time)
parallel: 0.576503 seconds (9 allocations: 381.470 MiB, 0.90% gc time)
parallel: 0.565595 seconds (9 allocations: 381.470 MiB, 0.94% gc time)
parallel: 0.575850 seconds (9 allocations: 381.470 MiB, 0.95% gc time)
parallel: 0.604621 seconds (9 allocations: 381.470 MiB, 0.91% gc time)
parallel: 0.578168 seconds (9 allocations: 381.470 MiB, 0.91% gc time)
parallel: 0.600408 seconds (9 allocations: 381.470 MiB, 0.92% gc time)
parallel: 0.576794 seconds (9 allocations: 381.470 MiB, 0.95% gc time)

***** Astruct *****
serial: 0.659020 seconds (3 allocations: 762.940 MiB, 0.18% gc time)
serial: 0.674630 seconds (3 allocations: 762.940 MiB, 1.63% gc time)
serial: 0.670532 seconds (3 allocations: 762.940 MiB, 1.86% gc time)
serial: 0.682483 seconds (3 allocations: 762.940 MiB, 1.57% gc time)
serial: 0.673673 seconds (3 allocations: 762.940 MiB, 1.62% gc time)
serial: 0.652506 seconds (3 allocations: 762.940 MiB, 1.72% gc time)
serial: 0.667336 seconds (3 allocations: 762.940 MiB, 1.69% gc time)
serial: 0.659686 seconds (3 allocations: 762.940 MiB, 1.76% gc time)
serial: 0.673387 seconds (3 allocations: 762.940 MiB, 1.73% gc time)
serial: 0.668930 seconds (3 allocations: 762.940 MiB, 1.67% gc time)
parallel: 0.646382 seconds (3.42 k allocations: 763.114 MiB, 0.09% gc time, 0.77% compilation time)
parallel: 0.665347 seconds (9 allocations: 762.940 MiB, 1.70% gc time)
parallel: 0.676490 seconds (9 allocations: 762.940 MiB, 1.84% gc time)
parallel: 22.732728 seconds (9 allocations: 762.940 MiB, 97.12% gc time)
parallel: 22.668091 seconds (9 allocations: 762.940 MiB, 97.15% gc time)
parallel: 0.629117 seconds (9 allocations: 762.940 MiB)
parallel: 22.948883 seconds (9 allocations: 762.940 MiB, 97.26% gc time)
parallel: 0.629834 seconds (9 allocations: 762.940 MiB)
parallel: 1.108239 seconds (9 allocations: 762.940 MiB, 42.54% gc time)
parallel: 0.648325 seconds (9 allocations: 762.940 MiB)
```

- After:
```
Julia Version 1.12.0-DEV.410
Commit 0735854* (2024-04-25 14:35 UTC)
Platform Info:
  OS: macOS (arm64-apple-darwin23.4.0)
  CPU: 12 × Apple M2 Max
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, apple-m2)
Threads: 8 default, 0 interactive, 4 GC (on 8 virtual cores)
***** Vector *****
serial: 0.600418 seconds (3 allocations: 381.470 MiB, 0.20% gc time)
serial: 0.613808 seconds (3 allocations: 381.470 MiB, 1.17% gc time)
serial: 0.596335 seconds (3 allocations: 381.470 MiB, 1.21% gc time)
serial: 0.589423 seconds (3 allocations: 381.470 MiB, 0.92% gc time)
serial: 0.621845 seconds (3 allocations: 381.470 MiB, 0.89% gc time)
serial: 0.580239 seconds (3 allocations: 381.470 MiB, 0.94% gc time)
serial: 0.604881 seconds (3 allocations: 381.470 MiB, 0.90% gc time)
serial: 0.581353 seconds (3 allocations: 381.470 MiB, 0.95% gc time)
serial: 0.603317 seconds (3 allocations: 381.470 MiB, 0.89% gc time)
serial: 0.599634 seconds (3 allocations: 381.470 MiB, 0.90% gc time)
parallel: 0.589694 seconds (3.33 k allocations: 381.640 MiB, 0.10% gc time, 1.01% compilation time)
parallel: 0.600812 seconds (9 allocations: 381.470 MiB, 0.98% gc time)
parallel: 0.610956 seconds (9 allocations: 381.470 MiB, 5.13% gc time)
parallel: 0.677189 seconds (9 allocations: 381.470 MiB, 14.26% gc time)
parallel: 0.571583 seconds (9 allocations: 381.470 MiB, 1.80% gc time)
parallel: 0.742617 seconds (9 allocations: 381.470 MiB, 17.81% gc time)
parallel: 0.624541 seconds (9 allocations: 381.470 MiB, 5.02% gc time)
parallel: 0.723963 seconds (9 allocations: 381.470 MiB, 14.03% gc time)
parallel: 0.609618 seconds (9 allocations: 381.470 MiB)
parallel: 0.708227 seconds (9 allocations: 381.470 MiB, 16.29% gc time)

***** Astruct *****
serial: 0.627853 seconds (3 allocations: 762.940 MiB)
serial: 0.647795 seconds (3 allocations: 762.940 MiB, 2.14% gc time)
serial: 0.640387 seconds (3 allocations: 762.940 MiB, 1.91% gc time)
serial: 0.698194 seconds (3 allocations: 762.940 MiB, 1.73% gc time)
serial: 0.673018 seconds (3 allocations: 762.940 MiB, 1.68% gc time)
serial: 0.641571 seconds (3 allocations: 762.940 MiB, 1.79% gc time)
serial: 0.669718 seconds (3 allocations: 762.940 MiB, 1.65% gc time)
serial: 0.661993 seconds (3 allocations: 762.940 MiB, 1.73% gc time)
serial: 0.661846 seconds (3 allocations: 762.940 MiB, 1.85% gc time)
serial: 0.661637 seconds (3 allocations: 762.940 MiB, 1.73% gc time)
parallel: 0.645750 seconds (3.42 k allocations: 763.114 MiB, 0.09% gc time, 0.83% compilation time)
parallel: 0.653331 seconds (9 allocations: 762.940 MiB, 1.92% gc time)
parallel: 0.673813 seconds (9 allocations: 762.940 MiB, 1.80% gc time)
parallel: 0.720503 seconds (9 allocations: 762.940 MiB, 9.40% gc time)
parallel: 0.889651 seconds (9 allocations: 762.940 MiB, 25.33% gc time)
parallel: 0.644618 seconds (9 allocations: 762.940 MiB, 2.26% gc time)
parallel: 0.919954 seconds (9 allocations: 762.940 MiB, 28.99% gc time)
parallel: 0.750246 seconds (9 allocations: 762.940 MiB, 9.03% gc time)
parallel: 0.767282 seconds (9 allocations: 762.940 MiB, 10.55% gc time)
parallel: 0.899596 seconds (9 allocations: 762.940 MiB, 25.69% gc time)
```

(cherry picked from commit dc0a477)
@N5N3 @KristofferC
typeintersect: fix another stack overflow caused by circular constrai…
1b21830 
…nts (#54363)

The added MWE has been broken since 1.8.
The intersect result still looks quite unsoundness, but at least stack
overflow get fixed.

close #54356

(cherry picked from commit e47fedd)
@fatteneder @KristofferC
Fix an off-by-one error in interpreter's do_invoke (#54443)
e1b2099
@KristofferC KristofferC force-pushed the backports-release-1.10 branch from 6832cfb to e1b2099 Compare May 25, 2024 17:37
NHDaly and others added 2 commits May 27, 2024 10:03
@NHDaly
Make TestLogger thread-safe (introduce a lock) (#54497)
94bef5d 
Fixes #54439.

- Lock around concurrent accesses to .logs, .message_limits, and
   .shouldlog_args.
- Copy the vector out of the logger at the end, to shield against
   dangling Tasks.

Before:
```julia
julia> Threads.nthreads()
8

julia> function foo(n)
           @info "Doing foo with n=$n"
           @sync for i=1:n
               Threads.@Spawn @info "Iteration $i"
           end
           42
       end
foo (generic function with 1 method)

julia> for _ in 1:1000
           @test_logs (:info,"Doing foo with n=10000") match_mode=:any foo(10_000)
       end
julia+RAI(56155,0x1f5157ac0) malloc: double free for ptr 0x128248000
julia+RAI(56155,0x1f5157ac0) malloc: *** set a breakpoint in malloc_error_break to debug

[56155] signal (6): Abort trap: 6
in expression starting at REPL[8]:1

signal (6) thread (1) __pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 54370881 (Pool: 54363911; Big: 6970); GC: 119
[1]    56154 abort      julia -tauto
```
After:
```julia
julia> Threads.nthreads()
8

julia> function foo(n)
           @info "Doing foo with n=$n"
           @sync for i=1:n
               Threads.@Spawn @info "Iteration $i"
           end
           42
       end
foo (generic function with 1 method)

julia> for _ in 1:1000
           @test_logs (:info,"Doing foo with n=10000") match_mode=:any foo(10_000)
       end
```
(no crash) :)

(cherry picked from commit 0437210)
@putianyi889
Add a missing doc (#53796)
7eeeb56 
(cherry picked from commit 0c65f6f)
@KristofferC
Copy link
Member Author

@N5N3, could you help backport the type intersection PRs that are marked for backport to here (some merge conflicts).

gbaraldi and others added 3 commits May 28, 2024 10:34
@gbaraldi @Keno @topolarity
Set storage class of julia globals to dllimport on windows to avoid a…
007d8de 
…uto-import weirdness (#54572)

Fixes #54550 on 1.10. Master
will require a similar but different PR.

Co-authored-by: Keno Fischer <keno@juliacomputing.com>
Co-authored-by: Cody Tapscott <84105208+topolarity@users.noreply.github.com>
@N5N3
typeintersect: conservative typevar subtitution during `finish_uniona…
c81d02c 
…ll` (#54465)

(cherry picked from commit a946631)
@N5N3
typeintersect: followup cleanup for the nothrow path of type instanti…
7ebe203 
…ation (#54514)

(cherry picked from commit af545b9)
@N5N3
Copy link
Member

N5N3 commented May 28, 2024 •
edited
Loading

@KristofferC, done.
BTW, it would be good to backport #53675 and #54545 to 1.10, if the latter get merged.
The free TypeVar regression shown in #54516 still exists.

@KristofferC
Copy link
Member Author

@nanosoldier runtests()

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

@KristofferC KristofferC merged commit a5540b3 into release-1.10 May 30, 2024
10 checks passed
@KristofferC KristofferC deleted the backports-release-1.10 branch May 30, 2024 09:07
@fingolfin
Copy link
Member

It seems #53815 did not make it in :-(. Any particular reason for that? It fixed a crash I can't otherwise work around

@KristofferC
Copy link
Member Author

#54632

@sumiya11
Copy link
Contributor

Thank you for fixing #54356 so quickly and for porting it to 1.10.4! I just tested 1.10.4 and the issue seems to be resolved.

@IanButterworth IanButterworth mentioned this pull request Sep 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
release Release management and versioning.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.