Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

correctly track freed bytes in jl_genericmemory_to_string #54331

Merged
merged 4 commits into from
May 6, 2024

Conversation

d-netto
Copy link
Member

@d-netto d-netto commented May 2, 2024

Master version of #54309.

Should fix #54275.

@d-netto d-netto added arrays [a, r, r, a, y, s] GC Garbage collector labels May 2, 2024
@d-netto d-netto requested review from vtjnash and oscardssmith May 2, 2024 14:10
src/genericmemory.c Outdated Show resolved Hide resolved
@vtjnash vtjnash added the backport 1.11 Change should be backported to release-1.11 label May 4, 2024
@vtjnash
Copy link
Member

vtjnash commented May 4, 2024

Ended up making some more major changes here in reviewing this, as it turns out we appear to have also been undercounting the scanned_bytes value and not including the amount of memory that could get returned by dropping an unsafe_wrap Array which was created with own=true. But most cases looked like they were actually computing the value correctly already, only specifically the case where it was a jl_gc_track_malloced_genericmemory object would we lose track of the nbytes.

@d-netto d-netto force-pushed the dcn-fix-54275-on-master branch from 8f992c7 to eb20906 Compare May 4, 2024 23:40
@d-netto d-netto force-pushed the dcn-fix-54275-on-master branch from eb20906 to b85c508 Compare May 5, 2024 22:25
@d-netto
Copy link
Member Author

d-netto commented May 6, 2024

Merge?

@oscardssmith
Copy link
Member

Some tests would be good, but otherwise lgtm.

@d-netto d-netto merged commit 92ccc74 into master May 6, 2024
7 checks passed
@d-netto d-netto deleted the dcn-fix-54275-on-master branch May 6, 2024 00:36
@d-netto
Copy link
Member Author

d-netto commented May 6, 2024

Followed up in #54371. (Sorry, the test should have been added in this PR).

d-netto added a commit that referenced this pull request May 6, 2024
KristofferC pushed a commit that referenced this pull request May 6, 2024
Master version of #54309.

Should fix #54275.

---------

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 92ccc74)
@KristofferC KristofferC mentioned this pull request May 6, 2024
59 tasks
KristofferC added a commit that referenced this pull request May 28, 2024
Backported PRs:
- [x] #53665 <!-- use afoldl instead of tail recursion for tuples -->
- [x] #53976 <!-- LinearAlgebra: LazyString in interpolated error
messages -->
- [x] #54005 <!-- make `view(::Memory, ::Colon)` produce a Vector -->
- [x] #54010 <!-- Overload `Base.literal_pow` for `AbstractQ` -->
- [x] #54069 <!-- Allow PrecompileTools to see MI's inferred by foreign
abstract interpreters -->
- [x] #53750 <!-- inference correctness: fields and globals can revert
to undef -->
- [x] #53984 <!-- Profile: fix heap snapshot is valid char check -->
- [x] #54102 <!-- Explicitly compute stride in unaliascopy for SubArray
-->
- [x] #54070 <!-- Fix integer overflow in `skip(s::IOBuffer,
typemax(Int64))` -->
- [x] #54013 <!-- Support case-changes to Annotated{String,Char}s -->
- [x] #53941 <!-- Fix writing of AnnotatedChars to AnnotatedIOBuffer -->
- [x] #54137 <!-- Fix typo in docs for `partialsortperm` -->
- [x] #54129 <!-- use correct size when creating output data from an
IOBuffer -->
- [x] #54153 <!-- Fixup IdSet docstring -->
- [x] #54143 <!-- Fix `make install` from tarballs -->
- [x] #54151 <!-- LinearAlgebra: Correct zero element in
`_generic_matvecmul!` for block adj/trans -->
- [x] #54213 <!-- Add `public` statement to `Base.GC` -->
- [x] #54222 <!-- Utilize correct tbaa when emitting stores of unions.
-->
- [x] #54233 <!-- set MAX_OS_WRITE on unix -->
- [x] #54255 <!-- fix `_checked_mul_dims` in the presence of 0s and
overflow. -->
- [x] #54259 <!-- Fix typo in `readuntil` -->
- [x] #54251 <!-- fix typo in gc_mark_memory8 when chunking a large
array -->
- [x] #54276 <!-- Fix solve for complex `Hermitian` with non-vanishing
imaginary part on diagonal -->
- [x] #54248 <!-- ensure package callbacks are invoked when no valid
precompile file exists for an "auto loaded" stdlib -->
- [x] #54308 <!-- Implement eval-able AnnotatedString 2-arg show -->
- [x] #54302 <!-- Specialised substring equality for annotated strs -->
- [x] #54243 <!-- prevent `package_callbacks` to run multiple time for a
single package -->
- [x] #54350 <!-- add a precompile signature to Artifacts code that is
used by JLLs -->
- [x] #54331 <!-- correctly track freed bytes in
jl_genericmemory_to_string -->
- [x] #53509 <!-- revert moving "creating packages" from Pkg.jl -->
- [x] #54335 <!-- When accessing the data pointer for an array, first
decay it to a Derived Pointer -->
- [x] #54239 <!-- Make sure `fieldcount` constant-folds for `Tuple{...}`
-->
- [x] #54288
- [x] #54067
- [x] #53715 <!-- Add read/write specialisation for IOContext{AnnIO} -->
- [x] #54289 <!-- Rework annotation ordering/optimisations -->
- [x] #53815 <!-- create phantom task for GC threads -->
- [x] #54130 <!-- inference: handle `LimitedAccuracy` in
`handle_global_assignment!` -->
- [x] #54428 <!-- Move ConsoleLogging.jl into Base -->
- [x] #54332 <!-- Revert "add unsetindex support to more copyto methods
(#51760)" -->
- [x] #53826 <!-- Make all command-line options documented in all
related files -->
- [x] #54465 <!-- typeintersect: conservative typevar subtitution during
`finish_unionall` -->
- [x] #54514 <!-- typeintersect: followup cleanup for the nothrow path
of type instantiation -->
- [x] #54499 <!-- make `@doc x` work without REPL loaded -->
- [x] #54210 <!-- attach finalizer in `mmap` to the correct object -->
- [x] #54359 <!-- Pkg REPL: cache `pkg_mode` lookup -->

Non-merged PRs with backport label:
- [ ] #54471 <!-- Actually setup jit targets when compiling
packageimages instead of targeting only one -->
- [ ] #54457 <!-- Make `String(::Memory)` copy -->
- [ ] #54323 <!-- inference: fix too conservative effects for recursive
cycles -->
- [ ] #54322 <!-- effects: add new `@consistent_overlay` macro -->
- [ ] #54191 <!-- make `AbstractPipe` public -->
- [ ] #53957 <!-- tweak how filtering is done for what packages should
be precompiled -->
- [ ] #53882 <!-- Warn about cycles in extension precompilation -->
- [ ] #53707 <!-- Make ScopedValue public -->
- [ ] #53452 <!-- RFC: allow Tuple{Union{}}, returning Union{} -->
- [ ] #53402 <!-- Add `jl_getaffinity` and `jl_setaffinity` -->
- [ ] #53286 <!-- Raise an error when using `include_dependency` with
non-existent file or directory -->
- [ ] #52694 <!-- Reinstate similar for AbstractQ for backward
compatibility -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
@KristofferC KristofferC removed the backport 1.11 Change should be backported to release-1.11 label May 28, 2024
lazarusA pushed a commit to lazarusA/julia that referenced this pull request Jul 12, 2024
…54331)

Master version of JuliaLang#54309.

Should fix JuliaLang#54275.

---------

Co-authored-by: Jameson Nash <vtjnash@gmail.com>
lazarusA pushed a commit to lazarusA/julia that referenced this pull request Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrays [a, r, r, a, y, s] GC Garbage collector
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Possible bug when constructing a string from an array of characters
4 participants