Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

relax assertion involving pg->nold to reflect that it may be a bit in… #50466

Merged
merged 4 commits into from
Jul 10, 2023

Conversation

d-netto
Copy link
Member

@d-netto d-netto commented Jul 7, 2023

…accurate with parallel marking

Should fix #50419.

@d-netto d-netto requested review from gbaraldi and vtjnash July 7, 2023 21:54
@d-netto d-netto added GC Garbage collector backport 1.10 Change should be backported to the 1.10 release labels Jul 7, 2023
@gbaraldi
Copy link
Member

gbaraldi commented Jul 7, 2023

If we're fine with losing a bit of precision due to data races we can probably relax the atomic here to separate loads and stores.

@gbaraldi
Copy link
Member

gbaraldi commented Jul 8, 2023

What I'm saying is that we can still use that information to skip a sweep even when multithreaded, so the mark_threads == 0 should only potentially guard the assertion.

@d-netto
Copy link
Member Author

d-netto commented Jul 8, 2023

I believe this could lead us to incorrectly skip the page.

As an example, we may have a case where the racy increment would cause nold to be exactly prev_nold, while a non-racy update would lead nold to be less than prev_nold.

In this case, the racy update will lead us to take an incorrect sweeping decision.

@gbaraldi
Copy link
Member

gbaraldi commented Jul 8, 2023

Fair enough, LGTM

src/gc.c Outdated Show resolved Hide resolved
@gbaraldi gbaraldi added the merge me PR is reviewed. Merge when all tests are passing label Jul 9, 2023
@IanButterworth IanButterworth merged commit 8e877cb into master Jul 10, 2023
2 checks passed
@IanButterworth IanButterworth deleted the dcn/fix-50419 branch July 10, 2023 00:40
@oscardssmith oscardssmith removed the merge me PR is reviewed. Merge when all tests are passing label Jul 10, 2023
KristofferC pushed a commit that referenced this pull request Jul 11, 2023
…accurate with parallel marking (#50466)

(cherry picked from commit 8e877cb)
KristofferC pushed a commit that referenced this pull request Jul 11, 2023
…accurate with parallel marking (#50466)

(cherry picked from commit 8e877cb)
@vtjnash
Copy link
Sponsor Member

vtjnash commented Jul 11, 2023

prev_nold is computed exactly (by the previous sweep), so the race-y update could be fine if done using atomics

jl_atomic_store_relaxed((_Atomic(uint16_t)*)&page->nold, 
    jl_atomic_load_relaxed((_Atomic(uint16_t)*)&page->nold + 1); // a guaranteed undercount of `nold`

@gbaraldi
Copy link
Member

I guess we might sweep more than necessary in this case then, which is fine, but still less than with this pr where we always sweep?

@d-netto
Copy link
Member Author

d-netto commented Jul 12, 2023

I believe this would be incorrect since a race on setting the mark-bits in gc_try_setmark_tag doesn't imply a race when incrementing page->nold in gc_setmark_pool_.

Two threads may end up marking the same object twice, but actually calling gc_setmark_pool_ at different times, in which case nold would be larger than it should be even with the use of atomics.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Jul 12, 2023

Ah, right, you might have still needed atomic_fetch_add_relaxed, so that it was at least a guaranteed overcount instead.

@gbaraldi
Copy link
Member

This causes quite a big regression in some cases. So it seems it's not worth it .

gbaraldi added a commit that referenced this pull request Jul 17, 2023
…a bit inaccurate with parallel marking (#50466)"

This reverts commit 8e877cb.
d-netto added a commit that referenced this pull request Jul 17, 2023
…a bit inaccurate with parallel marking (#50466)"

This reverts commit 8e877cb.
d-netto added a commit that referenced this pull request Jul 18, 2023
…a bit inaccurate with parallel marking (#50466)"

This reverts commit 8e877cb.
KristofferC added a commit that referenced this pull request Jul 24, 2023
Backported PRs:
- [x] #50411 <!-- Fix weird dispatch of * with zero arguments -->
- [x] #50202 <!-- Remove dynamic dispatch from _wait/wait2 -->
- [x] #50064 <!-- Fix numbered prompt with input only with comment -->
- [x] #50026 <!-- Store heapsnapshot files in tempdir() instead of
current directory -->
- [x] #50402 <!-- Add CPU feature helper function -->
- [x] #50387 <!-- update newpages pointer after actually sweeping pages
-->
- [x] #50424 <!-- avoid potential type-instability in _replace_(str,
...) -->
- [x] #50444 <!-- Optimize getfield lowering to avoid boxing in some
cases -->
- [x] #50474 <!-- docs: Fix a `!!! note` which was miscapitalized -->
- [x] #50466 <!-- relax assertion involving pg->nold to reflect that it
may be a bit in… -->
- [x] #50490 <!-- Fix compat annotation for italic printstyled -->
- [x] #50488 <!-- fix typo in `Base.isassigned` with `Tridiagonal` -->
- [x] #50476 <!-- Profile: Add specifying dir for `take_heap_snapshot`
and handling if current dir is unwritable -->
- [x] #50461 <!-- fix typo in the --gcthreads argument description -->
- [x] #50528 <!-- ssair: Correctly handle stmt insertion at end of basic
block -->
- [x] #50533 <!-- ensure internal_obj_base_ptr checks whether objects
past freelist pointer are in freelist -->
- [x] #49322 <!-- improve cat design / performance -->
- [x] #50540 <!-- gc: remove over-eager assertion -->
- [x] #50542 <!-- gf: remove unnecessary assert cycle==depth -->
- [x] #50559 <!-- Expand kwcall lowering positional default check to
vararg -->
- [x] #50058 <!-- Add unwrapping mechanism for triangular mul and solves
-->
- [x] #50551 <!-- typeintersect: also record chained `innervars` -->
- [x] #50552 <!-- read(io, Char): fix read with too many leading ones
-->
- [x] #50541 <!-- precompile: ensure globals are not accidentally
created where disallowed -->
- [x] #50576 <!-- use atomic compare exchange when setting the GC
mark-bit -->
- [x] #50578 <!-- gf: make method overwrite/delete an error during
precompile -->
- [x] #50516 <!-- Fix visibility of assert on GCC12/13 -->
- [x] #50597 <!-- Fix memory corruption if task is launched inside
finalizer -->
- [x] #50591 <!-- build: fix various makefile bugs -->
- [x] #50599 <!-- faster invalid object lookup in conservative gc -->
- [x] #50634 <!-- 🤖 [master] Bump the SparseArrays stdlib from b4b0e72
to 99c99b4 -->
- [x] #50639 <!-- Backport LLVM patches to fix various issues. -->
- [x] #50546 <!-- Revert storage of method instance in LineInfoNode -->
- [x] #50631 <!-- Shift DCE pass to optimize imaging mode code better
-->
- [x] #50525 <!-- only check that values are finite in `generic_lufact`
when `check=true` -->
- [x] #50587 <!-- isassigned for ranges with BigInt indices -->
- [x] #50144 <!-- Page based heap size heuristics -->


Need manual backport:
- [ ] #50595 <!-- Rename ENV variable `JULIA_USE_NEW_PARSER` ->
`JULIA_USE_FLISP_PARSER` -->



Non-merged PRs with backport label:
- [ ] #50637 <!-- Remove SparseArrays legacy code -->
- [ ] #50618 <!-- inference: continue const-prop' when concrete-eval
returns non-inlineable -->
- [ ] #50598 <!-- only limit types in stack traces in the REPL -->
- [ ] #50594 <!-- Disallow non-index Integer types in isassigned -->
- [ ] #50568 <!-- `Array(::AbstractRange)` should return an `Array` -->
- [ ] #50523 <!-- Avoid generic call in most cases for getproperty -->
- [ ] #50172 <!-- print feature flags used for matching pkgimage -->
@KristofferC KristofferC removed the backport 1.10 Change should be backported to the 1.10 release label Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GC Garbage collector
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gc_sweep_page assertion triggering in tests
6 participants