Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove restrictions on compare-exchange memory ordering. #98383

Merged
merged 2 commits into from
Jul 18, 2022
Merged

Remove restrictions on compare-exchange memory ordering. #98383

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a7434da
Remove restrictions on compare-exchange memory ordering.
m-ou-se Jun 22, 2022
a898f41
Only enable new cmpxchg memory orderings in cfg(not(bootstrap)).
m-ou-se Jun 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 35 additions & 30 deletions library/core/src/sync/atomic.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -581,8 +581,7 @@ impl AtomicBool {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on `u8`.
Expand Down Expand Up @@ -640,8 +639,7 @@ impl AtomicBool {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on `u8`.
Expand Down Expand Up @@ -905,8 +903,7 @@ impl AtomicBool {
/// Using [`Acquire`] as success ordering makes the store part of this
/// operation [`Relaxed`], and using [`Release`] makes the final successful
/// load [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`],
/// [`Acquire`] or [`Relaxed`] and must be equivalent to or weaker than the
/// success ordering.
/// [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on `u8`.
Expand Down Expand Up @@ -1265,8 +1262,7 @@ impl<T> AtomicPtr<T> {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on pointers.
Expand Down Expand Up @@ -1311,8 +1307,7 @@ impl<T> AtomicPtr<T> {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on pointers.
Expand Down Expand Up @@ -1368,8 +1363,7 @@ impl<T> AtomicPtr<T> {
/// Using [`Acquire`] as success ordering makes the store part of this
/// operation [`Relaxed`], and using [`Release`] makes the final successful
/// load [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`],
/// [`Acquire`] or [`Relaxed`] and must be equivalent to or weaker than the
/// success ordering.
/// [`Acquire`] or [`Relaxed`].
///
/// **Note:** This method is only available on platforms that support atomic
/// operations on pointers.
Expand Down Expand Up @@ -1850,8 +1844,7 @@ macro_rules! atomic_int {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note**: This method is only available on platforms that support atomic operations on
#[doc = concat!("[`", $s_int_type, "`].")]
Expand Down Expand Up @@ -1902,8 +1895,7 @@ macro_rules! atomic_int {
/// `failure` describes the required ordering for the load operation that takes place when
/// the comparison fails. Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the successful load
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The failure ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note**: This method is only available on platforms that support atomic operations on
#[doc = concat!("[`", $s_int_type, "`].")]
Expand Down Expand Up @@ -2140,8 +2132,7 @@ macro_rules! atomic_int {
///
/// Using [`Acquire`] as success ordering makes the store part
/// of this operation [`Relaxed`], and using [`Release`] makes the final successful load
/// [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`]
/// and must be equivalent to or weaker than the success ordering.
/// [`Relaxed`]. The (failed) load ordering can only be [`SeqCst`], [`Acquire`] or [`Relaxed`].
///
/// **Note**: This method is only available on platforms that support atomic operations on
#[doc = concat!("[`", $s_int_type, "`].")]
Expand Down Expand Up @@ -2658,22 +2649,29 @@ unsafe fn atomic_compare_exchange<T: Copy>(
let (val, ok) = unsafe {
match (success, failure) {
(Relaxed, Relaxed) => intrinsics::atomic_cxchg_relaxed_relaxed(dst, old, new),
//(Relaxed, Acquire) => intrinsics::atomic_cxchg_relaxed_acquire(dst, old, new),
//(Relaxed, SeqCst) => intrinsics::atomic_cxchg_relaxed_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, Acquire) => intrinsics::atomic_cxchg_relaxed_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, SeqCst) => intrinsics::atomic_cxchg_relaxed_seqcst(dst, old, new),
(Acquire, Relaxed) => intrinsics::atomic_cxchg_acquire_relaxed(dst, old, new),
(Acquire, Acquire) => intrinsics::atomic_cxchg_acquire_acquire(dst, old, new),
//(Acquire, SeqCst) => intrinsics::atomic_cxchg_acquire_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Acquire, SeqCst) => intrinsics::atomic_cxchg_acquire_seqcst(dst, old, new),
(Release, Relaxed) => intrinsics::atomic_cxchg_release_relaxed(dst, old, new),
//(Release, Acquire) => intrinsics::atomic_cxchg_release_acquire(dst, old, new),
//(Release, SeqCst) => intrinsics::atomic_cxchg_release_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Release, Acquire) => intrinsics::atomic_cxchg_release_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Release, SeqCst) => intrinsics::atomic_cxchg_release_seqcst(dst, old, new),
(AcqRel, Relaxed) => intrinsics::atomic_cxchg_acqrel_relaxed(dst, old, new),
(AcqRel, Acquire) => intrinsics::atomic_cxchg_acqrel_acquire(dst, old, new),
//(AcqRel, SeqCst) => intrinsics::atomic_cxchg_acqrel_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(AcqRel, SeqCst) => intrinsics::atomic_cxchg_acqrel_seqcst(dst, old, new),
(SeqCst, Relaxed) => intrinsics::atomic_cxchg_seqcst_relaxed(dst, old, new),
(SeqCst, Acquire) => intrinsics::atomic_cxchg_seqcst_acquire(dst, old, new),
(SeqCst, SeqCst) => intrinsics::atomic_cxchg_seqcst_seqcst(dst, old, new),
(_, AcqRel) => panic!("there is no such thing as an acquire-release failure ordering"),
(_, Release) => panic!("there is no such thing as a release failure ordering"),
#[cfg(bootstrap)]
_ => panic!("a failure ordering can't be stronger than a success ordering"),
}
};
Expand All @@ -2693,22 +2691,29 @@ unsafe fn atomic_compare_exchange_weak<T: Copy>(
let (val, ok) = unsafe {
match (success, failure) {
(Relaxed, Relaxed) => intrinsics::atomic_cxchgweak_relaxed_relaxed(dst, old, new),
//(Relaxed, Acquire) => intrinsics::atomic_cxchgweak_relaxed_acquire(dst, old, new),
//(Relaxed, SeqCst) => intrinsics::atomic_cxchgweak_relaxed_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, Acquire) => intrinsics::atomic_cxchgweak_relaxed_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Relaxed, SeqCst) => intrinsics::atomic_cxchgweak_relaxed_seqcst(dst, old, new),
(Acquire, Relaxed) => intrinsics::atomic_cxchgweak_acquire_relaxed(dst, old, new),
(Acquire, Acquire) => intrinsics::atomic_cxchgweak_acquire_acquire(dst, old, new),
//(Acquire, SeqCst) => intrinsics::atomic_cxchgweak_acquire_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Acquire, SeqCst) => intrinsics::atomic_cxchgweak_acquire_seqcst(dst, old, new),
(Release, Relaxed) => intrinsics::atomic_cxchgweak_release_relaxed(dst, old, new),
//(Release, Acquire) => intrinsics::atomic_cxchgweak_release_acquire(dst, old, new),
//(Release, SeqCst) => intrinsics::atomic_cxchgweak_release_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(Release, Acquire) => intrinsics::atomic_cxchgweak_release_acquire(dst, old, new),
#[cfg(not(bootstrap))]
(Release, SeqCst) => intrinsics::atomic_cxchgweak_release_seqcst(dst, old, new),
(AcqRel, Relaxed) => intrinsics::atomic_cxchgweak_acqrel_relaxed(dst, old, new),
(AcqRel, Acquire) => intrinsics::atomic_cxchgweak_acqrel_acquire(dst, old, new),
//(AcqRel, SeqCst) => intrinsics::atomic_cxchgweak_acqrel_seqcst(dst, old, new),
#[cfg(not(bootstrap))]
(AcqRel, SeqCst) => intrinsics::atomic_cxchgweak_acqrel_seqcst(dst, old, new),
(SeqCst, Relaxed) => intrinsics::atomic_cxchgweak_seqcst_relaxed(dst, old, new),
(SeqCst, Acquire) => intrinsics::atomic_cxchgweak_seqcst_acquire(dst, old, new),
(SeqCst, SeqCst) => intrinsics::atomic_cxchgweak_seqcst_seqcst(dst, old, new),
(_, AcqRel) => panic!("there is no such thing as an acquire-release failure ordering"),
(_, Release) => panic!("there is no such thing as a release failure ordering"),
#[cfg(bootstrap)]
_ => panic!("a failure ordering can't be stronger than a success ordering"),
}
};
Expand Down