-
Notifications
You must be signed in to change notification settings - Fork 12.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix non_blanket_impls iteration order #89016
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -304,17 +304,7 @@ pub fn provide(providers: &mut Providers) { | |
// traversal, but not globally minimal across all crates. | ||
let bfs_queue = &mut VecDeque::new(); | ||
|
||
// Preferring shortest paths alone does not guarantee a | ||
// deterministic result; so sort by crate num to avoid | ||
// hashtable iteration non-determinism. This only makes | ||
// things as deterministic as crate-nums assignment is, | ||
// which is to say, its not deterministic in general. But | ||
// we believe that libstd is consistently assigned crate | ||
// num 1, so it should be enough to resolve #46112. | ||
let mut crates: Vec<CrateNum> = (*tcx.crates(())).to_owned(); | ||
crates.sort(); | ||
|
||
for &cnum in crates.iter() { | ||
for &cnum in tcx.crates(()) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. are these changes just "cleanup" or is there some substantive change in the order in which things are done here? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. just cleanup, the only change of behavior is the second commit here.
|
||
// Ignore crates without a corresponding local `extern crate` item. | ||
if tcx.missing_extern_crate_item(cnum) { | ||
continue; | ||
|
@@ -323,35 +313,31 @@ pub fn provide(providers: &mut Providers) { | |
bfs_queue.push_back(DefId { krate: cnum, index: CRATE_DEF_INDEX }); | ||
} | ||
|
||
// (restrict scope of mutable-borrow of `visible_parent_map`) | ||
{ | ||
let visible_parent_map = &mut visible_parent_map; | ||
let mut add_child = |bfs_queue: &mut VecDeque<_>, child: &Export, parent: DefId| { | ||
if child.vis != ty::Visibility::Public { | ||
return; | ||
} | ||
let mut add_child = |bfs_queue: &mut VecDeque<_>, child: &Export, parent: DefId| { | ||
if child.vis != ty::Visibility::Public { | ||
return; | ||
} | ||
|
||
if let Some(child) = child.res.opt_def_id() { | ||
match visible_parent_map.entry(child) { | ||
Entry::Occupied(mut entry) => { | ||
// If `child` is defined in crate `cnum`, ensure | ||
// that it is mapped to a parent in `cnum`. | ||
if child.is_local() && entry.get().is_local() { | ||
entry.insert(parent); | ||
} | ||
} | ||
Entry::Vacant(entry) => { | ||
if let Some(child) = child.res.opt_def_id() { | ||
match visible_parent_map.entry(child) { | ||
Entry::Occupied(mut entry) => { | ||
// If `child` is defined in crate `cnum`, ensure | ||
// that it is mapped to a parent in `cnum`. | ||
if child.is_local() && entry.get().is_local() { | ||
entry.insert(parent); | ||
bfs_queue.push_back(child); | ||
} | ||
} | ||
Entry::Vacant(entry) => { | ||
entry.insert(parent); | ||
bfs_queue.push_back(child); | ||
} | ||
} | ||
}; | ||
} | ||
}; | ||
|
||
while let Some(def) = bfs_queue.pop_front() { | ||
for child in tcx.item_children(def).iter() { | ||
add_child(bfs_queue, child, def); | ||
} | ||
while let Some(def) = bfs_queue.pop_front() { | ||
for child in tcx.item_children(def).iter() { | ||
add_child(bfs_queue, child, def); | ||
} | ||
} | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,9 @@ | ||
use crate::ich; | ||
use crate::middle::cstore::CrateStore; | ||
use crate::ty::{fast_reject, TyCtxt}; | ||
use crate::ty::TyCtxt; | ||
|
||
use rustc_ast as ast; | ||
use rustc_data_structures::fx::{FxHashMap, FxHashSet}; | ||
use rustc_data_structures::fx::FxHashSet; | ||
use rustc_data_structures::stable_hasher::{HashStable, StableHasher}; | ||
use rustc_data_structures::sync::Lrc; | ||
use rustc_hir as hir; | ||
|
@@ -14,9 +14,6 @@ use rustc_span::source_map::SourceMap; | |
use rustc_span::symbol::Symbol; | ||
use rustc_span::{BytePos, CachingSourceMapView, SourceFile, Span, SpanData}; | ||
|
||
use smallvec::SmallVec; | ||
use std::cmp::Ord; | ||
|
||
fn compute_ignored_attr_names() -> FxHashSet<Symbol> { | ||
debug_assert!(!ich::IGNORED_ATTRIBUTES.is_empty()); | ||
ich::IGNORED_ATTRIBUTES.iter().copied().collect() | ||
|
@@ -241,39 +238,3 @@ impl<'a> rustc_span::HashStableContext for StableHashingContext<'a> { | |
} | ||
|
||
impl rustc_session::HashStableContext for StableHashingContext<'a> {} | ||
|
||
pub fn hash_stable_trait_impls<'a>( | ||
hcx: &mut StableHashingContext<'a>, | ||
hasher: &mut StableHasher, | ||
blanket_impls: &[DefId], | ||
non_blanket_impls: &FxHashMap<fast_reject::SimplifiedType, Vec<DefId>>, | ||
) { | ||
{ | ||
let mut blanket_impls: SmallVec<[_; 8]> = | ||
blanket_impls.iter().map(|&def_id| hcx.def_path_hash(def_id)).collect(); | ||
|
||
if blanket_impls.len() > 1 { | ||
blanket_impls.sort_unstable(); | ||
} | ||
|
||
blanket_impls.hash_stable(hcx, hasher); | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This actually looks like it is unsound, right @Aaron1011 & @cjgillot? If the order is observable, we must not "stabilize" it via sorting. |
||
|
||
{ | ||
let mut keys: SmallVec<[_; 8]> = | ||
non_blanket_impls.keys().map(|k| (k, k.map_def(|d| hcx.def_path_hash(d)))).collect(); | ||
keys.sort_unstable_by(|&(_, ref k1), &(_, ref k2)| k1.cmp(k2)); | ||
keys.len().hash_stable(hcx, hasher); | ||
for (key, ref stable_key) in keys { | ||
stable_key.hash_stable(hcx, hasher); | ||
let mut impls: SmallVec<[_; 8]> = | ||
non_blanket_impls[key].iter().map(|&impl_id| hcx.def_path_hash(impl_id)).collect(); | ||
|
||
if impls.len() > 1 { | ||
impls.sort_unstable(); | ||
} | ||
|
||
impls.hash_stable(hcx, hasher); | ||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,3 @@ | ||
use crate::ich::{self, StableHashingContext}; | ||
use crate::traits::specialization_graph; | ||
use crate::ty::fast_reject; | ||
use crate::ty::fold::TypeFoldable; | ||
|
@@ -7,8 +6,7 @@ use rustc_hir as hir; | |
use rustc_hir::def_id::DefId; | ||
use rustc_hir::definitions::DefPathHash; | ||
|
||
use rustc_data_structures::fx::FxHashMap; | ||
use rustc_data_structures::stable_hasher::{HashStable, StableHasher}; | ||
use rustc_data_structures::fx::FxIndexMap; | ||
use rustc_errors::ErrorReported; | ||
use rustc_macros::HashStable; | ||
|
||
|
@@ -66,11 +64,11 @@ pub enum TraitSpecializationKind { | |
AlwaysApplicable, | ||
} | ||
|
||
#[derive(Default, Debug)] | ||
#[derive(Default, Debug, HashStable)] | ||
pub struct TraitImpls { | ||
blanket_impls: Vec<DefId>, | ||
/// Impls indexed by their simplified self type, for fast lookup. | ||
non_blanket_impls: FxHashMap<fast_reject::SimplifiedType, Vec<DefId>>, | ||
non_blanket_impls: FxIndexMap<fast_reject::SimplifiedType, Vec<DefId>>, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hm, is this enough? It looks like insertion into non_blanket_impls happens in a loop which is a result of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The way We do have A bit out of my depth here, maybe @rust-lang/wg-incr-comp can help and clarify here 😅 ❤️ There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, we probably do end up with an ordering that is mostly stable from run to run (presuming no shifts in code or other sources), and I think incremental will track the query fine. My guess is that we could avoid marking the trait query as red if the crate load order changes (e.g., because the first code using that crate is moved?) if this used a stable crate ID, but it may not be worth it. Ultimately this PR does seem like it makes things strictly better here :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that crate ordering is indeed deterministic, since it isn't something that we discover dynamically? But indeed I bet @michaelwoerister or @petrochenkov would be certain. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Crate are assigned their CrateNum in the order in which they are required by the resolver. As the resolver is not incremental, this order is deterministic but not stable. On the other hand, it is not obvious at all. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that parts of the code might rely on |
||
} | ||
|
||
impl TraitImpls { | ||
|
@@ -249,11 +247,3 @@ pub(super) fn trait_impls_of_provider(tcx: TyCtxt<'_>, trait_id: DefId) -> Trait | |
|
||
impls | ||
} | ||
|
||
impl<'a> HashStable<StableHashingContext<'a>> for TraitImpls { | ||
fn hash_stable(&self, hcx: &mut StableHashingContext<'a>, hasher: &mut StableHasher) { | ||
let TraitImpls { ref blanket_impls, ref non_blanket_impls } = *self; | ||
|
||
ich::hash_stable_trait_impls(hcx, hasher, blanket_impls, non_blanket_impls); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to skip the leading
CrateNum(0)
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤷
CrateNum(0)
is the local crate so I don't think we need to do any linking for that.I think that explicitly dealing with the local crate and only iterating over all external ones is probably better than specialcasing the first item of the
crates
iter. But that's a fairly uninformed opinionThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand how that code works in the first place?
last_crate
istcx.crates(()).len()
and then it creates aCrateNum(last_crate)
? That should not be a validCrateNum
, right?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh,
tcx.crates(())
only returns upstreamCrateNum
s. That is far from obvious. I think the query should be renamed toupstream_crate_nums
or something. The same goes forCStore::crates_untracked
. Would you be up for doing that, @lcnr? (but maybe in a separate PR)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
extern_crates
maybe? 🤔 though i prefer doing that in a separate PR, want to merge this as it is blocking some further workThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've personally always used the term "upstream" (e.g. see
rust/compiler/rustc_middle/src/query/mod.rs
Line 1264 in 15d9ba0
extern crate foo
suggests otherwise:)
.Anyway, I'm fine with doing this in another PR. You can r=me on this one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
upstream
is also fine 😆 was thinking ofextern crate something
^^