Separate foreign items in HIR

[nikomatsakis]

PR #37660 separates impl items from impls in the HIR for improved interaction with incremental compilation. It probably makes sense to separate foreign items in this way too -- not so much for improved incremental compilation precision, as because it makes the layout more analogous.

[nikomatsakis]

Marking this as E-mentor. I'll give a short write-up here; happy to provide a more detailed one too upon request!

The work that needs to be done is basically to model foreign items on impl items -- we would get a hir::ForeignItemRef like hir::ImplItemRef, a separate map in the Krate to store them, and modify the lowering routines analogously. Finally, there are a few places (marked with FIXMEs attached to this issue number) where we could convert to using a ItemLikeVisitor if foreign items were considered "item like".

[eddyb]

Something I've wanted to do for a while is just remove the pointless "foreign mod" nonsense in HIR.
We just need to stick the attributes from the extern {...} somewhere at the crate level or something.

[aochagavia]

For someone who has never worked on anything hir-related... Could you give examples of Rust code that compiles down to:

• An ImplItem?
• A ForeignItem?

[petrochenkov]

@aochagavia

impl Trait for Type {
    fn f() {} // <-- ImplItem
}
impl Type {
    fn g() {} // <-- ImplItem
}
extern "C" {
    fn printf(); // <-- ForeignItem
}

[aochagavia]

So an ImplItem is any item that appears within the block of an impl? And a ForeignItem is any item that appears inside an extern block? What about extern "C" fn foo() { }?

[petrochenkov]

So an ImplItem is any item that appears within the block of an impl? And a ForeignItem is any item that appears inside an extern block?

Yes.

What about extern "C" fn foo() { }?

It's just a Fn Item using non-default ABI.

[aochagavia]

Thanks!

Now some more questions, about what @eddyb said:

1. Could you give examples of Rust code that are represented as a ForeignMod? Is my interpretation below correct?

extern "C" { // <-- ForeignMod?
    fn printf(); // <-- ForeignItem
}

2. What needs to be done in order to get rid of the "pointless foreign mod nonsense"? Would you like ForeignMod to be removed? What should happen with the data then?

[petrochenkov]

Is my interpretation below correct?

Yes.

What needs to be done in order to get rid of the "pointless foreign mod nonsense"?

Suppose we have a foreign module like this:

#[link(name = "qwerty")]
extern "C" {
    fn f();
}

, then foreign items (f) need to be transplanted from the foreign module into the normal non-foreign parent module, ABI ("C") moved from the foreign module to foreign items, and attributes like #[link(name = "qwerty")] moved... somewhere.

[aochagavia]

Does that mean that ForeignMod wouldn't be needed anymore? Or are there other types of ForeignMod?

[petrochenkov]

Yeah, the whole point of this exercise is to get rid of ForeignMod (in HIR).

[eddyb]

One question is whether we might want to have a bunch of imports that share some common information - AFAIK all the attributes on an extern "ABI" {...} don't affect the imports inside but rather the crate.

cc @retep998

[retep998]

#[link] affects each individual import in terms of whether to apply dllimport. #[cfg] obviously affects whether the imports even exist. I can't think of any other attributes that can be applied to an extern "ABI" {...}.

[eddyb]

@retep998 Is the dllimport thing something we could have as a flag on each "foreign item"?
#[cfg] is already gone by the time of the HIR lowering so that's not an issue.

Ugh I guess lint scoping is a thing. I just don't want ForeignMod and ForeignItem to exist 😞.

[retep998]

@eddyb Each foreign item needs to know which library it comes from so that when rustc is informed of what type of library it is (static and static-nobundle vs dylib) it can emit dllimport properly. You can ask @vadimcn for more info on that.

[vadimcn]

These associations are extracted pretty early on and stored in CStore (indexed by DefId), so I don't think there's any dependency on hir::ItemForeignMod for this feature.

[eddyb]

@retep998 @vadimcn I'm not sure it's feasible to lift "foreign items" to the surrounding scope, even if just because of the lint scoping issue, i.e. #[allow(x)] extern {...} applies to everything inside.

If we had a way of cheaply duplicating such attributes... maybe, but still. It's annoying because they're almost regular items, and having them separate from all the other items is complexity I'd like to get rid of.

[cramertj]

@aochagavia Are you working on this? If not, I'd like to try it.

[eddyb]

@nikomatsakis Did we even come to an consensus on this? It'd be nice not to have "foreign items".

[aochagavia]

@cramertj Go ahead! I never figured out what exactly needed to happen, so I didn't even start working on it.

[cramertj]

@aochagavia If I understand @eddyb's comment correctly, it doesn't sound like there's been a decision as to what should be done.

[nikomatsakis]

@eddyb to be perfectly honest, I've not been following the discussion that closely. I just read over it. I agree with your intuition that it would be nice to lift them, but there seem to be some practical hurdles in the way:

• First, that there is still a concept of a "foreign item" that these items are embedded within; I believe that is relevant to linking both on Windows (dllimport) and Mac OS/X (frameworks), but I'm no expert
• Second, that we would want to respect lint scoping.

It seems to me that my original plan -- to keep the ForeignMod in the HIR but make foreign items into "item-like" things -- would have much of the same benefit. If you use visit_all_item_likes_in_hir, then you can ignore the existence of ForeignMod, and if you care, it's still there.

Ultimately it seems like ForeignMod is a concept we will have to represent, just like impls and traits, whether we do it by having an item in the HIR or by having something "on the side".

[eddyb]

Can we move them to Item and effectively keep ForeignMod around as a weird sort of module?
And/or point to the NodeId of the ForeignMod from each item lifted from it, without actual parenting.

[nikomatsakis]

@eddyb

Reordering slightly:

Can we [...] point to the NodeId of the ForeignMod from each item lifted from it, without actual parenting.

I considered this, but it seems not obviously good. e.g., it means that the HIR will not reflect the DefPath hierarchy, and lints will have to work around it.

Can we move them to Item and effectively keep ForeignMod around as a weird sort of module?

This is sort of interesting-- since name resolution is already done, I guess we don't care too much. I'm still having trouble seeing the value here, I have to admit, over just making them item-like. Perhaps there is lots of code that gets simplified? I guess that compared to impl-item and trait-item, foreign-item seems like small potatoes.

[nikomatsakis]

(But maybe we can think of an approach that works for all of those things?)

[tamird]

Seems like this is not fully baked enough to be E-mentor.

[cjgillot]

Solved by #79318.