Tracking issue for stable -C emit-artifact-notifications rustc flag

[alexcrichton]

First added in #60006 the -Z emit-directives flag is currently intended to be paired with the --error-format=json flag. The emit-directives flag indicates that the compiler will emit a JSON directive for various things it does, currently only for when an rmeta metadata file is produced.

This issue is intended to track stabilizing this flag! The main use case for this is "pipelined compilation" which shows promise for accelerating Cargo's compile of multi-crate graphs.

Current known issues with this flag:

• It's unclear how Cargo will handle its own --message-format short flag with this. Cargo will want to switch to -Z emit-directives by default but --message-format short in Cargo translates to --error-format short in rustc, which means Cargo can't simultaneously get a directive JSON plus a short error format
• What should the stable form of this flag be called?
• What should the structure of the JSON directive be? Currently it only gives a path to the artifact emitted, but should the --emit kind of that artifact also be emitted?

cc @nnethercote

[ehuss]

* how Cargo will handle its own `--message-format short`

I'm working on diagnostic caching and have the same problem with the short format. I'm wondering if maybe another flag could be added to make rustc include the rendered short form as well as the human form? It seems a little clunky, so I'm not sure if this is going in the wrong direction. The only other option I can think of is to update the JSON format to have full fidelity so that any form can be generated from the base JSON structure. And then have some crate which can take that format and generate the output in any form (colored, human, short, json, etc.). I think there is a diagnostics wg that is maybe looking at moving things into different crates, though I don't really know what their plans are or what the timeframes are.

[alexcrichton]

I'd like to propose FCP for this issue for stabilization. A summary of this feature to stabilize would be:

• A new flag, -C emit-artifact-notifications is added to the compiler
• When the compiler is otherwise in JSON printing mode, this causes the compiler to print a JSON blob per artifact that it produces.
• JSON blobs have the form {"artifact":"path/to/the/artifact.rlib","emit":"link"}. The "artifact" key is the path to the file that is produced, and the "emit" key matches the --emit flag in the compiler. It's, for example, metadata for rmeta files, link for linked artifacts, etc.

There's only one other unresolved question with --error-format=short and how Cargo handles this, but I think that's best left to a different issue.

cc @rust-lang/compiler, could I poke one of y'all to propose stabilization of this feature?

[cramertj]

@rfcbot fcp merge

See above comment for summary. @alexcrichton could you also provide a link to the tests for this feature?

[rfcbot]

Team member @cramertj has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Zoxc
• @cramertj
• @eddyb
• @estebank
• @michaelwoerister
• @nagisa
• @nikomatsakis
• @oli-obk
• @petrochenkov
• @pnkfelix
• @varkor

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[eddyb]

We could also enable this by default in JSON mode without a flag, to stabilize it.

[alexcrichton]

@cramertj sure yeah, there's a simple smoke test in the repository which asserts the output of the flag, and Cargo thoroughly also excercises this in pipelined compilation so it'd break very quickly if anything went awry.

@eddyb I'd be fine with that as well! I would prefer to stabilize nothing and have Cargo simply use this internally with rustc, but that's not technically feasible right now. So long as Cargo has the option for this functionality I think that's ok.

[Zoxc]

I don't think this is a codegen option, so it shouldn't go under -C.

[alexcrichton]

@Zoxc hm I find that comment sort of curt and not super helpful. Do you have a suggestion of where it might go? Do you have a suggestion about simply enabling this by default in --error-format=json mode? It'd be great to expand on your thoughts!

[eddyb]

In #60987 (comment) I did mention I'd prefer if all the options related to the compiler's output were maybe grouped, like -C options are, but I didn't have a concrete suggestion.
I suspect @Zoxc might like that, but personally I wouldn't mind if we had the minimum amount of configurability and mostly just kept adding information to the JSON output without a flag.

[nikomatsakis]

I tagged @michaelwoerister as they are on parental leave.

[alexcrichton]

ping @Zoxc, @nagisa, @petrochenkov, @pnkfelix, y'all mind taking a look at this?

[Zoxc]

I think it would make sense to have an option which streams requested information to stdout in the form of JSON blobs, something like --json=short-errors,long-errors,artifacts, with each JSON blob indicating what kind of output it is with {"type": "artifact"} or something.

We could add to this list later if cargo, tools or IDEs want some more easily parsable information from rustc.

[alexcrichton]

@Zoxc do you intend to hold off on stabilization of this feature until that issue is resolved? That seems like a nice feature to have, but something we could always add after the fact.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[alexcrichton]

While we're waiting for FCP to finish, I've proposed an implementation of this stabilization at #62766

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.