Tracking issue for plugin stabilization (plugin, plugin_registrar features)

[aturon]

Tracks stabilization for the plugin and plugin_registrar feature gates.

[aturon]

Current status: awaiting a complete revamp of the plugin system.

cc @nrc

[jimmycuadra]

Is anyone actively working on this? Curious what the status is since using Serde without the need for code generation with syntex on stable Rust is very high on my Rust wishlist. 😃

[steveklabnik]

@nrc is

[aturon]

In particular, check out the discussion here, which points to a bunch of blog posts by @nrc amongst other things. He plans to finish the blog series and post an RFC soon.

[jimmycuadra]

Thanks Aaron—I had read a couple of the posts about the macro system but it didn't click that this was intended to replace compiler plugins for the purpose of syntax extensions. It makes sense now, and looks great. I can't wait. :}

[knight42]

Any progress now?

[SimonSapin]

Yes! The "Macros 1.1" RFC at rust-lang/rfcs#1681 was accepted. It’s implementation is tracked at #35900.

[SimonSapin]

The one remaining plugin in Servo is a register_late_lint_pass that analyses use of GC-managed types to make sure they’re "rooted" correctly. It traverses expressions HIR expression inside functions and methods, and requires access to the resolved type of these expressions as well as attributes on the definitions of these types. (Looking for custom attributes that indicate which types must be rooted. These attributes also need to be allowed by the compiler.)

Could this kind of analysis be built on top of RLS? Could that be a way forward for custom lints without relying on unstable rustc internals?

[eisterman]

Any progress now?

[0rvar]

Are there any plans for stabilizing plugins in the foreseeable future?

[oli-obk]

The plan is to remove the plugin API entirely.

[iddm]

@oli-obk Could you go into details please?

[oli-obk]

Even if we stabilized the plugin interface, it would be useless, because there is no stable API it offers. Instead things like proc macros 2.0, custom derive, ... are stabilized by offering an interface that does not require the language to guarantee a plugin interface to the compiler. All of these non-plugin-requiring APIs can be implemented with various other schemes, even if they are currently implemented via plugins in the background.

Note: this issue is not about allowing writing bin crates which can be extended via plugins, it is about plugins that can extend the compiler.

[iddm]

Is not rocket a plugin to a compiler? If so, that means we will bury it alive?

[oli-obk]

Rocket is a plugin to the compiler because proc macros 2.0, and most notably their attribute support is not implemented yet. Of course there is no desire to eliminate any uses of the plugin API without a viable alternative. But there is also no desire to stabilize it while there are much better alternatives on the horizon. Especially if stabilizing has a high cost for the compiler, while the alternatives do not have the same cost associated with them.

[Centril]

@SimonSapin What's the status on Servo's use of this?

[SimonSapin]

Unchanged. https://github.com/servo/servo/blob/c7cd1b83a11/components/script_plugins/lib.rs uses #[plugin_registrar], register_late_lint_pass, and register_attribute.

[syntacticsugarglider]

What's going to replace the support for custom lints provided by lint plugins? Some sort of alternative seems like a good idea.

[SimonSapin]

At this point there is no replacement. See #62868 for some discussion of this.

[joshtriplett]

At this time, what's still using plugin support?

[SimonSapin]

No change to report for Servo. https://github.com/servo/servo/tree/9693111c9/components/script_plugins is still an old-style plugin. I don’t know of another practical way to write a custom lint that has access to type resolution. Details in #62868.

Without this lint, any Rust code in Servo that manipulates values owned by the GC (which is a lot of code) would need to be marked as unsafe and just be very careful not to forget to "root" GC any pointer on the stack.

[dlight]

At this time, what's still using plugin support?

I'm not sure if this counts, but there's this crate https://github.com/draivin/rust-hotswap that although wasn't updated in a while (and I'm not sure if it even builds anymore), it's pretty cool.

[pnkfelix]

At this time, what's still using plugin support?

Discussed in T-compiler triage meeting today.

I think the T-compiler strategy should be this:

1. Develop API for MIR-based analyses that we can commit to. The stakeholders I know of for this are Servo (e.g. #14902 Write a MIR lint for rooting analysis servo/servo#20264) and https://github.com/model-checking/rmc
2. Once Servo is moved off of plugin and onto a MIR-based analysis, then we start officially deprecating and subsequent removal of plugin support.

@rustbot label: -I-compiler-nominated

[SimonSapin]

In my previous message I only replied to #29597 (comment) on a technical level.

Activity in the Servo project has been low lately and will be in the forseeable future. If a better API becomes available, it’s not clear when or if we’d be able to migrate the GC rooting lint to it. Given this, I feel it’s probably not fair to expect rustc to indefinitely keep the burden of maintaining plugin support for Servo’s sake.

[blyxyas]

Of course there is no desire to eliminate any uses of the plugin API without a viable alternative

Is there any viable alternatives? I really like compiler plugins, even if "they would be useless" (I don't see how)

[spoutn1k]

So what alternatives are there to replace the plugin interface ? Specifically, AST code injection. I was looking to create a plugin instrumenting code for specialized performance tracking, is this just not going to be supported going forward ?

Back in 2018 the following project had a proof of concept: https://github.com/da-x/instru.

Edit: I stumbled upon the following mentioning how the LLVM intrumentation coverage can be enabled in rustc. Does that means the LLVM backend is used reliably - ie can be used to inject code via LLVM passes ? I have seen conflicting opinions on the subject from the rust team.

[crlf0710]

Specifically, AST code injection.

For the ast-level things, isn't proc_macros enough? Just execute a proc_macro at the root module of the crate, parse everything, do whatever transform you want to the code, and put them back together as a TokenStream.

[dlight]

Specifically, AST code injection.

For the ast-level things, isn't proc_macros enough? Just execute a proc_macro at the root module of the crate, parse everything, do whatever transform you want to the code, and put them back together as a TokenStream.

If we could do #![macro_invocations] that wraps around the whole file, this would be much easier

[spoutn1k]

Just execute a proc_macro at the root module of the crate, parse everything

Does that entail tagging every function with a macro ? This project is meant to simplify the analysis of large scientific codebases. The plugin approach removes the need to dig into every twisted codebase, instead just compiling with a secret spice mix plugin to get instrumentation.

[blyxyas]

I find that creating a way to use custom inner macros would solve the problem. And if the plugin system is meant to simplify the analysis of large scientific codebases, there should be an alternative. Like custom inner attributes that are applied to every item.

[oli-obk]

Note that you can already replace rustc with your own by using the RUSTC_WRAPPER env var. This allows you to do your own preprocessing of the file and feed the output of that to the actual rustc. (Keyword "custom driver" if you want to search for ways to still access rustc internals like clippy and miri do)

The existing plugin API has not been maintained in years.