Execution order for plugin composition

[kekssw]

The problem

Plugins are advertised as a primary mechanism for customizing Kong's behavior.
In order for different plugins to interact with each other, they have to be executed in certain order (in addition to communicating through shared memory, ngx.ctx, custom headers, or other means).
The only mechanism currently available for defining the order, is using PRIORITY in a handler table, and that is interpreted the same way for all supported request handling phases: on the way up (:access()), or on the way down (:header_filter(), :body_filter(), or :log()).

This implementation restricts how one can compose plugins to build more complex systems of their own, or to customize the behavior of standard plugins.
More specifically, I would find it extremely useful to implement middleware plugins, i.e. to wrap existing plugins into new ones that would simply act as adapters -- either for the upstream or downstream components. With the current priority system, I can only execute my pieces of logic before forwarding the request and before processing the response (or after and after, with lower 'PRIORITY' value), which requires two additional custom plugins to wrap every single one (where we need before/after or after/before type of integration).

E.g. for our weighted rate limiting solution, I would love to adhere to @thefosk's advise, and to build it around advanced response-rate-limiting plugin, rather than forking simplistic rate-limiting plugin and embedding our custom weighting logic inline. Unfortunately, current Kong's design made it easier for us to fork an (otherwise less suitable) plugin, while cleaner design would suggest to get as much value as possible by integrating externally with a more feature-rich plugin, as a bonus avoiding the headache of maintaining our own forks across Kong upgrades.

Related issues and documentation

• Plugin Development Guide explains how PRIORITY works today, and mentions
• Order in which Plugins are executed #267, which is focused on making PRIORITY configurable, and in turn is linked to
• Applying a Plugin with a variety of global and specific properties #505 that suggests quite a redesign of the entire plugin subsystem

As we discussed with @thefosk on recent AWS conference, I'm filing a separate issue, as my specific problem might not require solving of any of the aforementioned issues.
E.g. we still may have a simple workflow, and PRIORITY may stay hardcoded in a plugin's code, but if it's interpreted as [+PRIORITY] on the way up, and [-PRIORITY] on the way down, that would enable building middleware plugins. (This would probably require introducing new field, something like UNSIGNED_PRIORITY, to keep PRIORITY field backward compatible).

At the same time, if it's decided not to solve the problem with execution order outside of the broader redesign scope (be it #505 or something else), I would like this issue to serve as a reminder that we need something more flexible than just defining same-for-all-phases priorities in configuration.

[thibaultcha]

Hi,

Thanks for the throughout explanation. We have this requirement on our radar and will take care of it when refactoring the plugins runloop and their interface. Thanks!

[SubratPadhi12345]

@thibaultcha I wanted to add plugins order of execution for a service, how can I do it?
Is it possible without creating a custom plugin?

[SubratPadhi12345]

Hi,

Thanks for the throughout explanation. We have this requirement on our radar and will take care of it when refactoring the plugins runloop and their interface. Thanks!

@thibaultcha Is this implemented now? How can I set plugin execution order for a service?