Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

📖 Improve plugin phase 1.5 EP #2075

Merged
merged 1 commit into from
Mar 16, 2021
Merged

📖 Improve plugin phase 1.5 EP #2075

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
137 changes: 107 additions & 30 deletions designs/extensible-cli-and-scaffolding-plugins-phase-1-5.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ The goal of this phase is to achieve one of the goals proposed for Phase 2: chai
Phase 2 includes several other challenging goals, but being able to chain plugins will be beneficial
for third-party developers that are using kubebuilder as a library.

The other main goal of phase 2, discovering and using external plugins, is out of the scope of this phase,
and will be tackled when phase 2 is implemented.

## Table of contents
- [Goal](#goal)
- [Motivation](#motivation)
Expand Down Expand Up @@ -46,36 +49,42 @@ Plugin chaining solves the aforementioned problems but the current plugin API, a

Design a Plugin API that combines the current [`Subcommand`](../pkg/plugin/interfaces.go) and
[`RunOptions`](../pkg/plugins/internal/cmdutil/cmdutil.go) interfaces and enables plugin-chaining.
Adirio marked this conversation as resolved.
Show resolved Hide resolved
The new `Subcommand` methods can be split in two different categories:
- Initialization methods
- Execution methods
The new `Subcommand` hooks can be split in two different categories:
- Initialization hooks
- Execution hooks

Initialization hooks are run during the dynamic creation of the CLI, which means that they are able to
modify the CLI, e.g. providing descriptions and examples for subcommands or binding flags.
Execution hooks are run after the CLI is created, and therefore cannot modify the CLI. On the other hand,
as they are run during the CLI execution, they have access to user-provided flag values, project configuration,
the new API resource or the filesystem abstraction, as opposed to the initialization hooks.

Additionally, some of these methods may be optional, in which case a non-implemented method will be skipped
when it should be called and consider it succeeded. This also allows to create some methods specific for
a certain subcommand call (e.g.: `Resource`-related methods for the `edit` subcommand are not needed).
Additionally, some of these hooks may be optional, in which case a non-implemented hook will be skipped
when it should be called and consider it succeeded. This also allows to create some hooks specific for
a certain subcommand call (e.g.: `Resource`-related hooks for the `edit` subcommand are not needed).

Different ordering guarantees can be considered:
- Method order guarantee: a method for a plugin will be called after its previous methods succeeded.
- Steps order guarantee: methods will be called when all plugins have finished the previous method.
- Plugin order guarantee: same method for each plugin will be called in the order specified
- Hook order guarantee: a hook for a plugin will be called after its previous hooks succeeded.
- Steps order guarantee: hooks will be called when all plugins have finished the previous hook.
- Plugin order guarantee: same hook for each plugin will be called in the order specified
by the plugin position at the plugin chain.

All of the methods will offer plugin order guarantee, as they all modify/update some item so the order
of plugins is important. Execution methods need to guarantee step order, as the items that are being modified
All of the hooks will offer plugin order guarantee, as they all modify/update some item so the order
of plugins is important. Execution hooks need to guarantee step order, as the items that are being modified
in each step (config, resource, and filesystem) are also needed in the following steps. This is not true for
initialization methods that modify items (metadata and flagset) that are only used in their own methods,
so they only need to guarantee method order.
initialization hooks that modify items (metadata and flagset) that are only used in their own methods,
so they only need to guarantee hook order.

Execution methods will be able to return an error. A specific error can be returned to specify that
no further methods of this plugin should be called, but that the scaffold process should be continued.
Execution hooks will be able to return an error. A specific error can be returned to specify that
no further hooks of this plugin should be called, but that the scaffold process should be continued.
This enables plugins to exit early, e.g., a plugin that scaffolds some files only for cluster-scoped
resources can detect if the resource is cluster-scoped at one of the first execution steps, and
therefore, use this error to tell the CLI that no further execution step should be called for itself.

### Initialization methods
### Initialization hooks

#### Update metadata
This method will be used for two purposes. It provides CLI-related metadata to the Subcommand (e.g.,
This hook will be used for two purposes. It provides CLI-related metadata to the Subcommand (e.g.,
command name) and update the subcommands metadata such as the description or examples.

- Required/optional
Expand All @@ -88,7 +97,7 @@ command name) and update the subcommands metadata such as the description or exa
- [x] Create webhook

#### Bind flags
This method will allow subcommands to define specific flags.
This hook will allow subcommands to define specific flags.

- Required/optional
- [ ] Required
Expand All @@ -102,7 +111,7 @@ This method will allow subcommands to define specific flags.
### Execution methods

#### Inject configuration
This method will be used to inject the `Config` object that the plugin can modify at will.
This hook will be used to inject the `Config` object that the plugin can modify at will.
The CLI will create/load/save this configuration object.

- Required/optional
Expand All @@ -115,7 +124,7 @@ The CLI will create/load/save this configuration object.
- [x] Create webhook

#### Inject resource
This method will be used to inject the `Resource` object.
This hook will be used to inject the `Resource` object created by the CLI.

- Required/optional
- [x] Required
Expand All @@ -127,9 +136,9 @@ This method will be used to inject the `Resource` object.
- [x] Create webhook

#### Pre-scaffold
This method will be used to take actions before the main scaffolding is performed, e.g. validations.
This hook will be used to take actions before the main scaffolding is performed, e.g. validations.

NOTE: a filesystem abstraction will be passed to this method that must be used for scaffolding.
NOTE: a filesystem abstraction will be passed to this hook, but it should not be used for scaffolding.

- Required/optional
- [ ] Required
Expand All @@ -141,9 +150,9 @@ NOTE: a filesystem abstraction will be passed to this method that must be used f
- [x] Create webhook

#### Scaffold
This method will be used to perform the main scaffolding.
This hook will be used to perform the main scaffolding.

NOTE: a filesystem abstraction will be passed to this method that must be used for scaffolding.
NOTE: a filesystem abstraction will be passed to this hook that must be used for scaffolding.

- Required/optional
- [x] Required
Expand All @@ -155,11 +164,14 @@ NOTE: a filesystem abstraction will be passed to this method that must be used f
- [x] Create webhook

#### Post-scaffold
This method will be used to take actions after the main scaffolding is performed, e.g. cleanup.
This hook will be used to take actions after the main scaffolding is performed, e.g. cleanup.

NOTE: a filesystem abstraction will **NOT** be passed to this method, as post-scaffold task do not require it.
NOTE: a filesystem abstraction will **NOT** be passed to this hook, as post-scaffold task do not require it.
In case some post-scaffold task requires a filesystem abstraction, it could be added.

NOTE 2: the project configuration is saved by the CLI before calling this hook, so changes done to the
configuration at this hook will not be persisted.

- Required/optional
- [ ] Required
- [x] Optional
Expand All @@ -168,10 +180,67 @@ In case some post-scaffold task requires a filesystem abstraction, it could be a
- [x] Edit
- [x] Create API
- [x] Create webhook

### Override plugins for single subcommand calls

Defining plugins at initialization and using them for every command call will solve most of the cases.
However, there are some cases where a plugin may be wanted just for a certain subcommand call. For
example, a project with multiple controllers may want to follow the declarative pattern in only one of
their controllers. The other case is also relevant, a project where most of the controllers follow the
declarative pattern may need a single controller not to follow it.

In order to achieve this, the `--plugins` flag will be allowed in every command call, overriding the
value used in its corresponging project initialization call.

### Plugin chain persistence

Currently, the project configuration v3 offers two mechanisms for storing plugin-related information.

- A layout field (`string`) that is used for plugin resolution on initialized projects.
- A plugin field (`map[string]interface{}`) that is used for plugin configuration raw storage.

Plugin resolution uses the `layout` field to resolve plugins. In this phase, it has to store a plugin
chain and not a single plugin. As this value is stored as a string, comma-separated representation can
be used to represent a chain of plugins instead.

NOTE: commas are not allowed in the plugin key.

While the `plugin` field may seem like a better fit to store the plugin chain, as it can already
contain multiple values, there are several issues with this alternative approach:
- A map does not provide any order guarantee, and the plugin chain order is relevant.
- Some plugins do not store plugin-specific configuration information, e.g. the `go`-plugins. So
the absence of a plugin key doesn't mean that the plugin is not part of the plugin chain.
- The desire of running a different set of plugins for a single subcommand call has already been
mentioned. Some of these out-of-chain plugins may need to store plugin-specific configuration,
so the presence of a plugin doesn't mean that is part of the plugin chain.

The next project configuration version could consider this new requirements to define the
names/types of these two fields.

### Plugin bundle

As a side-effect of plugin chaining, the user experience may suffer if they need to provide
several plugin keys for the `--plugins` flag. Additionally, this would also mean a user-facing
important breaking change.

In order to solve this issue, a plugin bundle concept will be introduced. A plugin bundle
behaves as a plugin:
- It has a name: provided at creation.
- It has a version: provided at creation.
- It has a list of supported project versions: computed from the common supported project
versions of all the plugins in the bundled.

Instead of implementing the optional getter methods that return a subcommand, it offers a way
to retrieve the list of bundled plugins. This process will be done after plugin resolution.

This way, CLIs will be able to define bundles, which will be used in the user-facing API and
the plugin resolution process, but later they will be treated as separate plugins offering
the maintainability and separation of concerns advantages that smaller plugins have in
comparison with bigger monolithic plugins.

Copy link
Member

@camilamacedo86 camilamacedo86 Mar 9, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User Stories

  • As operator developer would like to generate a project with custom plugins that will be used by default so that, after generating a project with a custom plugin such as kb init—plugins=go/v3,declarative all APIs scaffolded with kb create api subcommand all APIs scaffold will be customized with the optional plugin declarative.

  • As an operator developer would like to use a custom plugin just for a specific subcommand so that, I can, for example, use the declarative pattern only for a specific API and not for all project (e.g kb create api subcommand all APIs scaffold will be customized with the optional plugin declarative). More info; Allow every command to overwrite project-configured plugins #1941

  • As a KB maintainer/contributor and/or consumer I would like to know the plugins used to configure a project as its specific configurations and data when required so that, I can tooling upgrade and modifications actions to help the users implement smarter plugins. More info; Allow every command to overwrite project-configured plugins #1941

  • As a KB maintainer/contributor and/or consumer I would like to define a plugin is composed of 2 or more plugins so that, I can create a plugin that will re-use the code of other plugins and has customizations on top to address my need such as go/v3 + helm/v3 + plugin that does all required adjustments that make my hybrid operator works and to be able to extract the default configuration from the language files and provide different options for configuration without increase the maintainability and bring facilities to ensure the backwards compatibilities of the plugin. Also, I can easily imply that for example that any go/v3 plugin scaffold by my CLI also will be customized with my specific requirements which is a current SDK requirement.

## Implementation

The following types are used as input/output values of the described methods:
The following types are used as input/output values of the described hooks:
```go
// CLIMetadata is the runtime meta-data of the CLI
type CLIMetadata struct {
Expand All @@ -197,7 +266,7 @@ func (e ExitError) Error() string {
}
```

The described methods are implemented through the use of the following interfaces.
The described hooks are implemented through the use of the following interfaces.
```go
type RequiresCLIMetadata interface {
InjectCLIMetadata(CLIMetadata)
Expand All @@ -220,11 +289,11 @@ type RequiresResource interface {
}

type HasPreScaffold interface {
PreScaffold(afero.Fs) error
PreScaffold(machinery.Filesystem) error
}

type Scaffolder interface {
Scaffold(afero.Fs) error
Scaffold(machinery.Filesystem) error
}

type HasPostScaffold interface {
Expand Down Expand Up @@ -256,3 +325,11 @@ type CreateWebhookSubcommand interface {
Scaffolder
}
```

An additional interface defines the bundle method to return the wrapped plugins:
```go
type Bundle interface {
Plugin
Plugins() []Plugin
}
```