-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #10125 from hashicorp/docs-autoscaler-409-plugins
autoscaler plugin docs
- Loading branch information
Showing
10 changed files
with
275 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
49 changes: 49 additions & 0 deletions
49
website/content/docs/autoscaling/internals/plugins/apm.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
--- | ||
layout: docs | ||
page_title: APM Plugins | ||
sidebar_title: APM | ||
description: Learn how to author a Nomad Autoscaler APM plugin. | ||
--- | ||
|
||
# APM Plugins | ||
|
||
APM plugins are used by the autoscaler to interact with an external APM system, | ||
returning metrics that are used by the autoscaler to inform scaling actions. | ||
|
||
For a real-world example of a Nomad APM plugin implementation, see the | ||
[`prometheus` plugin](https://github.com/hashicorp/nomad-autoscaler/tree/main/plugins/builtin/apm/prometheus). | ||
|
||
## Authoring APM Plugins | ||
|
||
Authoring an APM plugin in Go can be accomplished by implementing the | ||
[`apm.APM`][apm_plugin] interface, alongside a `main` package to launch the plugin. | ||
|
||
The [`no-op` APM plugin][noop_plugin] can be used as a starting point for new APM | ||
plugins. | ||
|
||
## APM Plugin API | ||
|
||
The [base plugin][base_plugin] interface must be implemented in addition to the | ||
following functions. | ||
|
||
#### `Query(query string, timeRange sdk.TimeRange) (sdk.TimestampedMetrics, error)` | ||
|
||
The `Query` function is called by the agent during policy | ||
evaluation. The `query` argument is the opaque string from the scaling policy, | ||
and the `timeRange` indicates the period of time over which the query should be | ||
made. The response is a series of timestamped metrics; as such, the query semantics | ||
should be such that the backing APM will return a time series. An example is the | ||
CPU utilization of a task group, averaged over all current allocations. | ||
|
||
#### `QueryMultiple(query string, timeRange sdk.TimeRange) ([]sdk.TimestampedMetrics, error)` | ||
|
||
The `QueryMultiple` method is similar to `Query`, except that the interface allows | ||
multiple time series to be returned. This endpoint is currently only used for | ||
[Dynamic Application Sizing][das]. | ||
An example would be to return the CPU utilization for all allocations during | ||
the time range. | ||
|
||
[apm_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/apm/apm.go#L11 | ||
[base_plugin]: /docs/autoscaling/internals/plugins/base | ||
[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-apm | ||
[das]: /docs/autoscaling#dynamic-application-sizing |
34 changes: 34 additions & 0 deletions
34
website/content/docs/autoscaling/internals/plugins/base.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
--- | ||
layout: docs | ||
page_title: Base Plugin | ||
sidebar_title: Base | ||
description: Learn about how to author a Nomad Autoscaler plugin. | ||
--- | ||
|
||
# Base Plugin | ||
|
||
The base plugin is an interface required of all autoscaler plugins, | ||
providing a set of common operations. | ||
|
||
## Plugin API | ||
|
||
#### `PluginInfo() (*PluginInfo, error)` | ||
|
||
A `PluginInfo` contains metadata about the plugin. For example, | ||
the Prometheus APM plugin returns the following; | ||
|
||
```go | ||
PluginInfo{ | ||
// Name of the plugin | ||
Name: "prometheus", | ||
// Plugin type: "apm", "strategy", or "target" | ||
PluginType: "apm" | ||
} | ||
``` | ||
|
||
#### `SetConfig(config map[string]string) error` | ||
|
||
The `SetConfig` function is called when starting an instance of the plugin. It contains the | ||
configuration for a named instance of the plugin as provided in the autoscaler [agent config][plugin_config]. | ||
|
||
[plugin_config]: /docs/autoscaling/agent |
41 changes: 41 additions & 0 deletions
41
website/content/docs/autoscaling/internals/plugins/index.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
--- | ||
layout: docs | ||
page_title: Plugins | ||
sidebar_title: Plugins | ||
description: Learn about how external plugins work in the Nomad Autoscaler. | ||
--- | ||
|
||
# Plugins | ||
|
||
The Nomad Autoscaler uses a plugin framework which allows users to extend its | ||
functionality. The design of the plugin system is inspired by the | ||
[plugin system][nomad_plugin_system] used in Nomad for task drivers and | ||
devices. | ||
|
||
The following components are currently pluggable within the Nomad Autoscaler: | ||
|
||
- [APMs](/docs/autoscaling/internals/plugins/apm) | ||
- [Strategies](/docs/autoscaling/internals/plugins/strategy) | ||
- [Targets](/docs/autoscaling/internals/plugins/target) | ||
|
||
In addition, each plugin implements a [base](/docs/autoscaling/internals/plugins/base) | ||
plugin functionality. | ||
|
||
# Architecture | ||
|
||
The Nomad Autoscaler plugin framework uses the [go-plugin][goplugin] project to expose | ||
a language-independent plugin interface. Plugins implement a set of gRPC | ||
services and methods which the Autoscaler agent manages by running the plugin | ||
and calling the implemented RPCs. This means that plugins are free to be | ||
implemented in the author's language of choice. | ||
|
||
To make plugin development easier, a set of Go interfaces and structs exist for | ||
each plugin type that abstract the `go-plugin` and gRPC interfaces. The guides in | ||
this documentation reference these abstractions for ease of use. | ||
|
||
The existing plugins can serve as examples; in addition, no-op external plugins | ||
are available in the [autoscaler repo][noop_plugins]. | ||
|
||
[goplugin]: https://github.com/hashicorp/go-plugin | ||
[nomad_plugin_system]: /docs/internals/plugins | ||
[noop_plugins]: https://github.com/hashicorp/nomad-autoscaler/tree/main/plugins/test |
41 changes: 41 additions & 0 deletions
41
website/content/docs/autoscaling/internals/plugins/strategy.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
--- | ||
layout: docs | ||
page_title: Strategy Plugins | ||
sidebar_title: Strategy | ||
description: Learn how to author a Nomad Autoscaler Strategy plugin. | ||
--- | ||
|
||
# Strategy Plugins | ||
|
||
Strategy plugins are used by the autoscaler to implement the logic of scaling strategy. | ||
They typically consume metrics from the configured APM plugin and the current scaling value | ||
from the target plugin and compute a new desired value for the scaling target. | ||
|
||
For a real-world example of a Nomad strategy plugin implementation, see the | ||
[`target-value` plugin][target_value]. | ||
|
||
## Authoring Strategy Plugins | ||
|
||
Authoring a strategy plugin in Go can be accomplished by implementing the | ||
[`strategy.Strategy`][strategy_plugin] interface, alongside a `main` package to launch the plugin. | ||
|
||
The [`no-op` Strategy plugin][noop_plugin] can be used as a starting point for new Strategy | ||
plugins. | ||
|
||
## Strategy Plugin API | ||
|
||
The [base plugin][base_plugin] interface must be implemented in addition to the | ||
following functions. | ||
|
||
#### `Run(eval *sdk.ScalingCheckEvaluation, count int64) (*sdk.ScalingCheckEvaluation, error)` | ||
|
||
The `Run` function is called by the agent during policy | ||
evaluation. The `eval` argument contains [information](https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/sdk/eval_oss.go#L8) | ||
about the current policy evaluation, including the policy specification and the metrics from the APM. The `count` | ||
argument includes the current value of the scaling target. The returned struct should include the result | ||
from applying the strategy. | ||
|
||
[strategy_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/strategy/strategy.go#L11 | ||
[base_plugin]: /docs/autoscaling/internals/plugins/base | ||
[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-strategy | ||
[target_value]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/builtin/strategy/target-value |
51 changes: 51 additions & 0 deletions
51
website/content/docs/autoscaling/internals/plugins/target.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
--- | ||
layout: docs | ||
page_title: Target Plugins | ||
sidebar_title: Target | ||
description: Learn how to author a Nomad Autoscaler Target plugin. | ||
--- | ||
|
||
# Target Plugins | ||
|
||
Target plugins are used by the autoscaler to retrieve information about a | ||
scaling target and also to perform a scaling action against them. | ||
|
||
For a real-world example of a Nomad Target plugin implementation, see the | ||
[Nomad `group->count` plugin][nomad_group_count_plugin]. | ||
|
||
## Authoring Target Plugins | ||
|
||
Authoring a target plugin in Go can be accomplished by implementing the | ||
[`target.Target`][target_plugin] interface, alongside a `main` package to | ||
launch the plugin. | ||
|
||
The [`no-op` Target plugin][noop_plugin] can be used as a starting point for new Target | ||
plugins. | ||
|
||
## Target Plugin API | ||
|
||
The [base plugin][base_plugin] interface must be implemented in addition to the | ||
following functions. | ||
|
||
#### `Scale(action sdk.ScalingAction, config map[string]string) error` | ||
|
||
The `Scale` method is called by the agent during policy evaluation. The `action` | ||
argument specifies the [details][scaling_action_sdk] about the scaling action | ||
that should be made against the target. `config` includes the details about the | ||
scaling target that were provided in the [scaling policy][policy_target]. | ||
|
||
#### `Status(config map[string]string) (*sdk.TargetStatus, error)` | ||
|
||
The `Status` method is called by the agent in order to determine the current | ||
status of a scaling target. This is performed as part of policy evaluation, | ||
and the [information][target_status_sdk] returned may be used by the scaling | ||
strategy to inform the next scaling action. Information returned includes | ||
current scaling level, readiness, and arbitrary metadata. | ||
|
||
[nomad_group_count_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/builtin/target/nomad | ||
[target_plugin]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/plugins/target/target.go#L12 | ||
[base_plugin]: /docs/autoscaling/internals/plugins/base | ||
[noop_plugin]: https://github.com/hashicorp/nomad-autoscaler/tree/v0.3.0/plugins/test/noop-target | ||
[scaling_action_sdk]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/sdk/strategy.go#L25 | ||
[policy_target]: /docs/autoscaling/policy#target | ||
[target_status_sdk]: https://github.com/hashicorp/nomad-autoscaler/blob/v0.3.0/sdk/target.go#L6 |
24 changes: 24 additions & 0 deletions
24
website/content/docs/autoscaling/plugins/external/index.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
--- | ||
layout: docs | ||
page_title: 'Autoscaler Plugins: Community Supported' | ||
sidebar_title: Community | ||
description: A list of community-supported Autoscaler Plugins. | ||
--- | ||
|
||
# Community Supported | ||
|
||
If you have authored an autoscaler plugin that you believe will be useful to the | ||
broader Nomad community and you are committed to maintaining the plugin, please | ||
file a PR to add your plugin to this page. | ||
|
||
For details on authoring an autoscaler plugin, please refer to the [plugin | ||
authoring guide][plugin_guide]. | ||
|
||
Below is a list of community-support plugins available for the Nomad autoscaler. | ||
|
||
## Target Plugins | ||
|
||
- [OpenStack Senlin][senlin] | ||
|
||
[plugin_guide]: /docs/autoscaling/internals/plugins | ||
[senlin]: https://github.com/dkt26111/nomad-senlin-autoscaler |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters