Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate alternative names on secrets #3406

Merged
merged 14 commits into from
Feb 22, 2024
8 changes: 4 additions & 4 deletions docs/docs/20-usage/20-workflow-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,13 +164,13 @@ Allows you to specify the entrypoint for containers. Note that this must be a li

Woodpecker provides the ability to pass environment variables to individual steps.

For more details check the [environment docs](./50-environment.md).
For more details, check the [environment docs](./50-environment.md).

### `secrets`

Woodpecker provides the ability to store named parameters external to the YAML configuration file, in a central secret store. These secrets can be passed to individual steps of the workflow at runtime.

For more details check the [secrets docs](./40-secrets.md).
For more details, check the [secrets docs](./40-secrets.md).

### `failure`

Expand Down Expand Up @@ -574,10 +574,10 @@ For more details check the [matrix build docs](./30-matrix-workflows.md).

You can set labels for your workflow to select an agent to execute the workflow on. An agent will pick up and run a workflow when **every** label assigned to it matches the agents labels.

To set additional agent labels check the [agent configuration options](../30-administration/15-agent-config.md#woodpecker_filter_labels). Agents will have at least four default labels: `platform=agent-os/agent-arch`, `hostname=my-agent`, `backend=docker` (type of the agent backend) and `repo=*`. Agents can use a `*` as a wildcard for a label. For example `repo=*` will match every repo.
To set additional agent labels, check the [agent configuration options](../30-administration/15-agent-config.md#woodpecker_filter_labels). Agents will have at least four default labels: `platform=agent-os/agent-arch`, `hostname=my-agent`, `backend=docker` (type of the agent backend) and `repo=*`. Agents can use a `*` as a wildcard for a label. For example `repo=*` will match every repo.

Workflow labels with an empty value will be ignored.
By default each workflow has at least the `repo=your-user/your-repo-name` label. If you have set the [platform attribute](#platform) for your workflow it will have a label like `platform=your-os/your-arch` as well.
By default, each workflow has at least the `repo=your-user/your-repo-name` label. If you have set the [platform attribute](#platform) for your workflow it will have a label like `platform=your-os/your-arch` as well.

You can add additional labels as a key value map:

Expand Down
29 changes: 7 additions & 22 deletions docs/docs/20-usage/40-secrets.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,20 @@ once their usage is declared in the `secrets` section:
+ secrets: [ docker_username, docker_password ]
```

### Use secrets in settings
### Use secrets in settings and environment

Alternatively, you can get a `setting` from secrets using the `from_secret` syntax.
In this example, the secret named `secret_token` would be passed to the setting named `token`, which will be available in the plugin as environment variable named `PLUGIN_TOKEN`. See [Plugins](./51-plugins/20-creating-plugins.md#settings) for details.
Alternatively, you can get a setting or environment from secrets using the `from_secret` syntax. This also allows using a different name for the environment variable.
qwerty287 marked this conversation as resolved.
Show resolved Hide resolved

:::note
The `from_secret` syntax only works with the newer `settings` block.
:::
In this example, the secret named `secret_token` would be passed to the setting named `token`,which will be available in the plugin as environment variable named `PLUGIN_TOKEN` (See [plugins](./51-plugins/20-creating-plugins.md#settings) for details), and to the environment variable `TOKEN_ENV`.

```diff
steps:
- name: docker
image: my-plugin
settings:
+ environment:
+ TOKEN_ENV:
+ from_secret: secret_token
+ settings:
+ token:
+ from_secret: secret_token
```
Expand All @@ -60,21 +60,6 @@ Please note parameter expressions are subject to pre-processing. When using secr
secrets: [ docker_username, docker_password ]
```

### Alternate Names

There may be scenarios where you are required to store secrets using alternate names. You can map the alternate secret name to the expected name using the below syntax:

```diff
steps:
- name: docker
image: plugins/docker
repo: octocat/hello-world
tags: latest
+ secrets:
+ - source: docker_prod_password
+ target: docker_password
```

### Use in Pull Requests events

Secrets are not exposed to pull requests by default. You can override this behavior by creating the secret and enabling the `pull_request` event type, either in UI or by CLI, see below.
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/20-usage/50-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ Woodpecker provides the ability to pass environment variables to individual pipe
- name: build
image: golang
+ environment:
+ - CGO=0
+ - GOOS=linux
+ - GOARCH=amd64
+ CGO: 0
+ GOOS: linux
+ GOARCH: amd64
commands:
- go build
- go test
Expand Down
24 changes: 0 additions & 24 deletions docs/docs/20-usage/60-services.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,30 +53,6 @@ Service containers generally expose environment variables to customize service s
image: redis
```

qwerty287 marked this conversation as resolved.
Show resolved Hide resolved
## Detachment

Service and long running containers can also be included in the pipeline section of the configuration using the detach parameter without blocking other steps. This should be used when explicit control over startup order is required.

```diff
steps:
- name: build
image: golang
commands:
- go build
- go test

- name: database
image: redis
+ detach: true

- name: test
image: golang
commands:
- go test
```

Containers from detached steps will terminate when the pipeline ends.

## Initialization

Service containers require time to initialize and begin to accept connections. If you are unable to connect to a service you may need to wait a few seconds or implement a backoff.
Expand Down
3 changes: 3 additions & 0 deletions docs/docs/91-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ Some versions need some changes to the server configuration or the pipeline conf
- Removed `WOODPECKER_ROOT_PATH` and `WOODPECKER_ROOT_URL` config variables. Use `WOODPECKER_HOST` with a path instead
- Pipelines without a config file will now be skipped instead of failing
- Deprecated `includes` and `excludes` support from **event** filter
- Deprecated alternative names for secrets, use `environment` with `from_secret`
- Deprecated slice definition for labels and env vars
- Deprecated `detached` in favor of services
qwerty287 marked this conversation as resolved.
Show resolved Hide resolved

## 2.0.0

Expand Down
1 change: 0 additions & 1 deletion pipeline/backend/docker/convert_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,6 @@ func TestToConfigFull(t *testing.T) {
Type: backend.StepTypeCommands,
Image: "golang:1.2.3",
Pull: true,
Detached: true,
Privileged: true,
WorkingDir: "/src/abc",
Environment: map[string]string{"TAGS": "sqlite"},
Expand Down
1 change: 0 additions & 1 deletion pipeline/backend/types/step.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ type Step struct {
Type StepType `json:"type,omitempty"`
Image string `json:"image,omitempty"`
Pull bool `json:"pull,omitempty"`
Detached bool `json:"detach,omitempty"`
Privileged bool `json:"privileged,omitempty"`
WorkingDir string `json:"working_dir,omitempty"`
Environment map[string]string `json:"environment,omitempty"`
Expand Down
6 changes: 5 additions & 1 deletion pipeline/frontend/yaml/compiler/compiler.go
Original file line number Diff line number Diff line change
Expand Up @@ -178,11 +178,15 @@ func (c *Compiler) Compile(conf *yaml_types.Workflow) (*backend_types.Config, er
if c.metadata.Curr.Event == metadata.EventTag {
cloneSettings["tags"] = "true"
}
env := map[string]any{}
qwerty287 marked this conversation as resolved.
Show resolved Hide resolved
for k, v := range c.cloneEnv {
env[k] = v
}
container := &yaml_types.Container{
Name: defaultCloneName,
Image: cloneImage,
Settings: cloneSettings,
Environment: c.cloneEnv,
Environment: env,
}
step, err := c.createProcess(container, backend_types.StepTypeClone)
if err != nil {
Expand Down
17 changes: 9 additions & 8 deletions pipeline/frontend/yaml/compiler/convert.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ func (c *Compiler) createProcess(container *yaml_types.Container, stepType backe
var (
uuid = ulid.Make()

detached bool
workingDir string

workspace = fmt.Sprintf("%s_default:%s", c.prefix, c.base)
Expand Down Expand Up @@ -76,16 +75,15 @@ func (c *Compiler) createProcess(container *yaml_types.Container, stepType backe

// append default environment variables
environment := map[string]string{}
maps.Copy(environment, container.Environment)
maps.Copy(environment, c.env)

environment["CI_WORKSPACE"] = path.Join(c.base, c.path)

if stepType == backend_types.StepTypeService || container.Detached {
detached = true
if container.Detached {
stepType = backend_types.StepTypeService
}

if !detached || len(container.Commands) != 0 {
if stepType != backend_types.StepTypeService || len(container.Commands) != 0 {
workingDir = c.stepWorkingDir(container)
}

Expand All @@ -105,13 +103,17 @@ func (c *Compiler) createProcess(container *yaml_types.Container, stepType backe
return secret.Value, nil
}

// TODO: why don't we pass secrets to detached steps?
if !detached {
// TODO: why don't we pass settings to services?
if stepType != backend_types.StepTypeService {
if err := settings.ParamsToEnv(container.Settings, environment, getSecretValue); err != nil {
return nil, err
}
}

if err := settings.ParamsToEnv(container.Environment, environment, getSecretValue); err != nil {
return nil, err
}

for _, requested := range container.Secrets.Secrets {
secretValue, err := getSecretValue(requested.Source)
if err != nil {
Expand Down Expand Up @@ -184,7 +186,6 @@ func (c *Compiler) createProcess(container *yaml_types.Container, stepType backe
Type: stepType,
Image: container.Image,
Pull: container.Pull,
Detached: detached,
Privileged: privileged,
WorkingDir: workingDir,
Environment: environment,
Expand Down
32 changes: 32 additions & 0 deletions pipeline/frontend/yaml/linter/linter.go
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,38 @@ func (l *Linter) lintDeprecations(config *WorkflowConfig) (err error) {
}
}

for _, step := range parsed.Steps.ContainerList {
for i, c := range step.Secrets.Secrets {
if c.Source != c.Target {
err = multierr.Append(err, &errors.PipelineError{
Type: errors.PipelineErrorTypeDeprecation,
Message: "Secrets alternative names are deprecated, use environment with from_secret",
Data: errors.DeprecationErrorData{
File: config.File,
Field: fmt.Sprintf("steps.%s.secrets[%d]", step.Name, i),
Docs: "https://woodpecker-ci.org/docs/usage/secrets#use-secrets-in-settings-and-environment",
},
IsWarning: true,
})
}
}
}

for _, step := range parsed.Steps.ContainerList {
if step.Detached {
err = multierr.Append(err, &errors.PipelineError{
Type: errors.PipelineErrorTypeDeprecation,
Message: "Detached is deprecated, use services",
Data: errors.DeprecationErrorData{
File: config.File,
Field: fmt.Sprintf("steps.%s.detached", step.Name),
Docs: "https://woodpecker-ci.org/docs/usage/services",
},
IsWarning: true,
})
}
}

return err
}

Expand Down
3 changes: 1 addition & 2 deletions pipeline/frontend/yaml/parse.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import (

"go.woodpecker-ci.org/woodpecker/v2/pipeline/frontend/yaml/constraint"
"go.woodpecker-ci.org/woodpecker/v2/pipeline/frontend/yaml/types"
"go.woodpecker-ci.org/woodpecker/v2/pipeline/frontend/yaml/types/base"
)

// ParseBytes parses the configuration from bytes b.
Expand Down Expand Up @@ -53,7 +52,7 @@ func ParseBytes(b []byte) (*types.Workflow, error) {
// support deprecated platform filter
if out.PlatformDoNotUseIt != "" {
if out.Labels == nil {
out.Labels = make(base.SliceOrMap)
out.Labels = make(map[string]string)
}
if _, set := out.Labels["platform"]; !set {
out.Labels["platform"] = out.PlatformDoNotUseIt
Expand Down
121 changes: 0 additions & 121 deletions pipeline/frontend/yaml/types/base/base_types_test.go

This file was deleted.

Loading