Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate alternative names on secrets #3406

Merged
merged 14 commits into from
Feb 22, 2024
8 changes: 4 additions & 4 deletions docs/docs/20-usage/20-workflow-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,13 +164,13 @@ Allows you to specify the entrypoint for containers. Note that this must be a li

Woodpecker provides the ability to pass environment variables to individual steps.

For more details check the [environment docs](./50-environment.md).
For more details, check the [environment docs](./50-environment.md).

### `secrets`

Woodpecker provides the ability to store named parameters external to the YAML configuration file, in a central secret store. These secrets can be passed to individual steps of the workflow at runtime.

For more details check the [secrets docs](./40-secrets.md).
For more details, check the [secrets docs](./40-secrets.md).

### `failure`

Expand Down Expand Up @@ -574,10 +574,10 @@ For more details check the [matrix build docs](./30-matrix-workflows.md).

You can set labels for your workflow to select an agent to execute the workflow on. An agent will pick up and run a workflow when **every** label assigned to it matches the agents labels.

To set additional agent labels check the [agent configuration options](../30-administration/15-agent-config.md#woodpecker_filter_labels). Agents will have at least four default labels: `platform=agent-os/agent-arch`, `hostname=my-agent`, `backend=docker` (type of the agent backend) and `repo=*`. Agents can use a `*` as a wildcard for a label. For example `repo=*` will match every repo.
To set additional agent labels, check the [agent configuration options](../30-administration/15-agent-config.md#woodpecker_filter_labels). Agents will have at least four default labels: `platform=agent-os/agent-arch`, `hostname=my-agent`, `backend=docker` (type of the agent backend) and `repo=*`. Agents can use a `*` as a wildcard for a label. For example `repo=*` will match every repo.

Workflow labels with an empty value will be ignored.
By default each workflow has at least the `repo=your-user/your-repo-name` label. If you have set the [platform attribute](#platform) for your workflow it will have a label like `platform=your-os/your-arch` as well.
By default, each workflow has at least the `repo=your-user/your-repo-name` label. If you have set the [platform attribute](#platform) for your workflow it will have a label like `platform=your-os/your-arch` as well.

You can add additional labels as a key value map:

Expand Down
29 changes: 7 additions & 22 deletions docs/docs/20-usage/40-secrets.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,20 @@ once their usage is declared in the `secrets` section:
+ secrets: [ docker_username, docker_password ]
```

### Use secrets in settings
### Use secrets in settings and environment

Alternatively, you can get a `setting` from secrets using the `from_secret` syntax.
In this example, the secret named `secret_token` would be passed to the setting named `token`, which will be available in the plugin as environment variable named `PLUGIN_TOKEN`. See [Plugins](./51-plugins/20-creating-plugins.md#settings) for details.
You can set an setting or environment value from secrets using the `from_secret` syntax.

:::note
The `from_secret` syntax only works with the newer `settings` block.
:::
In this example, the secret named `secret_token` would be passed to the setting named `token`,which will be available in the plugin as environment variable named `PLUGIN_TOKEN` (See [plugins](./51-plugins/20-creating-plugins.md#settings) for details), and to the environment variable `TOKEN_ENV`.

```diff
steps:
- name: docker
image: my-plugin
settings:
+ environment:
+ TOKEN_ENV:
+ from_secret: secret_token
+ settings:
+ token:
+ from_secret: secret_token
```
Expand All @@ -60,21 +60,6 @@ Please note parameter expressions are subject to pre-processing. When using secr
secrets: [ docker_username, docker_password ]
```

### Alternate Names

There may be scenarios where you are required to store secrets using alternate names. You can map the alternate secret name to the expected name using the below syntax:

```diff
steps:
- name: docker
image: plugins/docker
repo: octocat/hello-world
tags: latest
+ secrets:
+ - source: docker_prod_password
+ target: docker_password
```

### Use in Pull Requests events

Secrets are not exposed to pull requests by default. You can override this behavior by creating the secret and enabling the `pull_request` event type, either in UI or by CLI, see below.
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/20-usage/50-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ Woodpecker provides the ability to pass environment variables to individual pipe
- name: build
image: golang
+ environment:
+ - CGO=0
+ - GOOS=linux
+ - GOARCH=amd64
+ CGO: 0
+ GOOS: linux
+ GOARCH: amd64
commands:
- go build
- go test
Expand Down
2 changes: 2 additions & 0 deletions docs/docs/91-migrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Some versions need some changes to the server configuration or the pipeline conf
- Removed `WOODPECKER_ROOT_PATH` and `WOODPECKER_ROOT_URL` config variables. Use `WOODPECKER_HOST` with a path instead
- Pipelines without a config file will now be skipped instead of failing
- Deprecated `includes` and `excludes` support from **event** filter
- Deprecated alternative names for secrets, use `environment` with `from_secret`
- Deprecated slice definition for labels and env vars

## 2.0.0

Expand Down
9 changes: 5 additions & 4 deletions pipeline/frontend/yaml/compiler/compiler.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ package compiler

import (
"fmt"
"maps"
"path"

backend_types "go.woodpecker-ci.org/woodpecker/v2/pipeline/backend/types"
Expand Down Expand Up @@ -179,11 +180,11 @@ func (c *Compiler) Compile(conf *yaml_types.Workflow) (*backend_types.Config, er
cloneSettings["tags"] = "true"
}
container := &yaml_types.Container{
Name: defaultCloneName,
Image: cloneImage,
Settings: cloneSettings,
Environment: c.cloneEnv,
Name: defaultCloneName,
Image: cloneImage,
Settings: cloneSettings,
}
maps.Copy(container.Environment, c.cloneEnv)
step, err := c.createProcess(container, backend_types.StepTypeClone)
if err != nil {
return nil, err
Expand Down
5 changes: 4 additions & 1 deletion pipeline/frontend/yaml/compiler/convert.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ func (c *Compiler) createProcess(container *yaml_types.Container, stepType backe

// append default environment variables
environment := map[string]string{}
maps.Copy(environment, container.Environment)
maps.Copy(environment, c.env)

environment["CI_WORKSPACE"] = path.Join(c.base, c.path)
Expand Down Expand Up @@ -112,6 +111,10 @@ func (c *Compiler) createProcess(container *yaml_types.Container, stepType backe
}
}

if err := settings.ParamsToEnv(container.Environment, environment, getSecretValue); err != nil {
return nil, err
}

for _, requested := range container.Secrets.Secrets {
secretValue, err := getSecretValue(requested.Source)
if err != nil {
Expand Down
17 changes: 17 additions & 0 deletions pipeline/frontend/yaml/linter/linter.go
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,23 @@ func (l *Linter) lintDeprecations(config *WorkflowConfig) (err error) {
}
}

for _, step := range parsed.Steps.ContainerList {
for i, c := range step.Secrets.Secrets {
if c.Source != c.Target {
err = multierr.Append(err, &errors.PipelineError{
Type: errors.PipelineErrorTypeDeprecation,
Message: "Secrets alternative names are deprecated, use environment with from_secret",
Data: errors.DeprecationErrorData{
File: config.File,
Field: fmt.Sprintf("steps.%s.secrets[%d]", step.Name, i),
Docs: "https://woodpecker-ci.org/docs/usage/workflow-syntax#event",
},
IsWarning: true,
})
}
}
}

return err
}

Expand Down
3 changes: 1 addition & 2 deletions pipeline/frontend/yaml/parse.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import (

"go.woodpecker-ci.org/woodpecker/v2/pipeline/frontend/yaml/constraint"
"go.woodpecker-ci.org/woodpecker/v2/pipeline/frontend/yaml/types"
"go.woodpecker-ci.org/woodpecker/v2/pipeline/frontend/yaml/types/base"
)

// ParseBytes parses the configuration from bytes b.
Expand Down Expand Up @@ -53,7 +52,7 @@ func ParseBytes(b []byte) (*types.Workflow, error) {
// support deprecated platform filter
if out.PlatformDoNotUseIt != "" {
if out.Labels == nil {
out.Labels = make(base.SliceOrMap)
out.Labels = make(map[string]string)
}
if _, set := out.Labels["platform"]; !set {
out.Labels["platform"] = out.PlatformDoNotUseIt
Expand Down
121 changes: 0 additions & 121 deletions pipeline/frontend/yaml/types/base/base_types_test.go

This file was deleted.

43 changes: 43 additions & 0 deletions pipeline/frontend/yaml/types/base/int_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// Copyright 2024 Woodpecker Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package base

import (
"testing"

"github.com/stretchr/testify/assert"
"gopkg.in/yaml.v3"
)

type StructStringOrInt struct {
Foo StringOrInt
}

func TestStringOrIntYaml(t *testing.T) {
for _, str := range []string{`{foo: 10}`, `{foo: "10"}`} {
s := StructStringOrInt{}
assert.NoError(t, yaml.Unmarshal([]byte(str), &s))

assert.Equal(t, StringOrInt(10), s.Foo)

d, err := yaml.Marshal(&s)
assert.NoError(t, err)

s2 := StructStringOrInt{}
assert.NoError(t, yaml.Unmarshal(d, &s2))

assert.Equal(t, StringOrInt(10), s2.Foo)
}
}
20 changes: 4 additions & 16 deletions pipeline/frontend/yaml/types/base/map.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ import (
)

// SliceOrMap represents a map of strings, string slice are converted into a map
type SliceOrMap map[string]string
type SliceOrMap map[string]any
6543 marked this conversation as resolved.
Show resolved Hide resolved

// UnmarshalYAML implements the Unmarshaler interface.
func (s *SliceOrMap) UnmarshalYAML(unmarshal func(any) error) error {
var sliceType []any
if err := unmarshal(&sliceType); err == nil {
parts := map[string]string{}
parts := map[string]any{}
for _, s := range sliceType {
if str, ok := s.(string); ok {
str := strings.TrimSpace(str)
Expand All @@ -47,21 +47,9 @@ func (s *SliceOrMap) UnmarshalYAML(unmarshal func(any) error) error {
return nil
}

var mapType map[any]any
var mapType map[string]any
if err := unmarshal(&mapType); err == nil {
parts := map[string]string{}
for k, v := range mapType {
if sk, ok := k.(string); ok {
if sv, ok := v.(string); ok {
parts[sk] = sv
} else {
return fmt.Errorf("cannot unmarshal '%v' of type %T into a string value", v, v)
}
} else {
return fmt.Errorf("cannot unmarshal '%v' of type %T into a string value", k, k)
}
}
*s = parts
*s = mapType
return nil
}

Expand Down
Loading