Skip to content

Latest commit

 

History

History
215 lines (167 loc) · 9.77 KB

DOCS.md

File metadata and controls

215 lines (167 loc) · 9.77 KB
Raw

Description

This plugin enables you to cache build resources in an s3 compatible store for a Vela pipeline.

Source Code: https://github.com/go-vela/vela-s3-cache

Registry: https://hub.docker.com/r/target/vela-s3-cache

Usage

NOTE:

Users should refrain from using latest as the tag for the Docker image.

It is recommended to use a semantically versioned tag instead.

Sample of restoring a cache:

steps:
  - name: restore_cache
    image: target/vela-s3-cache:latest
    pull: always
    parameters:
      action: restore
      bucket: mybucket
      server: mybucket.s3-us-west-2.amazonaws.com

Sample of rebuilding a cache:

steps:
  - name: rebuild_cache
    image: target/vela-s3-cache:latest
    pull: always
    parameters:
      action: rebuild
      bucket: mybucket
      server: mybucket.s3-us-west-2.amazonaws.com
      mount:
        - .gradle

Sample of rebuilding a cache while preserving the directory structure:

steps:
  - name: rebuild_cache
    image: target/vela-s3-cache:latest
    pull: always
    parameters:
      action: rebuild
      bucket: mybucket
      server: mybucket.s3-us-west-2.amazonaws.com
      preserve_path: true
      mount:
        - foo/test1
        - bar/test2

Sample of flushing a cache:

steps:
  - name: flushing_cache
    image: target/vela-s3-cache:latest
    pull: always
    parameters:
      action: flush
      bucket: mybucket
      server: mybucket.s3-us-west-2.amazonaws.com

Secrets

NOTE: Users should refrain from configuring sensitive information in your pipeline in plain text.

Internal

Users can use Vela internal secrets to substitute these sensitive values at runtime:

steps:
  - name: restore_cache
    image: target/vela-s3-cache:latest
    pull: always
+   secrets: [ s3_cache_access_key, s3_cache_secret_key ]
    parameters:
      action: restore
      bucket: mybucket
      server: mybucket.s3-us-west-2.amazonaws.com
-     access_key: AKIAIOSFODNN7EXAMPLE
-     secret_key: 123456789QWERTYEXAMPLE

This example will add the secrets to the restore_cache step as environment variables:

  • S3_CACHE_ACCESS_KEY=<value>
  • S3_CACHE_SECRET_KEY=<value>

External

The plugin accepts the following files for authentication:

Parameter Volume Configuration
access_key /vela/parameters/s3-cache/access_key, /vela/secrets/s3-cache/access_key
secret_key /vela/parameters/s3-cache/secret_key, /vela/secrets/s3-cache/secret_key
session_token /vela/parameters/s3-cache/session_token, /vela/secrets/s3-cache/session_token

Users can use Vela external secrets to substitute these sensitive values at runtime:

steps:
  - name: restore_cache
    image: target/vela-s3-cache:latest
    pull: always
+   secrets: [ s3_cache_access_key, s3_cache_secret_key ]
    parameters:
      action: restore
      bucket: mybucket
      server: mybucket.s3-us-west-2.amazonaws.com
-     access_key: AKIAIOSFODNN7EXAMPLE
-     secret_key: 123456789QWERTYEXAMPLE

This example will read the secret values in the volume stored at /vela/secrets/

Parameters

NOTE:

The plugin supports reading all parameters via environment variables or files.

Any values set from a file take precedence over values set from the environment.

The s3 bucket set with the bucket parameter is expected to be created beforehand.

The following parameters can used to configure all image actions:

Name Description Required Default Environment Variables
accelerated_endpoint s3 accelerated instance to communicate with false N/A PARAMETER_ACCELERATED_ENDPOINT
S3_CACHE_ACCELERATED_ENDPOINT
access_key access key for communication with s3 true N/A PARAMETER_ACCESS_KEY
S3_CACHE_ACCESS_KEY
AWS_ACCESS_KEY_ID
action action to perform against s3 true N/A PARAMETER_ACTION
S3_CACHE_ACTION
build_branch branch name from build for the repository false set by Vela PARAMETER_BUILD_BRANCH
VELA_BUILD_BRANCH
bucket name of the s3 bucket true N/A PARAMETER_BUCKET
S3_CACHE_BUCKET
log_level set the log level for the plugin false info PARAMETER_LOG_LEVEL
S3_CACHE_LOG_LEVEL
org name of the org for the repository true set by Vela PARAMETER_ORG
VELA_REPO_ORG
path custom path for the object(s) false N/A PARAMETER_PATH
S3_CACHE_PATH
prefix path prefix for the object(s) false N/A PARAMETER_PREFIX
S3_CACHE_PREFIX
repo name of the repository true set by Vela PARAMETER_REPO
VELA_REPO_NAME
repo_branch default branch for the Vela repository false set by Vela PARAMETER_REPO_BRANCH
VELA_REPO_BRANCH
secret_key secret key for communication with s3 true N/A PARAMETER_SECRET_KEY
S3_CACHE_SECRET_KEY
AWS_SECRET_ACCESS_KEY
server s3 instance to communicate with true N/A PARAMETER_SERVER
S3_CACHE_SERVER
session_token session token for communication with s3 true N/A PARAMETER_SESSION_TOKEN
S3_CACHE_SESSION_TOKEN
AWS_SESSION_TOKEN

Restore

The following parameters are used to configure the restore action:

Name Description Required Default Environment Variables
filename the name of the cache object true archive.tgz PARAMETER_FILENAME
S3_CACHE_FILENAME
timeout the timeout for the call to s3 false 10m PARAMETER_TIMEOUT
S3_CACHE_TIMEOUT

Rebuild

The following parameters are used to configure the rebuild action:

Name Description Required Default Environment Variables
filename the name of the cache object true archive.tgz PARAMETER_FILENAME
S3_CACHE_FILENAME
timeout the timeout for the call to s3 false 10m PARAMETER_TIMEOUT
S3_CACHE_TIMEOUT
preserve_path whether to preserve the relative directory structure during the tar process false false PARAMETER_PRESERVE_PATH
S3_PRESERVE_PATH
mount the file or directories locations to build your cache from true N/A PARAMETER_MOUNT
S3_CACHE_MOUNT

Flush

The following parameters are used to configure the flush action:

Name Description Required Default Environment Variables
age delete the objects past a specific age (i.e. 60m, 8h) false 336h PARAMETER_AGE
S3_CACHE_AGE

Template

COMING SOON!

Troubleshooting

You can start troubleshooting this plugin by tuning the level of logs being displayed:

steps:
  - name: restore_cache
    image: target/vela-s3-cache:latest
    pull: always
    parameters:
      action: restore
      bucket: mybucket
+     log_level: trace
      server: mybucket.s3-us-west-2.amazonaws.com

Below are a list of common problems and how to solve them:

Invalid duration value

The error may look like this:

could not parse \"14d\" as duration value for flag flush.age: time: unknown unit \"d\" in duration \"14d\"

Values for rebuild and restore timeout and flushing age are parsed using Go's time.ParseDuration function. Only h for hours, m for minutes, and so on for smaller time units are supported; d for days will cause an error unless added in subsequent versions of Go after v1.16, which is unlikely.