Replace Tokens task

Visual Studio Team Services Build and Release extension that replace tokens in files with variable values.

Usage

If you are using the UI, add a new task, select Replace Tokens from the Utility category and configure it as needed:

If your are using a YAML file, add a task with the following syntax:

- task: qetza.replacetokens.replacetokens-task.replacetokens@3
  displayName: 'Replace tokens'
  inputs:
    targetFiles: |
      **/*.config
      **/*.json => outputs/*.json

Parameters include (in parenthesis the yaml name):

• Root directory (rootDirectory): the base directory for searching files. If not specified the default working directory will be used.
• Target files (targetFiles): the absolute or relative newline-separated paths to the files to replace tokens. Wildcards can be used (eg: **\*.config for all .config files in all sub folders).

Syntax: {file path}[ => {output path}]

• web.config will replace tokens in web.config and update the file.
• web.tokenized.config => web.config will replace tokens in web.tokenized.config and save the result in web.config.
• config\web.tokenized.config => c:\config\web.config will replace tokens in config\web.tokenized.config and save the result in c:\config\web.config.

Wildcard support

• *.tokenized.config => *.config will replace tokens in all {filename}.tokenized.config target files and save the result in {filename}.config.
• **\*.tokenized.config => c:\tmp\*.config will replace tokens in all {filename}.tokenized.config target files and save the result in c:\tmp\{filename}.config.

Only the wildcard * in the target file name will be used for replacement in the output.
Relative paths in the output pattern are relative to the target file path.\

Negative pattern
If you want to use negative pattern in target file, use a semi-colon ; to separate the including pattern and the negative patterns. When using output syntax, only the wildcard in the first pattern will be used for generating the output path.

• **\*.tokenized.config;!**\dev\*.config => c:\tmp\*.config will replace tokens in all {filename}.tokenized.config target files except those under a dev directory and save the result in c:\tmp\{filename}.config.

• Files encoding (encoding): the files encoding used for reading and writing. The 'auto' value will determine the encoding based on the Byte Order Mark (BOM) if present; otherwise it will use ascii.
• Write unicode BOM (writeBOM): if checked writes an unicode Byte Order Mark (BOM).
• Escape type (escapeType): specify how to escape variable values. Value auto uses the file extension (.json and .xml) to determine the escaping and none as fallback.
• Escape character (escapeChar): when using custom escape type, the escape character to use when escaping characters in the variable values.
• Characters to escape (charsToEscape): when using custom escape type, characters in variable values to escape before replacing tokens.
• Verbosity (verbosity): specify the level of log verbosity. (note: error and system debug are always on)
• Action (actionOnMissing): specify the action to take on a missing variable.
  • silently continue: the task will continue without displaying any message.
  • log warning: the task will continue but log a warning with the missing variable name.
  • fail: the task will fail and log the missing variable name.
• Keep token (keepToken): if checked tokens with missing variables will not be replaced by empty string.
• Token prefix (tokenPrefix): the prefix of the tokens to search in the target files.
• Token suffix (tokenSuffix): the suffix of the tokens to search in the target files.
• Use legacy pattern (useLegacyPattern): if checked whitespaces between the token prefix/suffix and the variable name are not ignored.
• Empty value (emptyValue): the variable value that will be replaced with an empty string.
• Variable files (JSON) (variableFiles): the absolute or relative comma or newline-separated paths to the files containing additional variables. Wildcards can be used (eg: vars\**\*.json for all .json files in all sub folders of vars). Variables declared in files overrides variables defined in the pipeline.
• Variable separator (variableSeparator): the separtor to use in variable names for nested objects and arrays in variable files. Example: { 'My': { 'Value': ['Hello World!'] } } will create a variable My.Value.0 with the value Hello World!.
• Send anonymous usage telemetry (enableTelemetry): if checked anonymous usage data (hashed collection and pipeline id, no file parameter values, no variable values) will be sent to the task author only to analyze task usage.

Data/Telemetry

The Replace Tokens task for Azure Pipelines collects anonymous usage data and sends them to its author to help improve the product. If you don’t wish to send usage data, you can change your telemetry settings through Send anonymous usage telemetry parameter or by setting a variable or environment variable REPLACE_TOKENS_DISABLE_TELEMETRY to true.

Tips

If you want to use tokens in XML based configuration files to be replaced during deployment and also have those files usable for local development you can combine the Replace Tokens task with the XDT tranform task:

• create an XDT transformation file containing your tokens
• setup your configuration file with local developement values
• at deployment time
  • inject your tokens in the configuration file by using your transformation file
  • replace tokens in your updated configuration file

Release notes

New in 3.6.0

• Add parameter Use legacy pattern with default value to false.

New in 3.5.2

• Fix issue when token prefix present but not as a token prefix (#149).

New in 3.5.1

• Fix issue when variable System.ServerType is not defined (#147).

New in 3.5.0

• Add anonymous usage telemetry.
• Ignore spaces between token prefix/suffix and variable name (#143).

New in 3.4.1

• Fix JSON escaping of slash / (#138)

New in 3.4.0

• Add summary in logs with number of tokens found and replaced (#126).
• Add support for variables in external JSON files (#113).

New in 3.3.1

• Breaking change: If you were using negative pattern you need to use the semi colon ; as a separator instead of new-line in Target files.
• Fix negative pattern support (#127).

New in 3.3.0

• Add support for custom output file and wildcard support (#114).

New in 3.2.2

• Fix matching issue with directory (#122).

New in 3.2.1

• Fix log issue with escaped secret values.

New in 3.2.0

• Switch to jschardet for encoding detection when selecting auto in File encoding (#99).
• Switch to azure-pipelines-task-lib v2.8.0.
• Add auto to Escape type and set it as default value.
• Move Escape type, Escape character and Characters to escape to the main paramters section for easier discoverability.

New in 3.1.0

• Add Verbosity parameter to allow detail logs without using system.debug.

New in 3.0.0

• Breaking change: If you were using the character escaping feature you need to select custom in Escape values type parameter.
• Add support to escape JSON in variable values (contributed by Justin Gould)
• Add support to escape XML in variable values (contributed by Justin Gould)
• Add Windows 1252 and ISO 8859-1 encoding to File encoding (#67)

New in 2.3.0

• Add support to escape characters in variable values (#52)

New in 2.2.1

• Fix issue with backslash in default target files value on mac (#50)

New in 2.2.0

• Fix issue on file not found when using network paths (#40, #41).

New in 2.1.0

• Add support for comma-separator in Target files (#35).
• Add Empty value parameter to allow token replacement with an empty string (#32).

New in 2.0.2

• Fix invalid file permissions after saving files.

New in 2.0.0

• Breaking change: Migrated code to typescript to support cross-platform agent. This change requires the use of an agent at least in version 2.105.0 which is not compatible with TFS 2015. If you need to install the task on TFS 2015, download the vsix from the repository: https://github.com/qetza/vsts-replacetokens-task/releases/download/v1.4.1/qetza.replacetokens-1.4.1.vsix
• Breaking change: File encoding parameter is now used when reading and writing files. Previously it was only used when writing.
• Breaking change: File encoding doesn't support 'utf-32' and 'utf-32 (big endian)' anymore.
• Breaking change: Target files parameter now only uses the new line as a separator for multi-values (previously it used new-line and semi-colon).
• Removed required Root directory, an empty value is equivalent to $(System.DefaultWorkingDirectory).

New in 1.4.1

• Fix missing method issue with new xplat agent (2.104.1)

New in 1.4.0

• Add variables expansion in variable values.
• Escape token prefix and suffix in regex pattern.

New in 1.3.1

• Fix wrong encoding constructors parameters.

New in 1.3.0

• Replaced parameter Fail on missing with Action in Missing variables group.
• Add Keep token parameter in 'Missing variables' group.
• Fix issue on empty file.

New in 1.2.0

• Add Root directory task parameter to configure file search root directory (contributed by Jesse Houwing).
• Update Target files task parameter to support newline-separator (contributed by Jesse Houwing).