Add token pattern (#208)

qetza · Aug 22, 2021 · c2bb80b · c2bb80b
1 parent d6dfc15
commit c2bb80b
Show file tree

Hide file tree

Showing 8 changed files with 165 additions and 55 deletions.
diff --git a/README.md b/README.md
@@ -51,8 +51,9 @@ The parameters of the task are described bellow, in parenthesis is the YAML name
   - _log warning_: the task will continue but log a warning with the missing variable name.
   - _fail_: the task will fail and log the missing variable name.
 - **Keep token** (keepToken): if checked tokens with missing variables will not be replaced by empty string.
-- **Token prefix** (tokenPrefix): the prefix of the tokens to search in the target files.
-- **Token suffix** (tokenSuffix): the suffix of the tokens to search in the target files.
+- **Token pattern** (tokenPattern): specify the pattern of the tokens to search in the target files.
+- **Token prefix** (tokenPrefix): when using `custom` token pattern, the prefix of the tokens to search in the target files.
+- **Token suffix** (tokenSuffix): when using `custom` token pattern, the suffix of the tokens to search in the target files.
 - **Use legacy pattern** (useLegacyPattern): if checked whitespaces between the token prefix/suffix and the variable name are not ignored.  
 - **Empty value** (emptyValue): the variable value that will be replaced with an empty string.
 - **Default value** (defaultValue): the value to be used if a variable is not found. Do not set to disable default value feature. (to replace with an empty string set the default value to the _Empty value_)
@@ -87,7 +88,10 @@ If you want to use tokens in XML based configuration files to be replaced during
 
 ## Release notes
 **New in 4.0.0**
-- **Breaking change**: Add output variables ([#160](https://github.com/qetza/vsts-replacetokens-task/issues/160)). (some older version of TFS/Azure Pipelines doesn't support output variables when used in release pipelines)
+- Add support for multiple task versions.
+- Add task 4.x (preview)
+  - **Breaking change**: Add output variables ([#160](https://github.com/qetza/vsts-replacetokens-task/issues/160)). (some older version of TFS/Azure Pipelines doesn't support output variables when used in release pipelines)
+  - **Breaking change**: Add dropdown parameter _Token pattern_ to select token pattern ([#131](https://github.com/qetza/vsts-replacetokens-task/issues/131)). (users with customized token pattern will need to manually select one or `custom`)
 
 **New in 3.7.1**
 - Fix issue on binary files ([#193](https://github.com/qetza/vsts-replacetokens-task/issues/193)).

diff --git a/ReplaceTokens/ReplaceTokensV3/README.md b/ReplaceTokens/ReplaceTokensV3/README.md
@@ -73,4 +73,64 @@ If you want to use tokens in XML based configuration files to be replaced during
 - setup your configuration file with local developement values
 - at deployment time
   - inject your tokens in the configuration file by using your transformation file
-  - replace tokens in your updated configuration file
+  - replace tokens in your updated configuration file
+
+## Release notes
+**New in 3.7.1**
+- Fix issue on binary files ([#193](https://github.com/qetza/vsts-replacetokens-task/issues/193)).
+- Rollback output variables ([#196](https://github.com/qetza/vsts-replacetokens-task/issues/196)).
+
+**New in 3.7.0**
+- Add output variables _tokenReplacedCount_, _tokenFoundCount_ and _fileProcessedCount_ ([#160](https://github.com/qetza/vsts-replacetokens-task/issues/160)).
+- Add support for variable transformations with _Enable tranformations_ ([#96](https://github.com/qetza/vsts-replacetokens-task/issues/96)).
+- Add default value for tokens not found with _Default value_ (contribution from [ClemensSutor](https://github.com/ClemensSutor)).
+- Group log outputs in Azure Pipelines output.
+- Add support for variables in external YAML files with `.yml` or `.yaml` extension ([#177](https://github.com/qetza/vsts-replacetokens-task/issues/177)).
+
+**New in 3.6.0**
+- Add parameter _Use legacy pattern_ with default value to `false`. 
+
+**New in 3.5.2**
+- Fix issue when token prefix present but not as a token prefix ([#149](https://github.com/qetza/vsts-replacetokens-task/issues/149)).
+
+**New in 3.5.1**
+- Fix issue when variable `System.ServerType` is not defined ([#147](https://github.com/qetza/vsts-replacetokens-task/issues/147)).
+
+**New in 3.5.0**
+- Add anonymous usage telemetry.
+- Ignore spaces between token prefix/suffix and variable name ([#143](https://github.com/qetza/vsts-replacetokens-task/issues/143)).
+
+**New in 3.4.1**
+- Fix JSON escaping of slash `/` ([#138](https://github.com/qetza/vsts-replacetokens-task/issues/138))
+
+**New in 3.4.0**
+- Add summary in logs with number of tokens found and replaced ([#126](https://github.com/qetza/vsts-replacetokens-task/issues/126)).
+- Add support for variables in external JSON files ([#113](https://github.com/qetza/vsts-replacetokens-task/issues/113)).
+
+**New in 3.3.1**
+- **Breaking change**: If you were using negative pattern you need to use the semi colon `;` as a separator instead of new-line in _Target files_.
+- Fix negative pattern support ([#127](https://github.com/qetza/vsts-replacetokens-task/issues/127)).
+
+**New in 3.3.0**
+- Add support for custom output file and wildcard support ([#114](https://github.com/qetza/vsts-replacetokens-task/issues/114)).
+
+**New in 3.2.2**
+- Fix matching issue with directory ([#122](https://github.com/qetza/vsts-replacetokens-task/issues/122)).
+
+**New in 3.2.1**
+- Fix log issue with escaped secret values.
+
+**New in 3.2.0**
+- Switch to [jschardet](https://github.com/aadsm/jschardet) for encoding detection when selecting `auto` in _File encoding_ ([#99](https://github.com/qetza/vsts-replacetokens-task/issues/99)).
+- Switch to [azure-pipelines-task-lib](https://github.com/Microsoft/azure-pipelines-task-lib) v2.8.0.
+- Add `auto` to _Escape type_ and set it as default value.
+- Move _Escape type_, _Escape character_ and _Characters to escape_ to the main paramters section for easier discoverability.
+
+**New in 3.1.0**
+- Add _Verbosity_ parameter to allow detail logs without using `system.debug`.
+
+**New in 3.0.0**
+- **Breaking change**: If you were using the character escaping feature you need to select `custom` in _Escape values type_ parameter.
+- Add support to escape JSON in variable values (contributed by Justin Gould)
+- Add support to escape XML in variable values (contributed by Justin Gould)
+- Add `Windows 1252` and `ISO 8859-1` encoding to _File encoding_ ([#67](https://github.com/qetza/vsts-replacetokens-task/issues/67))
diff --git a/ReplaceTokens/ReplaceTokensV3/task.json b/ReplaceTokens/ReplaceTokensV3/task.json
@@ -3,7 +3,7 @@
     "name": "replacetokens",
     "friendlyName": "Replace Tokens",
     "description": "Replace tokens in files",
-    "helpMarkDown": "[More Information](https://github.com/qetza/vsts-replacetokens-task#readme)",
+    "helpMarkDown": "[Learn more about this task](https://github.com/qetza/vsts-replacetokens-task/blob/master/ReplaceTokens/ReplaceTokensV3/README.md)",
     "category": "Utility",
     "visibility": [
         "Build",

diff --git a/ReplaceTokens/ReplaceTokensV4/README.md b/ReplaceTokens/ReplaceTokensV4/README.md
@@ -49,8 +49,9 @@ The parameters of the task are described bellow, in parenthesis is the YAML name
   - _log warning_: the task will continue but log a warning with the missing variable name.
   - _fail_: the task will fail and log the missing variable name.
 - **Keep token** (keepToken): if checked tokens with missing variables will not be replaced by empty string.
-- **Token prefix** (tokenPrefix): the prefix of the tokens to search in the target files.
-- **Token suffix** (tokenSuffix): the suffix of the tokens to search in the target files.
+- **Token pattern** (tokenPattern): specify the pattern of the tokens to search in the target files.
+- **Token prefix** (tokenPrefix): when using `custom` token pattern, the prefix of the tokens to search in the target files.
+- **Token suffix** (tokenSuffix): when using `custom` token pattern, the suffix of the tokens to search in the target files.
 - **Use legacy pattern** (useLegacyPattern): if checked whitespaces between the token prefix/suffix and the variable name are not ignored.  
 - **Empty value** (emptyValue): the variable value that will be replaced with an empty string.
 - **Default value** (defaultValue): the value to be used if a variable is not found. Do not set to disable default value feature. (to replace with an empty string set the default value to the _Empty value_)
@@ -81,4 +82,9 @@ If you want to use tokens in XML based configuration files to be replaced during
 - setup your configuration file with local developement values
 - at deployment time
   - inject your tokens in the configuration file by using your transformation file
-  - replace tokens in your updated configuration file
+  - replace tokens in your updated configuration file
+
+## Release notes
+**New in 4.0.0**
+- **Breaking change**: Add output variables ([#160](https://github.com/qetza/vsts-replacetokens-task/issues/160)). (some older version of TFS/Azure Pipelines doesn't support output variables when used in release pipelines)
+- **Breaking change**: Add dropdown parameter _Token pattern_ to select token pattern ([#131](https://github.com/qetza/vsts-replacetokens-task/issues/131)). (users with customized token pattern will need to manually select one or `custom`)
diff --git a/ReplaceTokens/ReplaceTokensV4/index.ts b/ReplaceTokens/ReplaceTokensV4/index.ts
@@ -483,6 +483,7 @@ async function run() {
     try {
         // load inputs
         let root: string = tl.getPathInput('rootDirectory', false, true);
+        let tokenPattern: string = tl.getInput('tokenPattern', true);
         let tokenPrefix: string = tl.getInput('tokenPrefix', true);
         let tokenSuffix: string = tl.getInput('tokenSuffix', true);
         let useLegacyPattern: boolean = tl.getBoolInput('useLegacyPattern', true);
@@ -584,6 +585,34 @@ async function run() {
         });
 
         // initialize task
+        switch (tokenPattern)
+        {
+            case 'default':
+                tokenPrefix = '#{';
+                tokenSuffix = '}#';
+                break;
+
+            case 'rm':
+                tokenPrefix = '__';
+                tokenSuffix = '__';
+                break;
+
+            case 'octopus':
+                tokenPrefix = '#{';
+                tokenSuffix = '}';
+                break;
+
+            case 'azpipelines':
+                tokenPrefix = '$(';
+                tokenSuffix = ')';
+                break;
+
+            case 'doublebraces':
+                tokenPrefix = '{{';
+                tokenSuffix = '}}';
+                break;
+        }
+
         let escapedTokenPrefix: string = tokenPrefix.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&');
         let escapedTokenSuffix: string = tokenSuffix.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&');
         let pattern = useLegacyPattern 
@@ -624,6 +653,7 @@ async function run() {
         telemetryEvent.transformSuffix = transformSuffix;
         telemetryEvent.transformPattern = transformPattern;
         telemetryEvent.defaultValue = options.defaultValue;
+        telemetryEvent.tokenPattern = tokenPattern;
 
         // process files
         rules.forEach(rule => {

diff --git a/ReplaceTokens/ReplaceTokensV4/task.json b/ReplaceTokens/ReplaceTokensV4/task.json
@@ -3,7 +3,7 @@
     "name": "replacetokens",
     "friendlyName": "Replace Tokens",
     "description": "Replace tokens in files",
-    "helpMarkDown": "[Learn more about this task](https://github.com/qetza/vsts-replacetokens-task/blob/master/ReplaceTokens/ReplaceTokensV4/README.md)\n",
+    "helpMarkDown": "[Learn more about this task](https://github.com/qetza/vsts-replacetokens-task/blob/master/ReplaceTokens/ReplaceTokensV4/README.md)",
     "category": "Utility",
     "visibility": [
         "Build",
@@ -16,7 +16,7 @@
         "Patch": 0
     },
     "preview": true,
-    "releaseNotes": "Added output variables (breaking change).",
+    "releaseNotes": "Added output variables (breaking change).<br/>Added token pattern dropdown (breaking change).",
     "instanceNameFormat": "Replace tokens in $(targetFiles)",
     "minimumAgentVersion": "2.105.0",
     "groups": [
@@ -66,6 +66,40 @@
             },
             "helpMarkDown": "Specify the files encoding.<br/>The 'auto' value will determine the encoding based on the Byte Order Mark (BOM) if present; otherwise it will use ascii."
         },
+        {
+            "name": "tokenPattern",
+            "type": "pickList",
+            "label": "Token pattern",
+            "defaultValue": "default",
+            "required": true,
+            "options": {
+                "default": "#{ ... }#",
+                "rm": "__ ... __",
+                "octopus": "#{ ... }",
+                "azpipelines": "$( ... )",
+                "doublebraces": "{{ ... }}",
+                "custom": "custom"
+            },
+            "helpMarkDown": "Specify the token pattern. Use custom to specify your own prefix and suffix"
+        },
+        {
+            "name": "tokenPrefix",
+            "type": "string",
+            "label": "Token prefix",
+            "defaultValue": "#{",
+            "required": true,
+            "visibleRule": "tokenPattern == custom",
+            "helpMarkDown": "The prefix of the tokens to search in the target files."
+        },
+        {
+            "name": "tokenSuffix",
+            "type": "string",
+            "label": "Token suffix",
+            "defaultValue": "}#",
+            "required": true,
+            "visibleRule": "tokenPattern == custom",
+            "helpMarkDown": "The suffix of the tokens to search in the target files."
+        },
         {
             "name": "writeBOM",
             "type": "boolean",
@@ -138,24 +172,6 @@
             "required": true,
             "helpMarkDown": "If checked tokens with missing variables will not be replaced by empty string."
         },
-        {
-            "name": "tokenPrefix",
-            "type": "string",
-            "label": "Token prefix",
-            "defaultValue": "#{",
-            "groupName": "advanced",
-            "required": true,
-            "helpMarkDown": "The prefix of the tokens to search in the target files."
-        },
-        {
-            "name": "tokenSuffix",
-            "type": "string",
-            "label": "Token suffix",
-            "defaultValue": "}#",
-            "groupName": "advanced",
-            "required": true,
-            "helpMarkDown": "The suffix of the tokens to search in the target files."
-        },
         {
             "name": "useLegacyPattern",
             "type": "boolean",

diff --git a/ReplaceTokens/ReplaceTokensV4/telemetry.ts b/ReplaceTokens/ReplaceTokensV4/telemetry.ts
@@ -69,7 +69,8 @@ export default function trackEvent(event: TelemetryEvent, proxyUrl?: string): st
                         transformPattern: event.transformPattern,
                         transformExecuted: event.transformExecuted,
                         defaultValue: event.defaultValue,
-                        defaultValueReplaced: event.defaultValueReplaced
+                        defaultValueReplaced: event.defaultValueReplaced,
+                        tokenPattern: event.tokenPattern
                     }
                 }
             }
@@ -172,4 +173,5 @@ export interface TelemetryEvent {
   transformExecuted: number;
   defaultValue: string;
   defaultValueReplaced: number;
+  tokenPattern: string;
 }
diff --git a/make.js b/make.js
@@ -84,25 +84,32 @@ target.build = function() {
         stage: options.stage,
         taskId: options.taskId
     };
+    extensionOptions.version = computeVersion(extensionOptions.version, 4);
 
-    if (extensionOptions.version) {
-        if (extensionOptions.version === 'auto') {
+    var extensionVersion = updateExtensionManifest(path.join(binariesPath, 'vss-extension.json'), extensionOptions);
+    console.log('  version -> ' + extensionVersion);
+}
+
+var minor = null;
+var patch = null;
+computeVersion = function(version, majorVersion) {
+    if (version) {
+        if (version === 'auto') {
             var ref = new Date(2000, 1, 1);
             var now = new Date();
-            var major = 4
-            var minor = Math.floor((now - ref) / 86400000);
-            var patch = Math.floor(Math.floor(now.getSeconds() + (60 * (now.getMinutes() + (60 * now.getHours())))) * 0.5)
-            extensionOptions.version = major + '.' + minor + '.' + patch
+            var major = majorVersion
+            minor = minor || Math.floor((now - ref) / 86400000);
+            patch = patch || Math.floor(Math.floor(now.getSeconds() + (60 * (now.getMinutes() + (60 * now.getHours())))) * 0.5)
+            version = major + '.' + minor + '.' + patch
         }
 
-        if (!semver.valid(extensionOptions.version)) {
-            console.error('build', 'Invalid semver version: ' + extensionOptions.version);
+        if (!semver.valid(version)) {
+            console.error('build', 'Invalid semver version: ' + version);
             process.exit(1);
         }
     }
 
-    var extensionVersion = updateExtensionManifest(path.join(binariesPath, 'vss-extension.json'), extensionOptions);
-    console.log('  version -> ' + extensionVersion);
+    return version;
 }
 
 buildTask = function(majorVersion) {
@@ -138,22 +145,7 @@ buildTask = function(majorVersion) {
         stage: options.stage,
         taskId: options.taskId
     };
-
-    if (taskOptions.version) {
-        if (taskOptions.version === 'auto') {
-            var ref = new Date(2000, 1, 1);
-            var now = new Date();
-            var major = majorVersion
-            var minor = Math.floor((now - ref) / 86400000);
-            var patch = Math.floor(Math.floor(now.getSeconds() + (60 * (now.getMinutes() + (60 * now.getHours())))) * 0.5)
-            taskOptions.version = major + '.' + minor + '.' + patch
-        }
-
-        if (!semver.valid(taskOptions.version)) {
-            console.error('build', 'Invalid semver version: ' + taskOptions.version);
-            process.exit(1);
-        }
-    }
+    taskOptions.version = computeVersion(taskOptions.version, majorVersion);
 
     var taskVersion = updateTaskManifest(path.join(outputPath, 'task.json'), taskOptions);
     updateTelemetryScript(path.join(outputPath, 'telemetry.js'), options.iKey, taskVersion);