Recursive Token Replacement

[stoopered]

If you have a variable library with the values:
EnvName: Dev
TestValue: #{lower(EnvName)}

test.txt would have
#{TestValue}

To Replicate/Test:

1. Create a release with 2 variables
   • EnvName=Dev
   • TestValue=#{lower(EnvName)}
2. Add build steps
   • Inline Bash that writes the test file: echo "#{TestValue}" > test.yaml
   • Replace Tokens that target test.yaml, token prefix: #{, token suffix: }, enables transformations leave transform prefix and suffix
   • Inline Bash to print test file: cat test.yaml

Current result: #{lower(EnvName)}

Expected result: dev

Right now the Replace Token step needs to be run multiple times to achieve the correct value. But depending on the situation, you may not know how many times to run it if you are doing multi-level replacements. It should run until no eligible replacements are found.

Solution: Add a check box to run recursive to replace nested values.

[qetza]

Hi @stoopered,
Thanks for the feedback. Currently the variable value is just injected in place of the token, if i have some time i will see if it would be possible to easily support recursion on variable values.

[stoopered]

Thank you!
Yeah this would cut my libraries/variables in half.

FYI your tool is awesome, and I have been using it for years!
Sent a small donation. Keep up the good work

[henry-padilla]

+1
Literally just came up against this in my deployments.
We have several dev environments for each team. We have the passwords in a key vault named per the environment name. i.e. dev03-password, dev04-username, etc.
This leaves me with -
#{#{EnvironmentName}#-password}#
#{#{EnvironmentName}#-username}#

Currently, like stoopered, I have to run two passes: the first to replace #{EnvironmentName}# and a second to replace #{dev03-password}# with the value.

This would be so valuable.

[henry-padilla]

I see the merge and I tried replaceTokens@5 (and the enableRecursion: true) but I don't get the recursive replacement. Is there something I need to do for this?

[qetza]

Hi @henry-padilla,
That's because i haven't released a new version yet, i want to include another feature before doing a new release.

[qetza]

Hi @henry-padilla,
The new version with recursion is now deployed. You can enable recursion by setting the input enableRecursion to true.

[henry-padilla]

I'm still not seeing recursive replacement.

I am using the following task:

        - task: qetza.replaceTokens.replaceTokens-task.replaceTokens@5
          inputs:
            targetFiles: '$(Build.SourcesDirectory)\ ... \PrepScripts\**\*.*'
            enableRecursion: true
            keepToken: true
            encoding: 'auto'
            writeBOM: true
            actionOnMissing: 'warn'
            tokenPrefix: '#{'
            tokenSuffix: '}#'
          displayName: Replace Tokens in PrepScripts

In my variables I have the following set:

- name: EnvironmentName
  value: dev20

I have secrets in the key vault:

dev20-ASM-Password
dev21-ASM-Password
dev22-ASM-Password

In the PSScript file I have the following:

$localAsmPassword = '#{#{EnvironmentName}#-ASM-Password}#'
$AppSvcPassword = '#{#{EnvironmentName}#-AppSvc-Password}#'

I need the task to replace #{EnvironmentName}# with 'dev20' then replace #{dev20-ASM-Password}# with the value from the key vault.

Am I doing this right?

[qetza]

Hi @henry-padilla,
The recursion is applied on the variable value not on the variable name which cannot be done because this kind of parsing is not possible with regular expression (nested match).

What you can do is mixed azure pipelines recursive variables and replace tokens recursive variables like this:

variables:

- name: EnvironmentName
  value: dev20
- name: ASM-Password
  value: #{$(EnvironmentName)-ASM-Password}#
- name: AppSvc-Password
  value: #{$(EnvironmentName)-AppSvc-Password}#

file:

$localAsmPassword = '#{ASM-Password}#'
$AppSvcPassword = '#{AppSvc-Password}#'

[henry-padilla]

That did it!
Thank you so much.
Sorry for doubting you.