Replace pipeline stage

[cyriltovena]

Is your feature request related to a problem? Please describe.

Currently it's difficult to replace password or unwanted data in a log line.
This is mainly because you have to matches everything else than what you want to replace in a regexp stage and use that in a template stage. Quite difficult to get right.

Describe the solution you'd like
I'm suggesting we add a new pipeline stages named replace that will replace matches from a regex with string.

replace:
  # The RE2 regular expression. Each capture group will be replaced.
  expression: <string>

  # Name from extracted data to parse. If empty, uses the log message.
  [source: <string>]
  
  # The string to replace matches with.
  [replacement: <string>]

Alternatively replacement could be a template like template stage for more complex scenario.

Additional context
Couple of request in slack for this.

/cc @slim-bean @owen-d @sandeepsukhani WDYT ?

[adityacs]

Can we make this substage to regex? Outside of regex stage it will be duplicate of template stage except for accepting expression. May be we can just call template process from the regex stage if above configuration is defined.

[cyriltovena]

Could you show an example?

[adityacs]

It would be same as the one you have shown above. Just that instead of defining one more stage we can use the existing regex stage with an extra check to see if there is replace field

Default regex stage.

Here all captured group will be set to extracted.

regex:
  expression: <string>

  [source: <string>]

regex stage with replace field.

Here we check if replace field is present. If yes, we replace all captured group value with the value given in replace field and then set to extracted.

regex:
  expression: <string>

  [source: <string>]

  [replace: <string>]

The check can be made here and we can call the template process() in this loop

loki/pkg/logentry/stages/regex.go

Line 127 in cc98d10

extracted[name] = match[i]

template stage itself sets the extracted so we can ignore the above one from regex stage