Add support for template strings with the remap language

[binarylogic]

I'd like to unify our templating syntax with our remap language. This was inspired by Rune's template string support. The benefits are:

1. One familiar syntax and function reference for Vector.
2. Access to all of remap's functions for templating.
3. Less code to manage.

Option questions

1. Should we still support control flow (if statements) of some sort? I could see the case for ternary like operators:

   [sinks.s3]
   type = "aws_s3"
   path = "{{ severity >= 3 ? "errors" : "logs"}}"

2. Are we better off breaking this into smaller issues? For example, can we start with the template string code and unit tests, and then follow it up by implementing it into each relevent sink?

Requirements

• Depcreate the current template syntax but do not remove it.
• Preserve backward compatibility with the current syntax. We should be able to detect this easily.
• Update documentation to reflect the new syntax.
• Only select functions and if-statements should be supported. Anything that mutates the event, like del, should not.

---

If we find there are many open questions, let's start with an RFC.

[StephenWakely]

There are two issues with this change that need to be clarified.

1. It isn't that easy to detect if the old syntax or the Remap syntax is being used:

• A simple regex scan for {{...}} isn't sufficient to detect the old syntax, as there is valid Remap code that could contain this. (eg. contains(.foo, "{{thing}}").
• We could parse the string with Remap first. If it fails to parse then we could assume it is old syntax. However, this means that we won't be able to capture and report any errors if the user intended to write a Remap template, but made an error. So del(.foo) is not valid Remap in this instance, because del is a mutable function and not allowed. Remap will not parse this, the template will then just assume this is just a constant string to use. No error will be reported.

I think the user will need to explicitly state which syntax they are using in the configuration.

2. The Loki sink wants a list of the fields that were used in the template. It uses this list to remove these fields from the event sent to Loki. With Remap this list becomes dynamic. So the script if .foo { .bar } else { .baz } means sometimes .bar is used and other times .baz is used.
   We need to decide how this change would be able to accomodate this. Should the parser return all fields used in the script and the Loki sink removes them all? Should it return the fields used directly each time it is run? Or can we remove this feature - it is a regression, but perhaps it isn't that important and the user can fall back to the old syntax if they need it?

I think this needs an RFC.

[binarylogic]

I think this needs an RFC.

Go for it. Would you mind opening a separate issue for the RFC work specifically?

[binarylogic]

Closing, please see #4905 (comment) for reasoning.