[Lens] Support local variables in formula

[drewdaemon]

#153381 (comment)

Bears technical similarities to #135265

[drewdaemon]

@P1llus can you give us a couple of examples of Painless scripts you've had to rewrite as Lens formulas (or weren't able to rewrite as formulas) and describe the performance concerns?

[P1llus]

While I believe painless support was out of the question for now, let's look at some basic TSVB implementations that will either result in very big formulas, or currently not possible:

1. Anything using _interval context in painless to for example divide a value by the bucket time range size, there is already a separate tracking issue for this.

2. TSVB often have a combination of max, derivative etc stored as variables first, which is then accessed with painless. This is very useful and makes the overview much clearer.
   Since variables of any sort is not possible currently with formula we end up deeply nesting logic inside eachother, and I am unsure how this impacts performance.
   Could be maybe add a row of functions above the formula, that does not end up in the resulting visualizations, but are only accessible as variables in the formula itself? Example:
   

3. In the same example image above, this painless will most likely end up being quite nested, might still be possible, but will it even be readable? Or maintainable?
   Since we don't have variables, if you look at the screenshot + this painless part:

      if (params.deltaUsageDerivNormalizedValue > 0 && params.periodsDerivNormalizedValue >0  && params.quota > 0) {
        // if throttling is configured
        double  factor = params.deltaUsageDerivNormalizedValue / (params.periodsDerivNormalizedValue * params.quota * 1000); 

        return factor * 100; 
      }

      return null;

Since we don't have variables, and since we cannot simply stop/return on null, but rather use ifelse formulas, which again then ends up being another 3 deeper nested formulas, this will end up being huge.

[P1llus]

Talking about variables, if the UI and adding functions above the formula UI would be too much, we should be able to write multiple formulas in a single formula, storing them as temp variables. Using one of the steps on the picture above as an example:

deltaUsage = derivative(max(system.process.cgroup.somevalue))

Then later, we can simply create the formula for the painless, using deltaUsage inside it, making the formula much cleaner to read, which also helps making it prettier, ref: #153387

[dej611]

To me the main problem here sounds like the lack of variables support in Formula. Maybe we can rename the issue?

The formula above can be already written in formula, but I understand it is a bit verbose to do so:

ifelse(
  derivative(max(system.process.cgroup.somevalue)) +
  derivative(max(system.process.cgroup.someOthervalue)) +
  min(system.process.cgroup.somevalue) > 0,

  100 * derivative(max(system.process.cgroup.somevalue)) / 
  ( derivative(max(system.process.cgroup.someOthervalue)) * min(system.process.cgroup.somevalue) * 1000),

  0
)

With variables support it might be a bit easier to read, but probably it would require quite a bit work on the tinymath side of formula:

deltaUsageDerivNormalizedValue := derivative(max(system.process.cgroup.somevalue));
periodsDerivNormalizedValue := derivative(max(system.process.cgroup.someOthervalue));
quota := min(system.process.cgroup.somevalue);

ifelse(
  deltaUsageDerivNormalizedValue + periodsDerivNormalizedValue + quota > 0,
  100 * deltaUsageDerivNormalizedValue / (periodsDerivNormalizedValue * quota * 1000),
  0
)

What do you think?

[drewdaemon]

@P1llus do you agree that the main ask from your side is variables in formula?

[P1llus]

@drewdaemon I think being able to use local variables would remove the necessary nesting quite alot and make it easier to read :)

[markov00]

Closing this because it's not planned to be resolved in the foreseeable future. It will be tracked in our Icebox and will be re-opened if our priorities change. Feel free to re-open if you think it should be melted sooner.