Support for custom non-linear interpolation/progression functions in stages #2256
Comments
|
In the purpose of documenting the issue, [here's a collection of interpolation functions] that are commonly used in the context of animation/video-games. In this context, an interpolation function is of the form: |
|
ref #2255 |
na--
added
the
evaluation needed
na--
changed the title
I think we don't need to support all of these functions in the initial implementation. We can start with just 2 or 3, e.g. the current
Another reason I don't want to focus on the other more complicated interpolation/progression functions right now is that I'm not sure we should just have a few ones like For example, if we allow users to define a function that accepts an argument
But this can get very complicated very quickly, so it's probably better left for the future. |
My initial gut feeling proposal would be to have a dedicated: as well as various implementations for the functions we want to support: (I'm not hung up on the implementation or interface itself) However, I'm not sure if I understand what you meant when you say "For example, if we allow users to define a function that accepts an argument t with values between..." @na-- Do you mean that instead of an |
|
🤷♂️ about the name, I don't have any strong opinions here. @mstoykov, @codebien, @yorugac, WDYT about
I think it's probably too premature to discuss these details if we don't plan to implement it soon, but I think we should allow for a simplified mathematical notation here, similarly to how we do so for stages: [
{target: 100, duration: '1m', progression: 'linear' }, // allow simple shortcut labels for simple cases
{target: 200, duration: '1m', progression: 'r=t' }, // this is the same as the line above
{target: 300, duration: '1m', progression: 'instant' },
{target: 400, duration: '1m', progression: 'r=1' }, // this is also the same as the line above
{target: 500, duration: '1m', progression: 'r=t*t' }, // quadratic ease in
{target: 600, duration: '1m', progression: 'r=1-(1-t)*(1-t)' }, // quadratic ease out
// ...
],
I don't think this is very flexible, we probably shouldn't work with absolute values, neither for time nor for values. If both the argument and the result are percentages (representing parts of the stage time and the stage target), the interpolation functions should be completely agnostic and usable both for VUs and for arrival rates, with each executor doing the final conversion itselft. And it will be easier to write, since you won't care about absolute values 🤷♂️ So, something like this:
|
|
On names: Also, it seems to me that from user's perspective the speed of increase and option to change that speed is more important than complex math formulae. E.g. there is apparently a term "peak testing" used in some places, as a very fast increase which corresponds to the real traffic problem. Given that, as a user I'd prefer to have a Of course, |
|
First of all, I don't want to go through discussing and making decisions if we are not actually going to be implementing this in "full". IMO if we won't implement (to some extend) user-defined functions then there are really good chances that in the future we will need to rewrite the whole thing. Having said that I also am kind of against (at least at the beginning) having names such as Further given the exact place this code stands this will definitely need a lot of benchmarks and due diligence to not worsen performance. I am actually kind of surprised by the fact that the additional switch doesn't make this way worse although it might be due to the benchmarks we have 🤷 benchmarks
Even running real tests (with impossible requirements) against master and #1391 seems to show that I get the same (empty) iterations/s around 250k, but here the fluctuation is much bigger. I would still expect that adding more stuff to this will make it slower and also x86 is really good at branch prediction (usually), while something like AWS graviton might not be as good, so I would argue we should probably benchmark this specific code even more. (also we already had problems with ARM around this parts) On the same note - there is IMO a lot of headroom 250k empty iterations is already an absurdly big amount and will get better if instead of calculating for each iteration we were doing it for number(bucket) of them as mentioned in #1386. I also do prefer if we use the word Some ... technical notes:
Said slightly differently: for the first, the rate is constant throughout a stage, while the other calculates when an iteration should start given that the rate of that changes throughout the stage. This is also why they are done in two different ways and why in practice what What I am trying to say is that the two things aren't the same, so having the same implementation for both will require some amount of mathematics to go between the two ;). And from what I can see it will be only partially used for ramping-arrival-rate detailsRamping arrival rate does already use calculus(I need to reword parts of this and probably hide the equation in a collapsable section) to calculate when the next iteration should happen. It does this (as explained there) by calculating the t(ime) at which the area "below" the graph of the particular function will be equal to 1,2,3,4,5 ... ^ stolen from the code This means that while the "ramping" for that section is Also again this just calculates how many iter/s should be started at any given time not when they should be started. To get the latter we basically need to calculate the integral of that function and then solve it when it's equal to the iteration we want to do next and we get the Given all of this, I do think to implement this we will need some more .... experiments and finding some easy (at least computationally) ways to get the integral of a function, which is definitely possible as I have done it in (strawberry) Prolog class .... 10 years ago (this looks familiar, although I have no idea what any of this does 🤣 ) and we do IMO not need to do it for any possible function just for most ones that we care about. As a whole, this was one of the many reasons why I didn't even start on it back then but just sprinkled some comments in places where it will be good idea to think about it in the future. |
|
Ah, yeah, I forgot that for I don't see why we can't start with simple "shortcut" labels like |
|
@LaserPhaser, sorry for the confusion, as you can see, we're not quite sure ourselves yet... 😅 @mstoykov and anyone else with an opinion from the @grafana/k6-devs and community, please 👍 or {👎 + an explanation} this post to signal if you're ok with this as the first step here:
WDYT? On the one hand, it seems like a simple and safe backwards-compatible change. On the other hand, it doesn't really add anything functionally - we already support 0-duration stages that serve the exact same purpose as the proposed So I'm on the fence if we should have |
|
I am against it as I mentioned above. For me this is a shortcut that requires:
The benefit is marginal as IMO we still need to document this so anyone will use it or find it, so we can just as well document the current behaviour.
|
|
Maybe I can just update the documentation in k6-docs rep, with an example how to do steps in load testing. |
|
@LaserPhaser, good point. Regardless of what we decide here, we should document the existing solution, since it seems it isn't mentioned anywhere, so I created grafana/k6-docs#516 for that. |
|
The "first step" change seems a bit like a syntactic sugar right now but as a potential benefit, it might trigger additional feedback that will in turn help flesh out any further changes here. Though as Mihail mentioned, that might turn bizarre too 😄 |
Feature Description
In the current state of things, when
stagesare defined in a load test configuration (options); the interpolation observed during both ramping up and ramping down is always linear.We think, and heard from our users, that it could also be useful to support custom non-linear interpolation functions to drive the transition between stages. Their usage could be driven by a dedicated
interpolationin thestagesobjects.As a starting set of custom interpolation function, we propose:
linear: the default, and already implemented interpolation function transitions between stages following using a linear function.exponential: drives the interpolation/transition between two stages following an exponential function.instant: drives the interpolation/transition following a square-wave like function; effectively making transitions between stages seemingly instantaneous.easeIn: drives the interpolation/transition between two stages using a function that produces a slow progression at the beginning, and a fast/abrupt one at the end. (either we use a cubic, quadratic, or sine wave function under the hood is to be decided)easeOut: drives the interpolation/transition between two stages using a function that produces a fast/abrupt progression at the beginning, and slow at the end. (either we use a cubic, quadratic, or sine wave function under the hood is to be decided)normal: see k6-jslib-utils proposalThis proposal follows and refers to:
Suggested Solution (optional)
No response
The text was updated successfully, but these errors were encountered: