-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New executors #1007
Merged
Merged
New executors #1007
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
golangcibot
reviewed
Apr 23, 2019
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
mstoykov
reviewed
May 8, 2019
mstoykov
reviewed
May 9, 2019
mstoykov
reviewed
May 9, 2019
mstoykov
reviewed
May 9, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 14, 2019
mstoykov
reviewed
May 15, 2019
mstoykov
reviewed
May 15, 2019
mstoykov
reviewed
May 15, 2019
mstoykov
reviewed
May 15, 2019
mstoykov
reviewed
May 15, 2019
mstoykov
reviewed
May 16, 2019
Closed
golangcibot
reviewed
May 17, 2019
imiric
approved these changes
Jul 6, 2020
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😅 🎉 👏
Some flaky failures still, but we'll get them in #1357.
mstoykov
approved these changes
Jul 6, 2020
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still think this is way too big ... but it ain't gonna get smaller so ... merge it
This was referenced Jul 6, 2020
imiric
pushed a commit
that referenced
this pull request
Jul 7, 2020
imiric
pushed a commit
that referenced
this pull request
Jul 7, 2020
1170 is not a bug, and the other two are already mentioned.
imiric
pushed a commit
that referenced
this pull request
Jul 13, 2020
imiric
pushed a commit
that referenced
this pull request
Jul 13, 2020
1170 is not a bug, and the other two are already mentioned.
imiric
pushed a commit
that referenced
this pull request
Jul 14, 2020
* WIP Add v0.27.0 release notes * Address some PR comments * Address more PR comments * Reference bugs fixed by #1007 * Actually, remove mention of bugs squashed by #1007 1170 is not a bug, and the other two are already mentioned. * Apply suggestions from code review Co-authored-by: na-- <n@andreev.sh> * Minor cleanup, address some PR comments * Add scenario example to v0.27.0 release notes * Add links to issues and PRs, remove 'closes' Resolves #1507 (comment) * Mention WS hanging fix in release notes Resolves #1507 (comment) * Update scenarios docs link * Link to 'Breaking changes' section * Move gracefulStop breaking change higher up See #1507 (comment) * Mention new dropped_iterations metric in release notes Resolves https://github.com/loadimpact/k6/pull/1507/files#r452789367 * Mention more resolved issues in the v0.27.0 release notes * Apply suggestions from code review Co-authored-by: na-- <n@andreev.sh> * Update release notes/v0.27.0.md Co-authored-by: na-- <n@andreev.sh> * Apply suggestions from code review Co-authored-by: na-- <n@andreev.sh> Co-authored-by: na-- <n@andreev.sh>
This was referenced Jan 19, 2023
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request totally rewrites the old local executor and a bunch of other things. It builds on #913 and introduces quite a lot of new functionality, but it also has some minor breaking changes.
New features and changes
New executors
The biggest change in this PR is the introduction of 7 distinct new VU executors:
shared-iterations
, a.k.a.iterations
+vus
.constant-vus
, a.k.a.duration
+vus
.ramping-vus
, a.k.a.stages
+vus
.externally-controlled
is similar toramping-vus
, but the number of initialized and active VUs at any given time can be controlled by the k6 REST API - it's the only executor that could be paused or controlled while it's running (closes Rare panic(s) in goja #867, closes K6 scale not works with stage #1028).per-vu-iterations
is а new feature that allows each configured VU to execute an exact number of iterations; the shared iterations have caused confusion before and there are legitimate reasons people want an equal number of iterations for each VU (closes Make iterations a per-VU parameter? #381, Websocket loop executing only once. #663, Websocket requests are not being executed concurrently #627,iterations
does not work whenvus
is set #378, and probably others).constant-arrival-rate
executes a constant number of iterations per second (closes Arrival-rate based VU executor #550).ramping-arrival-rate
executes a variable number of iterations per second, specified as ramp-ups or ramp-downsstages
, similar to how theramping-vus
is configured.These executors can be configured via the new
execution
JS/JSON option. For now, they aren't directly configurable via CLI flags or environment variables.However, most of the old simple k6 uses where
stages
, oriterations
orduration
are specified in any way, including via CLI flags and env vars, are still supported. They are just transparently converted to their respective newexecution
executor configurations. There are some breaking changes here, described in detail in the Breaking Changes section below, but generally only when there is a conflict or ambiguity in the specified configuration.Other new features
startTime
property, which defines at what time relative to the test run start it will start executing.gracefulStop
property that allows for iterations to complete gracefully for some amount of time after the normal executor duration is over (closes Uninterruptible iterations #879); even withgracefulStop=0
, i.e. the old behavior, metrics are emitted and processed in a much saner way (closes Mismatch in check counts in the end-of-test summary #1033)default
exported one.Example of the new config
Breaking changes
There are a few breaking changes caused by this PR:
execution
,stages
,iterations
,duration
) from "upper" config layers overwrite execution options from "lower" (i.e. CLI flags > environment variables > JS options > JSON options) config layers. For example, the--iterations
CLI flag will overwrite any execution options set as environment variables (e.g.K6_DURATION
,K6_STAGES
, etc.) or script options (stages: [ /* ... */]
,execution: { /* ... */ }
, etc. ).k6 run --duration 10s --stages 5s:20 script.js
won't work (closes Stuck in infinite loop when specifying iterations and VUs #812). The only exception is combiningduration
anditerations
, which will result in ashared-iterations
executor with the specified non-defaultmaxDuration
( Specifying both duration and iterations is deprecated and won't be supported in the future k6 versions #1058).k6 pause
,k6 scale
commands) now only works when aexternally-controlled
executor is configured in theexecution
config. The initial pausing of a test (i.e.k6 run --paused script.js
) still works with all executor types, but once the test is started withk6 resume
(or the corresponding REST API call), it can't be paused again unless only theexternally-controlled
executor is used.k6 run --paused script.js
would have still executed the script'ssetup()
function (if it was present and wasn't explicitly disabled with--no-setup
) and paused immediately after. Now, k6 will pause before it executessetup()
.vusMax
/K6_VUS_MAX
/-m
/--max
option is deprecated - it was previously used for the control of the initialized VUs by the REST API. Since that has now been restricted to theexternally-controlled
executor, the equivalent option there is calledmaxVUs
.externally-controlled
executor.duration
expired, or when VUs were ramped down in a stage, any running iterations were interrupted. Now all executors besides theexternally-controlled
one have agracefulStop
period of30s
by default (closes Useduration
run,Thehttp_reqs
value of K6 is different from the service real request count value. #898). Additionally,ramping-vus
executor has agracefulRampDown
parameter that configures the ramp-down grace period. For those periods, no new iterations will be started by the executors, but any currently running iterations will be allowed up to the specified periods to finish their execution.Broken things and TODOs that remain be fixed in this PR:
externally-controlled
executor's parameters via the REST APIarrival-rate
distribution bugs, especially when there are execution segments: New executors #1007 (comment)k6 scale
command to theexternally-controlled
executor is runningramping-arrival-rate
bug that @mstoykov foundERRO context cancelled
message error message that's sometimes shown when an iteration is interruptedk6 inspect
doesn't fail when there are validation errors in optionsREADME
TODO
s in the code that will be handled in future pull requests, i.e. for now only issues will be created:tag metrics from different executors differently(already described in Tag metrics from different stages #796)execution
configuration, but with a better user experience than manually writing it as a JSONThings that need to be done shortly before or after this PR is merged in
master
: