Skip to content

Releases: grafana/k6

v0.38.3

18 May 07:35
Compare
Choose a tag to compare

k6 v0.38.3 is a patch release containing a single fix

Threshold over already defined sub-metrics will result in an error (#2538)

There was a bug where we were checking if a submetric had already been added. Unfortunately, we didn't check that this will work with the one submetric we have by default http_req_duration{expected_response:true}. After v0.38.0 defining a threshold on it would result in an error.

As this definitely shouldn't happen in that case and we don't see a particular case where that will be problematic - adding a submetric again just reuses the already added one instead.

This issue has been addressed in #2539, and k6 v0.38.3 will now lead you add a threshold on http_req_duration{expected_response:true}.

v0.38.2

11 May 13:55
5c35728
Compare
Choose a tag to compare

k6 v0.38.2 is a patch release containing a couple of bugfixes!

Threshold over sub-metrics without samples would result in NaN (#2520)

There was a bug in thresholds applied to sub-metrics set to abortOnFail: leading k6 to evaluate thresholds that would have likely aborted before they had a chance of passing (because no samples for the given metric were recorded yet). This bug would have led to such thresholds' results value to be NaN rather than a numerical value. The following script, for instance:

import { check, sleep } from 'k6';

import http from 'k6/http';

export const options = {
  scenarios: {
    iWillFail: {
      exec: 'iWillFail',
      executor: 'constant-vus',
      startTime: '2s',
      vus: 1,
      duration: '30s',
    },
  },

  thresholds: {
    'checks{type:read}': [{ threshold: 'rate>0.9', abortOnFail: true }],
  },
};

export function iWillFail() {
  let res = http.get(`https://test-api.k6.io/`);

  check(res, {
    'read status is 200': (r) => r.status === 200,
  }, { type: 'read' });

  sleep(1);
}

Would result in the following:

✗ { type:read }...: NaN% ✓ 0 ✗ 0  
vus...............: 0 min=0 max=0
vus_max...........: 1 min=1 max=1

This issue was introduced by recent changes to how we handle thresholds in the k6 engine and is now addressed in v0.38.2.

Sub-metrics without values rendered below an incorrect parent metric (#2518)

There was in how thresholds over sub-metrics that didn't receive any samples would be displayed under an incorrect parent metric. For instance, the following script:

import { Counter } from 'k6/metrics';

const counter1 = new Counter("one");
const counter2 = new Counter("two");

export const options = {
    thresholds: {
        'one{tag:xyz}': [],
    },
};

export default function() {
    console.log('not submitting metric1');
    counter2.add(42);
}

Would have led to the following output, where the {tag:xyz} sub-metric is displayed under iterations instead of one:

data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
iterations...........: 1 499.950005/s
  { tag:xyz }........: 0 0/s
two..................: 42 20997.90021/s

When we would have expected it to produce:

one..................: 0 0/s
  { tag:xyz }........: 0 0/s
two..................: 42

This issue has been addressed in #2519, and k6 v0.38.2 now displays sub-metrics under their actual parents, even when they have received no samples.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

v0.38.1

06 May 09:02
44cc995
Compare
Choose a tag to compare

k6 v0.38.1 is a patch release containing a bugfix!

Threshold sub-metric selectors containing reserved symbols would fail (#2512)

There was a bug in threshold sub-metric selector parsing, which led to errors when users would use specific symbols such as {, } or : as part of their definitions. For instance, thresholds used for sub-metrics with URL Grouping like http_req_duration{name:"http://example.com/${}"} would have led to failures in v0.38.0.

The error messages for invalid metric, sub-metric and threshold definitions were also improved.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

v0.38.0

05 May 07:27
0835311
Compare
Choose a tag to compare

k6 v0.38.0 is here! 🎉

New Features!

AWS JSLib

There's a new addition to the officially supported k6 JavaScript libraries: k6-jslib-aws.
This library lets users interact with a selection of AWS services directly from their scripts. The library currently implements support for the S3 and the Secrets Manager services.

The AWS JS lib documentation has examples and details on how to use the library in your scripts.

Accessing the consolidated and derived options from the default function (#2493)

The k6/execution core module now lets you access the consolidated and derived options that k6 computed before it started executing the test. You can access consolidated options through the exec.test.options property. Note that consolidated options are frozen and cannot be modified. The k6 execution module's documentation has examples and details on how to use the functionality in your scripts.

import exec from "k6/execution";

export const options = {
    vus: 10,
    duration: "30s",
};

export default function () {
    console.log(exec.test.options.scenarios.default.vus); // 10
}

Tagging metric values with the current scenario stage

With the new consolidated script options, we've added a few helper functions to the k6-jslib-utils library. You can use them to automatically tag all the emitted metric samples by k6 with the currently running stage.

The k6 documentation has examples and details on how to use it.

Dumping SSL keys to an NSS formatted key log file (#2487)

This release adds the ability to dump SSL keys while making TLS connections.
You then can use these keys to decrypt all traffic that k6 generates.

To accomplish this, set the SSLKEYLOGFILE environment variable to some file path and run k6.
This will populate the file with the keys.
Then you can use Wireshark to capture the traffic, decrypt it, and use that for debugging.

Here's an example that uses curl to inspect TLS traffic.

Breaking Changes

console methods now pretty print objects and arrays (2375)

For convenience, all console methods such as console.log() and console.info() will now automatically JSON.stringify() objects and arrays passed to them. Thus, instead of console.log({'foo': 'bar'}) printing [object Object], it will now print {'foo': 'bar'}, which will make the experience of debugging k6 scripts easier and more intuitive.

To achieve the previous behavior, cast the Object to a String, as in console.log(String({'foo': 'bar'})).

export default function () {
    console.log([1, 2, "test", ["foo", "bar"], { user: "Bob" }]);
    // before: 1,2,test,foo,bar,[object Object]
    // after: [1,2,"test",["foo","bar"],{"user":"Bob"}]
}

The Go types in the stats package were moved to the metrics package #2433

For convenience and to facilitate further developments, the types and functionalities that used to live in k6's stats package have been moved to the metrics package. The stats package is, as of v0.38.0, removed in favor of the metrics package. Besides, #2442 removed the stats.New function in favor of initializing new metric via a register.NewMetric call instead.

Deprecation

  • #2499 removed support for the deprecated maxVUs option.
    It had been removed in k6 v0.27.0, however using the CLI flag resulted only in a deprecation warning.
    Now, using this flag will generate an error.
  • This release drops some leftovers from the previous version of our JS module Go APIs. As of v0.38.0, these are now unsupported:
    • The deprecated common.Bind (#2488) and common.BindToGlobal (#2451) functions.
    • The context-based (common/context.go #2488) utils have also been removed.

Enhancements and UX improvements

Stricter thresholds' evaluation before the execution starts (#2330)

k6 v0.37.0 already improved threshold parsing by switching its underlying implementation from JavaScript to Go. k6 v0.38.0 introduces two additional improvements:

  • k6 will now parse and evaluate thresholds before the execution starts. If a threshold is invalid, as described below, k6 will immediately exit without starting the load test.
  • k6 will now detect invalid thresholds:
    export const options = {
        // ...
        thresholds: {
            // Incorrect thresholds expressions:
            http_req_failed: ["rave<0.01"], // e.g. "rave" is not a valid aggregation method
            // Thresholds applying to a non-existing metrics:
            iDoNotExist: ["p(95)<200"], // e.g. the metric 'iDoNotExist' does not exist
            // Thresholds applying an aggregation method that's unsupported by the metric they apply to:
            my_counter: ["p(95)<200"], // Counter metrics do not support the p() aggregation method
        },
    };

Disabling colors (#2410)

In addition to the --no-color CLI flag, the ANSI color escape codes emitted by k6 can now also be disabled by setting the NO_COLOR or K6_NO_COLOR environment variables, following the NO_COLOR standard.

# No color output
K6_NO_COLOR=true k6 run script.js

# No color output
NO_COLOR= k6 run script.js

Support for encrypted TLS private keys (#2488)

You can now use passphrase-protected private keys when authenticating with TLS. Using the password property of an options' tlsAuth object, you can now indicate the passphrase to decrypt a private key. Note that this support is limited to the scope of RFC1423 and does not support PKCS8 keys, as they're not yet supported by the Golang standard library.

export const options = {
    tlsAuth: [
        {
            domains: ["example.com"],
            cert: open("mycert.pem"),
            key: open("mycert-key.pem"),
            password: "mycert-passphrase",
        },
    ],
};

Thanks, @Gabrielopesantos, for the contribution.

Improve JSON output's performance (#2436)

The JSON output was optimized and now should be around 2x more performant at outputting metrics. This means that it either can export twice as many metrics, or use half the resources to do the same amount of metrics.

As a side effect, there is a slight breaking change: the tags field is no longer sorted.

Treat panics as interrupt errors (#2453)

We changed the behavior of how k6 treats Go panics, which may happen because of bugs in k6 or in a JavaScript k6 extension. Previously, the behavior was to catch the panic and log it as an error.

Starting with v0.38.0, whenever k6 observes a Go panic, it logs an error like before, but more importantly, it will abort the script execution and k6 will exit with a non-0 exit code. This will help extension authors to identify issues in their extensions more easily.

Miscellaneous

  • #2411 The k6 command-line UI (logo, test description, and progress bars) can now effectively be disabled using the --quiet flag.
  • #2429 lib/types now exposes the source of the NullHostnameTrie to simplify access to an original list of the hostnames.

Extensions

PoC for a new Web Sockets JS API

We built a new xk6 extension, https://github.com/grafana/xk6-websockets, with a proof of concept implementation for a new JavaScript Web Sockets API. This API uses the global event loops introduced in k6 v0.37.0 to allow a single VU to have multiple concurrent web socket connections open simultaneously, greatly reducing the resources needed for large tests. It also is a step towards supporting the official Web Sockets JS standard, potentially allowing the usage of more third-party JS libraries in the future.

Please share any feedback you have about the new extension since it's likely that we'll adopt a future version of it into the k6 core in one of the next several k6 releases.

gRPC module refactored to enable gRPC extensions to use it

#2484 moved out in a new dedicated Go lib/netext/grpcext package all the parts not strictly required from the js/k6/net/grpc module for binding the gRPC operations and the JavaScript runtime. It facilitates the development of extensions based on gRPC without the direct dependency on the goja runtime. Furthermore, the new [Dial](https://github.com/grafana/k6/blob/bef458906f6884a99843573223028981c0a8b8db/l...

Read more

v0.37.0

15 Mar 09:50
67657f4
Compare
Choose a tag to compare

k6 v0.37.0 is here! 🎉 Mainly it contains fixes and ongoing efforts with refactoring.

New Features!

Added experimental basic event loop (#882)

We added basic event loop support in k6 (#2228 and #2373) 🎉 This was just the first step and isn't used by any of the existing k6 JS APIs yet. For now, it is only available to xk6 extensions like this one that adds support for setTimeout(), setInterval(), clearTimeout() and clearInterval().

Expect to see more changes related to event loops in the next k6 releases, where event loops will start being used by some core k6 modules! For example, by improving some existing JavaScript APIs to have support for callbacks or return Promise values, so they can be used asynchronously. We expect this change will unlock a lot of previously difficult use cases (see #882), though we'll likely iterate on these new APIs as experimental extensions for a while, to stabilize them before we merge them into the core.

ℹ️ If you are an extension developer, please use it and give your feedback. But take into consideration that it's likely that the current Go API may change.

Added an option to output k6 logs to a file through --log-output (#2285)

This is on top of the already supported options for sending logs to stdout/stderr and to Grafana Loki. This new feature speaks for itself with simple usage examples:

k6 run --log-output file=./k6.log --logformat json ./examples/stages.js

And one more with a defined minimal log level:

k6 run --log-output file=./k6.log,level=info --logformat json ./examples/stages.js

Thanks, @alyakimenko for the contribution!

Docs: Using file output

Breaking changes

Introduced stricter thresholds parsing (#2400)

In the past, thresholds were evaluated using a JavaScript runtime. For a multitude of reasons, this wasn't satisfying. As of v0.37.0, thresholds are now parsed directly in Go. As a result, k6 will now return an error message on thresholds that do not strictly match the documented specification, instead of just silently ignoring them. Another change is that when a non syntactically correct threshold expression is detected, k6 will immediately interrupt its execution before even starting the load test run.

Below you can find examples of the thresholds expressions that won't work anymore:

export const options = {
    thresholds: {
        "http_req_duration": [
            // although the aggregation method and values are correct, 
            // the equal sign is invalid; use == or ===
            "rate=200",
            // thresholds do not support javascript expressions anymore
            "throw new Error('wat')",
            // it fails, as foo is not a valid threshold expression's aggregation method keyword
            "foo>500",
        ],
    },
};

Extensions

v0.37.0 finalizes (#2376) the switching of our internal modules (gRPC module refactoring) to a new Go/JavaScript module API.

⚠️ It's important to highlight that the old API (e.g. methods like context.WithRuntime, common.Bind and others #2384) is deprecated and will be removed in the next k6 release (v0.38.0). For this release, every extension that isn't using the new API will get a warning message like this:

WARN[0000] Module 'k6/x/sql' is using deprecated APIs that will be removed in k6 v0.38.0, for more details on how to update it see https://k6.io/docs/extensions/guides/create-an-extension/#advanced-javascript-extension

We did migrations for some xk6 extensions (see connected issues to the task #2344). The pull requests can serve as examples on how to transition your extension to the new API.

Docker Repository

We migrated our Docker Hub repository from loadimpact/k6 to grafana/k6 (#2377).

docker run -i grafana/k6 run - <script.js

We will continue publishing our docker image releases as both loadimpact/k6 and grafana/k6 for several more releases, but if you use the old one in your local or CI environments, please plan the migration.

Enhancements and UX improvements

Bugs fixed!

Internals

  • We updated our CI to improve developer experience. Dependency and linter checks now run only for pull requests (#2403).
  • This release also contains a few refactoring PRs that fix linters errors (#2334, #2331 and #2341), remove global variable usage (#2336, #2358, #2353 and #2357) and remove an unnecessary dependency (#2313) which makes our codebase more consistent and maintainable.
  • The headers parameter in k6's GRPC module is marked as deprecated (#2370).
  • Switched envconfig to our own fork (#2337) in order to abstract the os package and improve testability.

v0.36.0

24 Jan 09:59
Compare
Choose a tag to compare

k6 v0.36.0 is here! 🎉 It introduces a couple of new features which enhance its usability, includes a number of fixes, and the result of ongoing refactoring efforts.

New Features!

Source Maps support (#2082)

Following #2082, k6 now has support for Source Maps. k6 will try to load source maps either from the file system(s) or from inside the script, based on the standard(-ish) //#sourceMappingURL= comments. Furthermore, as k6 internally uses Babel to transform ES6+ scripts to ES5.1+, it will now make use of its ability to generate source maps, including combining with previously generated ones, to report correct line numbers. This should fix #1804; however, we anticipate some more issues will arise, and further tweaking will be necessary.

Thus, given an imported.js module such as:

export function f1() {
  throw "line 2";
}

  throw "line 6";
}

export function f3() {
  throw "line 10";
}

and a k6 test script importing it as such:

import { f2 } from "./imported.js"

export default function() {
  f2();
}

Previous versions of k6 would report an error stack trace indicating an invalid line number in imported.js (10):

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:10:61(2))
        at file:///some/path/sourcemap.js:4:20(4) executor=per-vu-iterations scenario=default source=stacktrace

Starting with v0.36.0 and source maps support, k6 would now report the exception at the correct line in imported.js:

ERRO[0000] line 6
        at f2 (file:///some/path/imported.js:6:2(2))
        at file:///some/path/loadtest.js:4:2(4)
        at native executor=per-vu-iterations scenario=default source=stacktrace

Temporary warning

Note that if a file size is greater than 250kb and the internal Babel is needed, Babel will not generate source map. This is because during internal testing it was found this takes 3x to 4x more memory, potentially leading to OOM (standing for "Out Of Memory", a state in which the OS kills a process for using too much memory) on bigger inputs. If required, you can control the accepted file size limit via the temporary K6_DEBUG_SOURCEMAP_FILESIZE_LIMIT=524288 environment variable; which will be removed after we no longer rely on Babel (#2296). A pre-generated source map will always be loaded. For more details, check #2345.

Ability to abort tests (#2093)

Thanks to the contribution of @gernest (#2093), k6 now has the ability to abort a test run from within the test script. The newly added test.abort() function in the k6/execution module allows k6 scripts to immediately abort the test execution - the VU that called it will abort immediately and any other VUs in the same or other instances (in the case of k6 cloud) will also be interrupted and abort soon after. Local k6 run tests will exit with a code of 108, so this event can also be easily detected in a CI script.

Aborting is possible during initialization:

import exec from "k6/execution";
exec.test.abort();

As well as inside the default function:

import exec from "k6/execution";

export default function() {
  // Note that you can abort with a specific message too
  exec.test.abort("this is the reason");
}

export function teardown() {
  console.log("teardown will still be called after test.abort()");
}

k6 inspect extended output (#2279)

Following #2279, the k6 inspect command now supports an --execution-requirements flag. When used, the command's output will include fields related to the execution requirements, by deriving k6's configuration from the execution context, and including the maxVUs and totalDuration fields in the output.

Forcing HTTP/1 protocol (#2222)

Thanks to the work of @sjordhani22, #2222 made it possible to force k6 to use version 1.1 of the protocol when firing HTTP requests.

It can be done by setting the http2client=0 value in the GODEBUG environment variable:

GODEBUG=http2client=0 k6 run testscript.js

N.B: the usage of the GODEBUG variable is considered temporary, and expected to change in the future. If you start using this feature, keep an eye out for potential future changes.

Extensions

v0.36.0 marks the switch of some of our internal modules to a new Go/JavaScript module API. We expect this change to make the process of developing internal JavaScript modules and advanced JavaScript extensions easier and more streamlined in the future. Although this switch to a new API does not introduce breaking changes for existing extensions yet, we anticipate deprecating the old extension API (e.g. common.Bind(), lib.WithState(), etc.) at an undecided point in the future.

For more details, see: #2243, #2241, #2239, #2242, #2226, and #2232.

Breaking changes

Restricting file opening to init context

VUs are now restricted to only open() files that were also opened in the init context of the first VU - the one that was initialized to get the exported options from the JS script (__VU==0). While it was somewhat possible to open files only in other VUs (e.g __VU==2) in the past, it was unreliable. #2314 ensures that k6 would now throw an error in a similar scenario. This means that you can still open files only for some VUs, but you need to have opened all of those files in the initial VU (__VU==0).

let file;

if (__VU == 0) {
  open("./file1.bin")
  open("./file2.bin")
} else if (__VU % 2 == 0) {
  file = open("./file1.bin")
} else {
  file = open("./file2.bin")
}

export default () => {
  // use file for something
}

Bugs Fixed!

  • We addressed an issue uncovered by our community, which kept our users from using GRPC with multiple services definition in a single proto file. This issue was solved in #2265.
  • Thanks to the contribution of @Resousse, we've now updated k6's go-ntlmssp dependency. The updating PR #2290 indeed fixes issues with NTLM Authentication backends returning two authorization headers.

Maintenance

  • We have refactored our implementation of the RampingVU executor, for better clarity and maintainability. See #2155.
  • #2316 relaxed quite a few of the code linting rules we applied to k6's code. It also revamped our Makefile, so the new make ci-like-lint target will run the exact same golangci-lint version that will be used in our GitHub Actions CI pipeline.
  • #2304 prepared the removal of external dependencies from k6's JSONAPI compliant REST API, and deprecated the api.v1's client.Call method in favor of its newer client.CallAPI counterpart. It allows us to both reduce our reliance on external dependencies and improve its maintainability.
  • We have updated our Goja dependency, our JS interpreter, to its latest available version. Unfortunately, some of the new features are not always usable, yet. Namely, Goja now supports the optional chaining syntax, but the Babel version we use presently does not. Which means that if Babel needs to be used, optional chaining can't be. See #2317 and #2238.
  • Thanks to @knittl, #2312 upgraded loadimpact/k6 docker image base to Alpine 3.15.

Known Bugs

  • #2226 introduced an unintended breaking change to http.head(). The signature in k6 v0.35.0 was http.head(url, [params]) and was inadvertently changed to http.head(url, [body], [params]) in v0.36.0. That change will be reverted in k6 v0.37.0, but until then, we suggest users use the stable http.request('HEAD', url, null, params) API for HTTP HEAD requests that need to specify custom parameters. Thanks, @grantyoung, for reporting the problem (#2401)!

v0.35.0

17 Nov 10:00
Compare
Choose a tag to compare

k6 v0.35.0 is here! 🎉 It introduces several new features that nicely enhance its usability and also contains a whole lot of fixes and ongoing efforts with refactoring.

In total, we have closed 14 issues. We have also branched out three new xk6 extensions during this release ⭐

New features

Ability to set VU-wide custom metric tags (#2172)

k6 now supports setting tags for VUs as part of the Execution API with an easy key-value interface. These tags are attached to the metrics emitted by the VU. Example usage:

import http from 'k6/http';
import exec from 'k6/execution';

export const options = {
    duration: '10s',
    vus: 3,
};

export default function () {
    exec.vu.tags['mytag'] = 'value';
    exec.vu.tags['vuId'] = exec.vu.idInTest;

    console.log(`mytag is ${exec.vu.tags['mytag']} and my VU's ID in tags ${exec.vu.tags['vuId']}`);

    // the metrics these HTTP requests emit will get tagged with `mytag` and `vuId`:
    http.batch(['https://test.k6.io', 'https://test-api.k6.io']);
}

One of the most requested use cases for this feature is that now we can tag all metrics with the current stage number. With a bit of JS code it is possible to calculate which stage of a ramping-vus or ramping-arrival-rate scenario the VU is currently in. This in turn allows the setting of thresholds only on the metrics that were emitted in specific stages of the test run! 🎉

There are some caveats, however: values can be only of String, Number or Boolean type, while values of other types will result either in a warning or an exception if throw option is enabled. Additionally, given that k6 has a whole bunch of system tags, one should be careful with using them as keys. You can read complete information about VU tags in k6/execution docs.

Initial basic support for JS promises

With the goja update in #2197, you can now make a Promise and chain it in your k6 scripts:

export default function () {
    var p = new Promise((resolve, reject) => {
        console.log('do something promising!');
        reject('here');
    });

    p.then(
        (s) => { console.log('fulfilled with', s) },
        (s) => { console.log('rejected with', s) },
    );
}

It must be noted that Promises are not used by k6 itself yet but this addition is a stepping stone for implementing async functionality in future releases. Thanks, @dop251, for your awesome work in developing goja! ❤️

Support for gRPC server reflection (#2160)

k6's gRPC capabilities were extended with a support for server reflection which allows one to use gRPC even without a proto file at hand. In other words, the following script is now possible:

import grpc from 'k6/net/grpc';
import { check } from "k6";

let client = new grpc.Client();

export default () => {
	client.connect("127.0.0.1:10000", {plaintext: true, reflect: true})
	const response = client.invoke("main.RouteGuide/GetFeature", {
		latitude: 410248224,
		longitude: -747127767
	})

	check(response, {"status is OK": (r) => r && r.status === grpc.StatusOK});
	console.log(JSON.stringify(response.message))

	client.close()
}

You can read more about the protocol here. Thanks, @joshcarp, for putting a lot of effort into this feature!

Other changes and UX improvements

  • Support for cookie jars in k6/ws (#2193).
  • Forbid metric.Add calls to let NaN values through. Instead, k6 will log nice warnings or throw an exception if --throw is enabled (#1876, #2219).
  • Support for compression in websockets (#2162). Thanks, @cooliscool!
  • Switch to camel case for CLI options to the outputs (#2150). Thanks, @josephwoodward!
  • Much neater error message on nil response body (#2195). Thanks, @daniel-shuy!

New xk6 extensions

xk6-browser

xk6-browser is a browser automation extension which relies on Chrome Devtools Protocol. With xk6-browser, you can interact with the browser to test your web applications end-to-end while accessing all of the k6 core features, including protocol-level APIs and other k6 extensions. It’s a single tool for both protocol and browser-level testing.

The browser extension comes with an API that aims for rough compatibility with the Playwright API for NodeJS, meaning k6 users do not have to learn an entirely new API.

xk6-output-remote-write

Prometheus is now officially supported in k6 OSS with a xk6-output-remote-write extension. This is an output extension with implementation for Prometheus Remote-Write protocol which means that beyond Prometheus, any compatible remote-write solution can be used with it. You can read the full guide to using the extension in the relevant tutorial.

xk6-output-influxdb

After hard work at working out how to integrate InfluxDB v2 API, it was decided to pull that integration into a new xk6-output-influxdb extension for now. The built-in influxdb output in k6 still supports only InfluxDB v1, as before, with some minor optimizations (#2190).

Please try out the new extensions and tell us what you think!

Breaking changes

  • The addition of common metrics registry (#2071) no longer allows defining custom metrics with the same name as one of the builtin metrics, e.g. new Counter("http_req_duration") will now abort. Similarly, an attempt to redefine a metric with the same name but with different type will error out. Builtin metrics may no longer be referenced as global objects in xk6 extensions either.
  • Fix inconsistency in environment variables' names: use K6_NO_SETUP and K6_NO_TEARDOWN options instead of NO_SETUP and NO_TEARDOWN (#2140).
  • Module interfaces were changed as part of refactoring efforts. Any JS module that needs access to the VU must now implement the new interfaces. This change can impact some xk6 extensions (#2234).

Bugs fixed!

  • Fix of a misleading sorting of custom submetrics in the default end-of-test summary (#2198). Thanks, @knittl!
  • Fix for extensions depending on afero.FS: implement a newer version of the afero.FS interface for internal filesystems so that extension depending on that or newer version can be built (#2216).
  • Fix for websockets: websockets now use the global User-Agent setting (#2151). Thanks, @cooliscool!
  • Fixes for tests, Github actions, and Loki integration (#2205, #2153, #2220).

Maintenance

hacktoberfest

k6 participated in this year's hacktoberfest and we would like to thank all contributors! Here're some additional improvements made by the community members:

  • Add multi-message WebSockets tests (#2184).
  • Try out the new and shiny Github forms which are already improving the formatting of k6's new issues (#2174,#2179).
  • An improved writing style and correctness in our README (#2189, #2152, #2169, #2181) and in some other places (#2182).

Thank you, @knittl, @cooliscool, @josephwoodward, @b5710546232, @Nontw, @divshacker, @daniel-shuy, @Sayanta66, @marooncoder09, @idivyanshbansal, @saintmalik, @EricSmekens, for helping make k6 better 😄

v0.34.1

16 Sep 08:32
v0.34.1
Compare
Choose a tag to compare

k6 v0.34.1 is a patch release with a few minor bugfixes:

  • There was a minor bug in the new k6/execution API added to k6 v0.34.0 - some of its properties weren't usable with the externally-controlled executor (#2132).
  • goja, the JavaScript runtime that k6 uses, was updated to its latest version (#2135), which fixed a couple of bugs:
    • A newly introduced JS bug from k6 v0.34.0, where rest parameters were undefined when the functions also had an internal lambda (#2131). Thanks for reporting, @efdknittlfrank!
    • An old JS bug, first introduced in k6 v0.28.0, which caused Response.json() to not have a length property when the response was a JSON array, i.e.response.json().hasOwnProperty('length') was returning false (#2133). Thanks for reporting, @julien-sugg!

v0.34.0

09 Sep 11:17
Compare
Choose a tag to compare

k6 v0.34.0 is here! 🎉 It introduces the already announced k6/execution API and includes some more enhancements and a bunch of minor bug fixes.

New k6 JavaScript API - k6/execution

The k6/execution module allows the fetching of information about the current VU, instance (mostly relevant in distributed/cloud) or scenario execution state through a series of exported properties.

How to use

Some of the previously suggested solutions with the globally available __VU and __ITER values, such as for getting a unique object per iteration from an array or SharedArray, can be done by using the scenario.iterationInTest property, which is guaranteed to be unique across VUs, even for distributed or cloud tests. For example:

import exec from "k6/execution";
import { SharedArray } from "k6/data";

const data = new SharedArray("my dataset", function(){
  return JSON.parse(open('my-large-dataset.json'));
})

export const options = {
  scenarios :{
    "use-all-the-data": {
      executor: "shared-iterations",
      vus: 100,
      iterations: data.length,
      maxDuration: "1h"
    }
  }
}

export default function() {
  // this is unique even in the cloud
  var item = data[exec.scenario.iterationInTest];
  http.post("https://httpbin.test.k6.io/anything?endpoint=amazing", item)
}

You can read the full documentation here.

Enhancements and UX improvements

  • Warn Windows users on importing dependencies or opening files as absolute paths (#2078).
  • Pass setup data object into handleSummary callback (#2103). Thanks, @SamuelJohnson01997!

Breaking changes

  • The deprecated outputs Datadog and Kafka have been removed (#2081).

Bugs fixed!

  • Use the POST HTTP request method instead of GET for pushing logs to Loki (#2100).
  • Encode the blacklistIPs option using the CIDR notation in JSON (#2083).
  • ext.loadimpact option has the same precedence as the script configuration during the consolidation process (#2099).
  • The WebSocket connection used for tailing logs from the k6 Cloud is reestablished in the case of an unexpected error (#2090).

Internals

  • A simpler and clearer API has been added as an alternative to common.Bind, which also gives JS modules and extensions easy access to some useful internal objects and runtime information (#2108). This API is not yet stable, it's very likely to change more in future k6 versions.
  • Speeding ups TC39 tests using a pool of Babel compilers (#1839).
  • Goja and some internal dependencies have been updated adding the native support for Arrow functions, Destructuring, Default arguments and Computed properties features. For the same reason, the relative Babel's plugins supporting those features are not required anymore so they have been disabled (#2109, #2092).

v0.33.0

29 Jun 10:47
v0.33.0
Compare
Choose a tag to compare

k6 v0.33.0 is here! 🎉 It's a small release that includes a bunch of minor bugfixes and enhancements, but is also laying the groundwork for some major new features like the upcoming k6/execution API in k6 v0.34.0.

Acquired by Grafana Labs

Load Impact, the company behind k6, was acquired by Grafana Labs! 🎉 Nothing changes regarding the k6 development for now, and any changes in the future will only be in the direction of accelerating our existing roadmap and plans, as well as better integration between k6 and the awesome Grafana tools. For more details, see the official Grafana press release.

Enhancements and UX improvements

  • The --verbose help message and the statsd warning message were improved (#2005). Thanks, @vishalkuo!
  • The noColor k6 option and the current UI state are now propagated to the handleSummary() function. The state object has the isStdErrTTY, isStdOutTTY and testRunDurationMs keys (#1975).
  • The error message when an HTTP request times out was improved (previously it was context deadline exceeded) and it now has an error_code value of 1050 (#2008). Thanks, @vishalkuo!
  • Script errors will no longer have the confusing GoError prefix in their error messages (#1775).
  • All custom metric objects now have a name property (#2058 and #2076). Thanks, @olimpias and @david-gourde!
  • Top-level JS arrays will now be properly encoded when sent in the body of a application/x-www-form-urlencoded request (#2060). Thanks, @noelzubin!

Bugs fixed!

  • The minIterationDuration option was uninterruptible and could delay the stopping of a scenario even after gracefulStop had expired. (#2035).
  • The error_code detection for HTTP/2, x509 and TLS (and potentially others) was unreliable (#2025).
  • k6 used to panic when responseType was binary, but there was no response body actually returned, e.g. when there was an HTTP error (#2041).
  • The throw option was not respected when there was an invalid URL (#2045). Thanks, @gchaincl!
  • k6 would return an exit code of 103 instead of 107 for script errors when initializing non-service VUs (#2046).
  • Deleted library versions from cdnjs could previously cause a panic (#2047).
  • The correct error message for missing files was not shown when the filename contained spaces (#1973).
  • The regular expressions for the github and cdnjs "magic" loaders were slightly wrong (#2066).
  • A potential (harmless) data race could have been caused by an unintentional copying of a data struct (#2067).
  • The segmentation of small ramping-arrival-rate scenarios was not optimal (#1863).

Internals

  • The default end-of-test summary is now completely generated by the same k6-summary JS code that is hosted on jslib.k6.io (#1975). That PR also improved the k6 TTY detection and removed a few Go dependencies and code hacks, though it also caused us to bump the minimum required Go version for compiling k6 to Go 1.16 (because of its usage of go:embed).
  • Arrival-rate executors will no longer create a new goroutine for every new iteration (#1957 and #2038).
  • We have enabled GitHub's CodeQL checks for the Go parts of the repo (#1961). Thanks, @jfcg!
  • We have added the necessary k6 core changes for providing execution information to scripts #1863! This was the groundwork for the extended replacement of the __VU and __ITER execution context variables we plan to introduce. The new API will be able to return other information as well, for example which scenario the current iteration is in, what's the number of the current VU/iteration globally across all k6 instances, or in the current scenario, etc. These APIs are still not available to JS scripts, but we plan to expose them via the k6/x/execution xk6 extension for now and iterate on them in the following weeks, releasing a stable version in k6 v0.34.0.

Breaking changes

  • The k6 cloud exit code for a failed cloud test was changed from 99 to 97 (#2046).
  • The default value of K6_STATSD_TAG_BLOCKLIST and K6_DATADOG_TAG_BLACKLIST is now vu,iter,url (#2063).
  • The __ITER execution context variable is no longer set in setup() and teardown() due to #1863. This might be better classified as removing a previously undefined behavior instead of a breaking change, but it's still worth mentioning.