-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: script validation #5
Conversation
@@ -13,5 +13,6 @@ | |||
"plugin:import/typescript", | |||
"prettier" | |||
], | |||
"parser": "@typescript-eslint/parser" | |||
"parser": "@typescript-eslint/parser", | |||
"ignorePatterns": ["resources/group_snippet.js"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as the script is a snippet to be added to a k6 script I'm ignoring normal validation for it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a first draft of the approach, this gets inserted in the user script by us.
Right now it's dumb and it's just looking for a line k6/http
and inserts this code after it.
# pop the group header we set on validation | ||
if "X-K6-Group" in flow.request.headers: | ||
group = flow.request.headers.pop("X-K6-Group") | ||
f["comment"] = group |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using comment, if we need to pass more information this could be a json string, but I'm keeping it simple for now
} | ||
|
||
export interface K6Log { | ||
level: 'info' | 'debug' | 'warning' | 'error' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't actually get debug
logs, those are only emitted when k6 is run with the -v
flag
} | ||
|
||
const k6 = spawn( | ||
'k6', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we are running the locally installed k6 in this pr
const modifiedScriptPath = path.join( | ||
app.getPath('temp'), | ||
'k6-studio-script.js' | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the modified script, enhanced with our snippet is stored locally on a temp file, a cons of this approach is that when an errors happens it will showcase that path instead of the original script path 🤔
console.error(`stdout: ${data}`) | ||
}) | ||
|
||
stderrReader.on('line', (data) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
by default k6 logs are printed in stderr 🤷
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't tested it just yet, but added a few comments/suggestions around mutation of function arguments, which IMO we should avoid.
request(method, url, ...args) { | ||
const group = { 'X-k6-group': trimAndRemovePrefix(execution.vu.tags.group) } | ||
args = instrumentArguments(group, ...args) | ||
|
||
return request(method, url, ...args) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer not mutating the args.
request(method, url, ...args) { | |
const group = { 'X-k6-group': trimAndRemovePrefix(execution.vu.tags.group) } | |
args = instrumentArguments(group, ...args) | |
return request(method, url, ...args) | |
} | |
request(method, url, ...args) { | |
const group = { 'X-k6-group': trimAndRemovePrefix(execution.vu.tags.group) } | |
return request(method, url, instrumentArguments(group, args) | |
) | |
} |
This suggestion would require instrumentArguments
to accept args
as a single parameter rather than a spread of potentially multiple params. I will provide more detail in a separate comment.
return trimmedInput | ||
} | ||
|
||
function instrumentArguments(groupName, ...args) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can as well be replaced with
function instrumentArguments(groupName, args) {
// ...
}
As long as the function calls are updated accordingly
function instrumentArguments(groupName, ...args) { | ||
switch (args.length) { | ||
case 0: | ||
args.push(null) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's avoid mutating function arguments here as well. I'll post a suggested change in a bit (initially misinterpreted the code and wrote a snippet that wouldn't work)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is my suggestion (it needs to be tested though):
function instrumentArguments(groupName, args) {
switch (args.length) {
case 0:
return [null, { headers: groupName }];
case 1:
return [args[0], { headers: groupName }];
default:
return args.map((arg, i) => {
if (i !== 1) return arg
return !arg ? { headers: groupName } : { headers: {...arg.headers, groupName }}
})
}
}
Could you also make a comment explaining what this function does?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
note: I will be extracting this as a separate ticket to fully cleanup the draft of the group snippet 🙌
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree with Uladzimir's comments, also added a comment about extracting resource reader function
const getGroupSnippet = async () => { | ||
let groupSnippetPath: string | ||
|
||
// if we are in dev server we take resources directly, otherwise look in the app resources folder. | ||
if (MAIN_WINDOW_VITE_DEV_SERVER_URL) { | ||
groupSnippetPath = path.join( | ||
app.getAppPath(), | ||
'resources', | ||
'group_snippet.js' | ||
) | ||
} else { | ||
groupSnippetPath = path.join(process.resourcesPath, 'group_snippet.js') | ||
} | ||
|
||
return readFile(groupSnippetPath, { encoding: 'utf-8' }) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This logic to read resources is repeated in a few places, could be extracted into helper function:
function getResource(resource: string) {
if (MAIN_WINDOW_VITE_DEV_SERVER_URL) {
return path.join(app.getAppPath(), 'resources', resource)
} else {
return path.join(process.resourcesPath, resource)
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
my rule of thumbs is 3 occurrences for extracting common behaviour, here we should also handle the case where the path is built also depending on os/architecture 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's also not reusable in every place because when there are multiple paths, it behaves differently between the local resource and the packaged app, there are some caveats.
Because of that I have a preference to have them explicit for now and consolidate when we know all the resources we need and refactor that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Logic for Validating a k6 script.
Group info
To retrieve group information there is a snippet of javascript that will be added to the script after the http client is imported. This snippet replaces the client with a wrapper that will add the
X-k6-Group
header containing group information.This approach is similar to what the
tracing
instrumentation module does.This group information is then extracted by the proxy before making the request and stored in the
comment
field of theProxyData
datastructure.Events
There are three events added:
script:select
- opens a dialog to select a script locally and returns the path to itscript:run
script:stop
The name of these could be improved.
Requests
For requests the approach is the same as with the Recorder, the proxy captures and send the events about requests/responses in the same way, with the added group info in the
comment
field.