Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: script validation #5

Merged
merged 6 commits into from
Jun 11, 2024
Merged

feat: script validation #5

merged 6 commits into from
Jun 11, 2024

Conversation

Llandy3d
Copy link
Member

Logic for Validating a k6 script.

Group info

To retrieve group information there is a snippet of javascript that will be added to the script after the http client is imported. This snippet replaces the client with a wrapper that will add the X-k6-Group header containing group information.
This approach is similar to what the tracing instrumentation module does.

This group information is then extracted by the proxy before making the request and stored in the comment field of the ProxyData datastructure.

Events

There are three events added:

  • script:select - opens a dialog to select a script locally and returns the path to it
  • script:run
  • script:stop

The name of these could be improved.

Requests

For requests the approach is the same as with the Recorder, the proxy captures and send the events about requests/responses in the same way, with the added group info in the comment field.

@@ -13,5 +13,6 @@
"plugin:import/typescript",
"prettier"
],
"parser": "@typescript-eslint/parser"
"parser": "@typescript-eslint/parser",
"ignorePatterns": ["resources/group_snippet.js"]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as the script is a snippet to be added to a k6 script I'm ignoring normal validation for it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a first draft of the approach, this gets inserted in the user script by us.
Right now it's dumb and it's just looking for a line k6/http and inserts this code after it.

# pop the group header we set on validation
if "X-K6-Group" in flow.request.headers:
group = flow.request.headers.pop("X-K6-Group")
f["comment"] = group
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using comment, if we need to pass more information this could be a json string, but I'm keeping it simple for now

}

export interface K6Log {
level: 'info' | 'debug' | 'warning' | 'error'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't actually get debug logs, those are only emitted when k6 is run with the -v flag

}

const k6 = spawn(
'k6',
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are running the locally installed k6 in this pr

Comment on lines +27 to +30
const modifiedScriptPath = path.join(
app.getPath('temp'),
'k6-studio-script.js'
)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the modified script, enhanced with our snippet is stored locally on a temp file, a cons of this approach is that when an errors happens it will showcase that path instead of the original script path 🤔

console.error(`stdout: ${data}`)
})

stderrReader.on('line', (data) => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by default k6 logs are printed in stderr 🤷

Copy link
Collaborator

@going-confetti going-confetti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested it just yet, but added a few comments/suggestions around mutation of function arguments, which IMO we should avoid.

Comment on lines +9 to +14
request(method, url, ...args) {
const group = { 'X-k6-group': trimAndRemovePrefix(execution.vu.tags.group) }
args = instrumentArguments(group, ...args)

return request(method, url, ...args)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer not mutating the args.

Suggested change
request(method, url, ...args) {
const group = { 'X-k6-group': trimAndRemovePrefix(execution.vu.tags.group) }
args = instrumentArguments(group, ...args)
return request(method, url, ...args)
}
request(method, url, ...args) {
const group = { 'X-k6-group': trimAndRemovePrefix(execution.vu.tags.group) }
return request(method, url, instrumentArguments(group, args)
)
}

This suggestion would require instrumentArguments to accept args as a single parameter rather than a spread of potentially multiple params. I will provide more detail in a separate comment.

return trimmedInput
}

function instrumentArguments(groupName, ...args) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can as well be replaced with

function instrumentArguments(groupName, args) {
  // ...
}

As long as the function calls are updated accordingly

function instrumentArguments(groupName, ...args) {
switch (args.length) {
case 0:
args.push(null)
Copy link
Collaborator

@going-confetti going-confetti Jun 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's avoid mutating function arguments here as well. I'll post a suggested change in a bit (initially misinterpreted the code and wrote a snippet that wouldn't work)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my suggestion (it needs to be tested though):

function instrumentArguments(groupName, args) {
  switch (args.length) {
    case 0:
      return [null, { headers: groupName }];
    case 1:
      return [args[0], { headers: groupName }];
    default:
      return args.map((arg, i) => {
        if (i !== 1) return arg
        
        return !arg ? { headers: groupName } : { headers: {...arg.headers, groupName }}
      })
  }
}

Could you also make a comment explaining what this function does?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: I will be extracting this as a separate ticket to fully cleanup the draft of the group snippet 🙌

Copy link
Collaborator

@e-fisher e-fisher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with Uladzimir's comments, also added a comment about extracting resource reader function

Comment on lines +93 to +108
const getGroupSnippet = async () => {
let groupSnippetPath: string

// if we are in dev server we take resources directly, otherwise look in the app resources folder.
if (MAIN_WINDOW_VITE_DEV_SERVER_URL) {
groupSnippetPath = path.join(
app.getAppPath(),
'resources',
'group_snippet.js'
)
} else {
groupSnippetPath = path.join(process.resourcesPath, 'group_snippet.js')
}

return readFile(groupSnippetPath, { encoding: 'utf-8' })
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic to read resources is repeated in a few places, could be extracted into helper function:

function getResource(resource: string) { 
  if (MAIN_WINDOW_VITE_DEV_SERVER_URL) {
    return path.join(app.getAppPath(), 'resources', resource)
  } else {
    return path.join(process.resourcesPath, resource)
  }
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my rule of thumbs is 3 occurrences for extracting common behaviour, here we should also handle the case where the path is built also depending on os/architecture 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also not reusable in every place because when there are multiple paths, it behaves differently between the local resource and the packaged app, there are some caveats.
Because of that I have a preference to have them explicit for now and consolidate when we know all the resources we need and refactor that

Copy link
Collaborator

@e-fisher e-fisher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Llandy3d Llandy3d merged commit 65a6ec1 into main Jun 11, 2024
@Llandy3d Llandy3d deleted the validation branch June 11, 2024 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants