Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handleSummary called before all metrics are crunched #1887

Closed
na-- opened this issue Mar 8, 2021 · 0 comments · Fixed by #1888
Closed

handleSummary called before all metrics are crunched #1887

na-- opened this issue Mar 8, 2021 · 0 comments · Fixed by #1888
Labels
bug evaluation needed proposal needs to be validated or tested before fully implementing it in k6 high prio
Milestone

Comments

@na--
Copy link
Member

na-- commented Mar 8, 2021

A check I added on Friday (#1885) for xk6 JS and output extensions seems to have caught a problem with the end-of-test summary 😕 From https://github.com/loadimpact/k6/runs/2041232632:

2021-03-05T16:29:41.7585634Z + ./k6ext run --out outputtest=output-results.txt xk6-test.js
2021-03-05T16:29:41.7809953Z 
2021-03-05T16:29:41.7814813Z           /\      |‾‾| /‾‾/   /‾‾/   
2021-03-05T16:29:41.7815511Z      /\  /  \     |  |/  /   /  /    
2021-03-05T16:29:41.7816200Z     /  \/    \    |     (   /   ‾‾\  
2021-03-05T16:29:41.7816746Z    /          \   |  |\  \ |  (‾)  | 
2021-03-05T16:29:41.7817112Z   / __________ \  |__| \__\ \_____/ .io
2021-03-05T16:29:41.7817365Z 
2021-03-05T16:29:42.4474038Z   execution: local
2021-03-05T16:29:42.4475936Z      script: xk6-test.js
2021-03-05T16:29:42.4477250Z      output: test output extension
2021-03-05T16:29:42.4477895Z 
2021-03-05T16:29:42.4478714Z   scenarios: (100.00%) 1 scenario, 1 max VUs, 10m30s max duration (incl. graceful stop):
2021-03-05T16:29:42.4479781Z            * default: 5 iterations shared among 1 VUs (maxDuration: 10m0s, gracefulStop: 30s)
2021-03-05T16:29:42.4480493Z 
2021-03-05T16:29:42.4589016Z 
2021-03-05T16:29:42.4590100Z running (00m00.0s), 0/1 VUs, 5 complete and 0 interrupted iterations
2021-03-05T16:29:42.4591017Z default   [ 100% ] 1 VUs  00m00.0s/10m0s  5/5 shared iters
2021-03-05T16:29:42.6040590Z ++ cat output-results.txt
2021-03-05T16:29:42.6051809Z + OUTPUT_RESULT=10
2021-03-05T16:29:42.6052563Z + [[ 10 != \1\0 ]]
2021-03-05T16:29:42.6056159Z ++ cat summary-results.txt
2021-03-05T16:29:42.6064677Z + SUMMARY_RESULT=6
2021-03-05T16:29:42.6065207Z summary result was not as expected: 6
2021-03-05T16:29:42.6066851Z + [[ 6 != \1\0 ]]
2021-03-05T16:29:42.6067948Z + echo 'summary result was not as expected: 6'
2021-03-05T16:29:42.6068629Z + exit 12
2021-03-05T16:29:42.6096475Z ##[error]Process completed with exit code 12.

The test script emits the __ITER value for 5 iterations. It seems like the output received and processed all metrics (0+1+2+3+4==10), but the data handleSummary() got was missing its last value... 😕

I think I haven't broken this with #1869, but we should investigate, since it likely means we have another data race on our hands... 😞

@na-- na-- added bug high prio evaluation needed proposal needs to be validated or tested before fully implementing it in k6 labels Mar 8, 2021
na-- added a commit that referenced this issue Mar 8, 2021
@na-- na-- added this to the v0.31.0 milestone Mar 8, 2021
imiric pushed a commit that referenced this issue Mar 8, 2021
harrytwigg pushed a commit to APITeamLimited/globe-test that referenced this issue Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug evaluation needed proposal needs to be validated or tested before fully implementing it in k6 high prio
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant