VAULT-17592 Extract failed Go test results across runners

[kubawi]

No description provided.

[miagilepner]

This is awesome!!!

I have 2 things that I think we should eventually consider doing, though it doesn't need to be in this PR:

1. Group tests by name, and list all of the jobs where that test failed so that a single failure doesn't appear 3 times in the table. I have a jq that is a starting point for that:

jq -r -n '[ inputs | select(.Action == "fail") | select(.Test != null) | {package: .Package, test: .Test, type: (input_filename | sub("failure-summary-";"") | sub(".json";"")), key: (.Package+" "+.Test)}  ] | group_by(.key) | map({ key: (.[0].key), value: [.[].type]}) | from_entries ' failure-summary-*.json

2. If we can add the run ID into the failure summary, then we could build the URL for the logs of the run ID and link that in the table. The run ID could either be part of the filename, or appended to the file.

Because these additions would be useful and the jq is just going to get more complicated, I wonder if we should consider doing this either in python or in a separate bash script with intermediary files. That might make it easier to debug and understand what's happening.

[kubawi]

This is awesome!!!

I have 2 things that I think we should eventually consider doing, though it doesn't need to be in this PR:

1. Group tests by name, and list all of the jobs where that test failed so that a single failure doesn't appear 3 times in the table. I have a jq that is a starting point for that:

jq -r -n '[ inputs | select(.Action == "fail") | select(.Test != null) | {package: .Package, test: .Test, type: (input_filename | sub("failure-summary-";"") | sub(".json";"")), key: (.Package+" "+.Test)}  ] | group_by(.key) | map({ key: (.[0].key), value: [.[].type]}) | from_entries ' failure-summary-*.json

2. If we can add the run ID into the failure summary, then we could build the URL for the logs of the run ID and link that in the table. The run ID could either be part of the filename, or appended to the file.

Because these additions would be useful and the jq is just going to get more complicated, I wonder if we should consider doing this either in python or in a separate bash script with intermediary files. That might make it easier to debug and understand what's happening.

Hey @miagilepner, thanks for the suggestions, I fully agree with both points. Would you mind if I merged this in first?
This PR as it is, will give us a single, simple summary and remove all of the individual ones, which are pretty useless at this point anyways. I can then raise another PR with a script that dedupes the tests and adds links to the logs.