roachtest: Save store directories on failure

[bdarnell]

Occasionally roachtests will fail in a way that is hard to reproduce (Current example: #35986. Past example: #32166). These failures often don't include enough information to tell what's going on; the store directories would be extremely helpful. I think we should tar up all the data directories for failed roachtests and upload them to s3/gce. (with maybe a 1-week expiration)

This will have a cost, but I think it'll be cheaper than the repeated runs we have to do to reproduce rare failures. It might be even better to keep the VMs running for inspection, but that would be much more expensive. I think saving the store directories has the better cost/benefit.

[github-actions]

We have marked this issue as stale because it has been inactive for
18 months. If this issue is still relevant, removing the stale label
or adding a comment will keep it active. Otherwise, we'll close it in
5 days to keep the issue queue tidy. Thank you for your contribution
to CockroachDB!