Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework of bucket path handling in resultstore.go #825

Merged
merged 8 commits into from
Aug 15, 2023

Conversation

maxfisher-g
Copy link
Contributor

@maxfisher-g maxfisher-g commented Aug 10, 2023

This PR makes changes to the ResultStore type to simplify bucket URL and path processing.

The change is needed due to google/go-cloud#3294 which means special handling of file bucket URLs is required. Also fixes #811.

  • workaround for fileblob: os.Rename does not work between different mount points google/go-cloud#3294 by adding ?no_tmp_files=true to file bucket URLs
  • use URL object to parse and store bucket path on construction
  • remove extra parsing of bucket URL in cmd/analyze/main.go
  • rename the generatePath function to generateKey which takes in the filename and prefixes it with the package info if the constructPath option is used
  • remove and inline MakeFilename function which was only used in two places and with different options
  • add internal function for opening a bucket, which exposes it for unit tests
  • add unit tests for opening and writing to a file bucket (todo: test other types of bucket)

More unit tests should be added to test the bucket type used by the worker. This can be done in another PR before the next release.

Signed-off-by: Max Fisher <maxfisher@google.com>
Signed-off-by: Max Fisher <maxfisher@google.com>
…le buckets

Signed-off-by: Max Fisher <maxfisher@google.com>
Signed-off-by: Max Fisher <maxfisher@google.com>
…Store, fix up duplicate folder in bucket path, tweak tests

Signed-off-by: Max Fisher <maxfisher@google.com>
…nerateKey

Signed-off-by: Max Fisher <maxfisher@google.com>
@maxfisher-g maxfisher-g merged commit 04b86c6 into main Aug 15, 2023
9 checks passed
@maxfisher-g maxfisher-g deleted the resultstore_rework branch August 15, 2023 04:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Analysed package is not saved with correct name
2 participants