Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle unfetchable Go versions that have name collisions. #2687

Merged
merged 3 commits into from
Jan 25, 2021
Merged

Handle unfetchable Go versions that have name collisions. #2687

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
673b4a1
Skip versions that are returning nothing from go proxy bc of name col…
tiegz Jan 24, 2021
40b8a97
More terse comment
tiegz Jan 25, 2021
01fef3f
Keep unscrapeable versions but mark published_at as nil.
tiegz Jan 25, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions app/models/package_manager/go.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,14 @@ def self.versions(project, _name)
known_versions[v].slice(:number, :published_at)
else
info = get("#{PROXY_BASE_URL}/#{project[:name]}/@v/#{v}.info")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any way you can think of to correct to the proper case before updating? Some sort of case insensitive search mechanism?

Copy link
Contributor Author

@tiegz tiegz Jan 25, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unfortunately an issue with the files in the Go Module and not with the name itself so I think we have to wait on a fix from Golang 😬 Here's the response for the example one above, where it has two filepaths in the same folder with same-name-different-casing:

not found: create zip: case-insensitive file name collision: "models/apis/GuardDuty/2017-11-28/api-2.json" and "models/apis/guardduty/2017-11-28/api-2.json"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(or alternatively we could keep the version but just display its published_at as ? or something?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a really really small amount of packages huh? (Like 2 maybe?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this example had 3 versions with name collisions, but we've got a ton of these errors in bugsnag: https://app.bugsnag.com/tidelift/libraries-dot-io/errors/60076c589e7ce30018c6b586

# Store nil published_at for known Go Modules issue where case-insensitive name collisions break go get
# e.g. https://proxy.golang.org/github.com/ysweid/aws-sdk-go/@v/v1.12.68.info
version = info.nil? ? v : info["Version"]
published_at = info && info["Time"].presence && Time.parse(info["Time"])

{
number: info["Version"],
published_at: info["Time"].presence && Time.parse(info["Time"]),
number: version,
published_at: published_at,
}
end
rescue Oj::ParseError
Expand Down