Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Client support for competitions #231

Merged
merged 26 commits into from
Jan 10, 2025
Merged

Feature: Client support for competitions #231

merged 26 commits into from
Jan 10, 2025

Conversation

Andrewq11
Copy link
Contributor

Changelogs

  • Client methods for downloading a competition from the Hub and submitting predictions
  • New CompetitionSpecification class that inherits from the DatasetV2 class
  • New CompetitionPredictions class that inherits from the BenchmarkPredictions class for standardized prediction uploads
  • Removal of old competition code (e.g. differing URLs based on the ArtifactSubtype)
  • Removal of the CompetitionResults class since these results never reach the client currently
  • New method for loading a competition that accepts a local Zarr path for cached dataset downloads

Checklist:

  • Was this PR discussed in an issue? It is recommended to first discuss a new feature into a GitHub issue before opening a PR.
  • Add tests to cover the fixed bug(s) or the newly introduced feature(s) (if appropriate).
  • Update the API documentation if a new function is added, or an existing one is deleted.
  • Write concise and explanatory changelogs above.
  • If possible, assign one of the following labels to the PR: feature, fix, chore, documentation or test (or ask a maintainer to do it for you).

Important Notes

This PR is ready in terms of features, but is currently missing:

  • Documentation/tutorials for these changes
  • Updated tests

I will leave this PR as a draft to gather comments while I work on these next.

@Andrewq11 Andrewq11 added the feature Annotates any PR that adds new features; Used in the release process label Dec 10, 2024
@Andrewq11 Andrewq11 self-assigned this Dec 10, 2024
Copy link
Collaborator

@cwognum cwognum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Andrewq11 !

I left some comments. Nothing major!

polaris/competition/_base.py Outdated Show resolved Hide resolved
polaris/benchmark/_base.py Outdated Show resolved Hide resolved
polaris/evaluate/_predictions.py Outdated Show resolved Hide resolved
polaris/evaluate/_predictions.py Show resolved Hide resolved
polaris/loader/load.py Outdated Show resolved Hide resolved
polaris/competition/_base.py Outdated Show resolved Hide resolved
polaris/competition/_base.py Outdated Show resolved Hide resolved
polaris/competition/_base.py Outdated Show resolved Hide resolved
polaris/competition/_base.py Outdated Show resolved Hide resolved
polaris/competition/_base.py Outdated Show resolved Hide resolved
@Andrewq11
Copy link
Contributor Author

@cwognum going to mark this ready for review despite missing those couple docs and tutorial. Everything else should be ready 💪

@Andrewq11 Andrewq11 marked this pull request as ready for review December 12, 2024 19:25
Copy link
Contributor

@jstlaurent jstlaurent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, nice work. I did have some questions/suggestions. Since Andrew is out, I can take a stab at implementing them.

polaris/competition/_base.py Outdated Show resolved Hide resolved
polaris/competition/_base.py Outdated Show resolved Hide resolved
polaris/hub/client.py Show resolved Hide resolved
polaris/loader/load.py Outdated Show resolved Hide resolved
polaris/hub/client.py Outdated Show resolved Hide resolved
@cwognum
Copy link
Collaborator

cwognum commented Dec 17, 2024

@jstlaurent If you have the time, it would be great if you could take a stab at this. I was planning to take over from Andrew, but there are some other things on my plate I would really like to wrap before EOY so we can hit the ground running in 2025.

polaris/benchmark/_split.py Outdated Show resolved Hide resolved
Copy link
Contributor

@jstlaurent jstlaurent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nitpick. Looks good. 🚢 it!

polaris/benchmark/_task.py Show resolved Hide resolved
@cwognum cwognum merged commit 0df06a0 into main Jan 10, 2025
10 checks passed
@cwognum cwognum deleted the feat/competitions-v2 branch January 10, 2025 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Annotates any PR that adds new features; Used in the release process
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants