Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix submission checker for GPTJ for datacenter+edge #1483

Merged
merged 2 commits into from
Aug 17, 2023

Conversation

nvzhihanj
Copy link
Contributor

No description provided.

@nvzhihanj nvzhihanj requested a review from a team as a code owner August 16, 2023 22:26
@github-actions
Copy link

github-actions bot commented Aug 16, 2023

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@nvzhihanj
Copy link
Contributor Author

@pgmpablo157321 @mrasquinha-g @arjunsuresh can you reivew this?

@nv-ananjappa
Copy link
Contributor

@mrasquinha-g @mrmhodak Could you review and merge ASAP? It is quite an uncontroversial and small diff.

@arjunsuresh
Copy link
Contributor

Like bert-99.9, gptj-99.9 is datacenter only right?

@nvzhihanj
Copy link
Contributor Author

Like bert-99.9, gptj-99.9 is datacenter only right?

@arjunsuresh I think in the LLM WG, we approved GPTJ edge submission for both 99 and 99.9 accuracy threshold.

@arjunsuresh
Copy link
Contributor

@nvzhihanj IIRC gptj-99.9 was decided as datacenter only similar to bert-99.9. But if this is not the case, this change must be done in the edge category section also.

@nvzhihanj
Copy link
Contributor Author

@arjunsuresh we can discuss this in tomorrow's meeting. For now, I will add them to the optional scenarios. Does that make sense?

@arjunsuresh
Copy link
Contributor

@nvzhihanj I'm not sure about optional scenarios. @pgmpablo157321 what would you suggest?

@nvzhihanj
Copy link
Contributor Author

Added to required for now.

@mrasquinha-g mrasquinha-g merged commit 448c984 into mlcommons:master Aug 17, 2023
7 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Aug 17, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants