You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Submitters (i.e., those who have submitted results for review and verification by MLCommons) are not allowed to publish any results for a given benchmark version before its official publication date.
For example, if an organization only submits ResNet50 results to the upcoming v3.1 round, they are not allowed to disclose these results until the publication date. (However, they can seemingly obtain a similar set of results for a previous version of the benchmark e.g. v3.0, and disclose them as unverified results anytime they like, including during the v3.1 review period.)
Non-submitters (i.e., those who have not submitted results for review and verification by MLCommons) are not allowed to publish any results until two weeks after the official publication date for that benchmark version.
Now, if the same organization , who has only submitted ResNet50 results, obtains some unverified results, say, for GPT3, can they disclose them right after the official publication date, without waiting for two weeks?
On the one hand, the answer could be yes: this organization is a submitter, since they have submitted something to this round, just not this benchmark. On the other hand, the answer should be no: they have submitted nothing for this benchmark.
The text was updated successfully, but these errors were encountered:
The MLPerf Results Messaging Guidelines contain the following:
For example, if an organization only submits ResNet50 results to the upcoming v3.1 round, they are not allowed to disclose these results until the publication date. (However, they can seemingly obtain a similar set of results for a previous version of the benchmark e.g. v3.0, and disclose them as unverified results anytime they like, including during the v3.1 review period.)
Now, if the same organization , who has only submitted ResNet50 results, obtains some unverified results, say, for GPT3, can they disclose them right after the official publication date, without waiting for two weeks?
On the one hand, the answer could be yes: this organization is a submitter, since they have submitted something to this round, just not this benchmark. On the other hand, the answer should be no: they have submitted nothing for this benchmark.
The text was updated successfully, but these errors were encountered: