Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Postmortem 4.1] Make mlperf.conf static in loadgen, enable automatic Pypi release #1882

Merged
merged 59 commits into from
Oct 22, 2024

Conversation

arjunsuresh
Copy link
Contributor

@arjunsuresh arjunsuresh commented Oct 21, 2024

This PR does the below changes:

Make loadgen automatically load the mlperf.conf file.

  1. mlperf.conf file is now distributed along with the loadgen source and is moved to the loadgen folder. There is a soft link at the repo root so that old workflows won't break. For this a MANIFEST.in file is added.
  2. Reference implementations are modified to remove the --mlperf_conf option.
  3. Restrict the keys that can be provided in the user.conf file.

Fix compilation warnings in loadgen

  1. Compilation warnings for loadgen except -Wunused-parameter are fixed.
  2. Github test for loadgen is modified to fail on any change which introduces a compilation warning.
  3. All the modified files are formatted to get the github action to pass.

Enabled Automatic pypi upload for loadgen wheels

  1. pip install mlcommons_loadgen can now be used to install loadgen on Linux, macOS and Windows
  2. A VERSION.txt file is added for loadgen for version info with an option for patch version
  3. The VERSION.txt file is automatically updated (only the patch version being updated) and new wheels released whenever there is a change inside the loadgen directory.
  4. https://pypi.org/manage/project/mlcommons-loadgen/releases/

Action item for the submitters

  1. Stop passing the mlperf.conf file to the loadgen - if this is done, mlperf_log_detail will have an error and the submission checker will fail.
  2. For python applications, pip install mlcommons_loadgen can be used

@arjunsuresh arjunsuresh requested a review from a team as a code owner October 21, 2024 12:31
Copy link

github-actions bot commented Oct 21, 2024

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@arjunsuresh arjunsuresh changed the title [Postmortem 4.1] Make mlperf.conf static in loadgen [Postmortem 4.1] Make mlperf.conf static in loadgen, fix Pypi release Oct 22, 2024
@arjunsuresh arjunsuresh changed the title [Postmortem 4.1] Make mlperf.conf static in loadgen, fix Pypi release [Postmortem 4.1] Make mlperf.conf static in loadgen, enable automatic Pypi release Oct 22, 2024
@mrmhodak mrmhodak merged commit ecb8801 into mlcommons:master Oct 22, 2024
12 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants