Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic Slurm Container job #298

Merged
merged 22 commits into from
Nov 18, 2024
Merged

Generic Slurm Container job #298

merged 22 commits into from
Nov 18, 2024

Conversation

amaslenn
Copy link
Contributor

@amaslenn amaslenn commented Nov 11, 2024

Summary

Introduce a new workload type: GenericSlurmContainer.

  1. Works with Slurm only and have limited set of arguments to set.
  2. Expanded support for Installables to GitRero, previously it was supported only via PythonExecutable.

Test Plan

CI

Additional Notes

@amaslenn amaslenn marked this pull request as ready for review November 13, 2024 11:10
@TaekyungHeo TaekyungHeo added enhancement New feature or request Jan25 Jan'25 release feature labels Nov 13, 2024
Copy link
Contributor

@srivatsankrishnan srivatsankrishnan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the call, this makes sense for the VFM use case currently we have. We cannot use existing Nemo since VFM uses Nemo 2.0. This can be generic and currently the PR has two repos and hashes. These fields can be made optional to make it truly generic. Future PR will address this.

Also the report generation logic also needs to be updated accordingly to make it truly generic. But for this VFM use case, this makes sense as agreed in the call.

@srivatsankrishnan srivatsankrishnan merged commit d61a046 into main Nov 18, 2024
2 checks passed
@srivatsankrishnan srivatsankrishnan deleted the am/slurm-cont branch November 18, 2024 16:17
@amaslenn amaslenn restored the am/slurm-cont branch November 18, 2024 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feature Jan25 Jan'25 release feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants