Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify in tasks on which environment you depend. #750

Closed
ruben-arts opened this issue Feb 1, 2024 · 1 comment
Closed

Specify in tasks on which environment you depend. #750

ruben-arts opened this issue Feb 1, 2024 · 1 comment
Labels
✨ enhancement Feature request

Comments

@ruben-arts
Copy link
Contributor

Problem description

In the case of multiple machines you would like to run the same command. But pixi should figure out which environment to run. Either with some help or even automatically.

This could be a use-case for multiple machines running the same scripts on different hardware accelerated chipsets.
start can't depend on these tasks as they are ambiguous so you would have to define a start per env and then select that env. So this doesn't work, if you want start from default but train-model from cuda:

[tasks]
train-model = "python train.py"
evaluate-model = "python test.py"
start = { depends_on = ["train-model", "evaluate-model"] }

[feature.cuda.tasks]
train-model = "python train.py --cuda"
evaluate-model = "python test.py --cuda"

[feature.mlx.tasks]
train-model = "python train.py --mlx"
evaluate-model = "python test.py --mlx"

It would be great to have some sort of auto selection logic. Like checking which envs would be able to run on the current machine. And if that would not be enough let the user define which task it depends on.
E.g.: task_name = { depends_on = ["cuda:train_model"] }

@ruben-arts ruben-arts added the ✨ enhancement Feature request label Feb 1, 2024
@ruben-arts
Copy link
Contributor Author

For smart selection we could do something like:

[environments.cuda]
features = ["cuda"]
[environments.cuda.default-when]
system-requirements = { cuda = "12.2" }
platforms = [ "linux-64", "win-64" ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ enhancement Feature request
Projects
None yet
Development

No branches or pull requests

1 participant