Note: This repository is currently under heavy development.
torchtrain contains PyTorch native parallelisms, tools and utilities to train large models.
Install PyTorch from source or install the latest pytorch nightly, then install requirements by
pip install -r requirements.txt
Install additional dev requirements if you want to contribute to the repo:
pip install -r dev-requirements.txt
run the llama debug model locally to verify the setup is correct:
./run_llama_train.sh