Modify LLM Trainer to support BERT and Tiny LLaMA (kubeflow#2031)

* Modify LLM Trainer to support BERT and Tiny LLaMA Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Access PVC access modes to storage config Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Format Python files Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Distribute datasets Fix access modes in storage config Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Update example to fine tune BERT with Train API Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Remove dataset download twice Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> --------- Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
deepanker13 · Apr 8, 2024 · 30cdba5 · 30cdba5
1 parent ecbeefe
commit 30cdba5
Show file tree

Hide file tree

Showing 16 changed files with 1,072 additions and 791 deletions.
diff --git a/.github/workflows/test-python.yaml b/.github/workflows/test-python.yaml
@@ -18,7 +18,7 @@ jobs:
       - name: Check Python code with Black
         uses: psf/black@stable
         with:
-          version: 23.9.1
+          version: 24.2.0
           options: --check --exclude '/*kubeflow_org_v1*|__init__.py|api_client.py|configuration.py|exceptions.py|rest.py'
           src: sdk/
 

diff --git a/...ication/Train CNN with FashionMNIST.ipynb → ...ication/Train-CNN-with-FashionMNIST.ipynb b/...ication/Train CNN with FashionMNIST.ipynb → ...ication/Train-CNN-with-FashionMNIST.ipynb