Code

Basic Llama2 Tuning#39

Code

Merged

tengomucho merged 10 commits intomainhuggingface/optimum-tpu:mainfrom basic-traininghuggingface/optimum-tpu:basic-trainingCopy head branch name to clipboardMay 15, 2024

+5,430-18

Commits

Commits on May 6, 2024

• chore: import run_clm from transformers

  Show description for 5fcebf6
  tengomucho

  committed
  5fcebf6

  Copy full SHA for 5fcebf6

Commits on May 7, 2024

• feat(llama): torch_dtype is now bfloat16

  Show description for 1990414
  tengomucho

  committed
  1990414

  Copy full SHA for 1990414

Commits on May 12, 2024

• chore(llama): added changes to prepare for spmd

  tengomucho

  committed
  8570131

  Copy full SHA for 8570131

• chore: import trainer from transformers

  Show description for 3194a1c
  tengomucho

  committed
  3194a1c

  Copy full SHA for 3194a1c

Commits on May 13, 2024

• feat(xla model parallel): use Linear on SPMD

  Show description for f721d7c
  tengomucho

  committed
  f721d7c

  Copy full SHA for f721d7c

• feat(llama): pick Linear implementation using create

  tengomucho

  committed
  ef50868

  Copy full SHA for ef50868

• fix(llama): disable cache when using SPMD

  Show description for 5001cad
  tengomucho

  committed
  5001cad

  Copy full SHA for 5001cad

• feat(training): adapt trainer and run_clm to optimum and SPMD

  tengomucho

  committed
  6b3c3b8

  Copy full SHA for 6b3c3b8

• feat(examples): add a short example on llama fine-tuning

  tengomucho

  committed
  bd772b9

  Copy full SHA for bd772b9

Commits on May 14, 2024

• feat(examples): add Llama3-8b tuning instructions

  Show description for 3a12371
  tengomucho

  committed
  3a12371

  Copy full SHA for 3a12371