Skip to content

Commit

Permalink
fix(mistral): download all tokenizer files
Browse files Browse the repository at this point in the history
Mistral required some vocab file that was not downloaded anymore, maing
the slow tests fail.
  • Loading branch information
tengomucho committed Sep 24, 2024
1 parent 193953e commit 2d62758
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion optimum/tpu/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ def fetch_model(
local_path = snapshot_download(
repo_id=model_id,
revision=revision,
allow_patterns=["*.json", "model*.safetensors", SAFE_WEIGHTS_INDEX_NAME],
allow_patterns=["*.json", "model*.safetensors", SAFE_WEIGHTS_INDEX_NAME, "tokenizer.*"],
)
end = time.time()
logger.info(f"Model successfully fetched in {end - start:.2f} s.")
Expand Down

0 comments on commit 2d62758

Please sign in to comment.