Skip to content

Commit

Permalink
update doc
Browse files Browse the repository at this point in the history
  • Loading branch information
jerryzh168 committed Sep 19, 2024
1 parent dc76271 commit f4d7c51
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/source/en/quantization/torchao.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ quantized_model.save_pretrained(output_dir, safe_serialization=False)
# quantized_model.push_to_hub(save_to, safe_serialization=False)

# load quantized model
ckpt_id = "llama3-8b-int4wo-128" # or huggingface hub id
ckpt_id = "llama3-8b-int4wo-128" # or huggingface hub model id
loaded_quantized_model = AutoModelForCausalLM.from_pretrained(ckpt_id, device_map="cuda")


Expand Down

0 comments on commit f4d7c51

Please sign in to comment.