Adding quantization support in torchtune

[HDCharles]

Stack from ghstack (oldest at bottom):

• int4 gptq working. #654
• -> Adding quantization support in torchtune #653

Summary:
Allows user to specify quantization_mode in generating model in full_finetune_single_device.py
and inference with the quantized model in generate.py

Test Plan:
tested locally

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/653

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 3 Unrelated Failures

As of commit 5a0472e with merge base 32d66df ():

NEW FAILURES - The following jobs have failed:

• Lint / lint (3.10) (gh)
  recipes/generate.py:42:2: E999 TabError: inconsistent use of tabs and spaces in indentation
• Multi-GPU Recipe Tests / recipe_test_multi_gpu (3.10) (gh)
  tests/recipes/test_eleuther_eval.py::TestEleutherEval::test_torchune_checkpoint_eval_results
• Multi-GPU Recipe Tests / recipe_test_multi_gpu (3.11) (gh)
• Multi-GPU Recipe Tests / recipe_test_multi_gpu (3.8) (gh)
  ##[error]The operation was canceled.
• Multi-GPU Recipe Tests / recipe_test_multi_gpu (3.9) (gh)
  ##[error]The operation was canceled.
• Recipe Tests / recipe_test (3.11) (gh)
  tests/recipes/test_eleuther_eval.py::TestEleutherEval::test_torchune_checkpoint_eval_results

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Recipe Tests / recipe_test (3.10) (gh)
  ##[error]The operation was canceled.
• Recipe Tests / recipe_test (3.8) (gh)
  ##[error]The operation was canceled.
• Recipe Tests / recipe_test (3.9) (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[kartikayk]

@HDCharles whats the difference between this PR and #632?

[jerryzh168]

this is a temporary PR, Charlie just wants to put up the gptq changes so I can add them to README, we can close this one