Naive Run Compressed Support

[Satrat]

Summary of Changes

• Refactor compressors to run on pytorch modules and single layer state_dicts. Only the base class implements the loop through the state dict
• Add new CompressedLinear layer type for running in compressed mode
• Rather than decompressing on model load, we decompress on each forward pass. This can later be swapped out for specific kernels but decompressing will be the fallback case
• Allow for being able to run a forward pass even if a zero point isn't loaded (its assumed to be 0)

Example

from transformers import AutoTokenizer
from llmcompressor.transformers import SparseAutoModelForCausalLM
import torch

model_dir = "nm-testing/Meta-Llama-3-8B-Instruct-fp8-compressed"
model = SparseAutoModelForCausalLM.from_pretrained(model_dir, torch_dtype=torch.float16, device_map="auto", run_compressed=True)

tokenizer = AutoTokenizer.from_pretrained(model_dir)
sample_input = ["I love 4 bit quantization because"]
inputs = tokenizer(sample_input, return_tensors="pt").to(model.device)
generated_ids = model.generate(**inputs, max_length=50)
outputs = tokenizer.batch_decode(generated_ids)
print(outputs)

Output: ["<|begin_of_text|>I love 4 bit quantization because it's so simple and yet so effective. It's a great example of how you can achieve a good balance between complexity and performance in a model.\n\nI think it's also worth noting that 4-bit"]

[bfineran]

LGTM pending unresolved comment