[v0.3.3] Release Tracker

[WoosukKwon]

ETA: Feb 29th - Mar 1st

Major changes

• StarCoder2 support
• Performance optimization and LoRA support for Gemma
• Performance optimization for MoE kernel
• 2/3/8-bit GPTQ support
• [Experimental] AWS Inferentia2 support

PRs to be merged before the release

• Add Support for 2/3/8-bit GPTQ Quantization Models #2330 Add GPTQ quantization kernels for 2, 3, 8-bit use cases #2223
• GPTQ & AWQ Fused MOE #2761
• Add guided decoding for OpenAI API server #2819
• Use of logits_processors has become very slow in v0.3.2 #3087 [Fix] Don't deep-copy LogitsProcessors when copying SamplingParams #3099
• Support starcoder2 architecture #3089

[simon-mo]

#2819
#3087
Starcoder2

[njhill]

#3099 is a fix for the #3087 regression

[hanzhi713]

#2760 Fixes for custom all reduce on some platforms

[robertgshaw2-redhat]

#2497 Adds support for Marlin INT4 kernels ~3x faster than current GPTQ kernels

[HyperdriveHustle]

#3016
Fix: Output text is always truncated in some models