Advancing low‑bit quantization for LLMs: AutoRound x LLM Compressor

AutoRound, Intel's post-training quantization algorithm, is now integrated into LLM Compressor, enabling high-accuracy low-bit quantization (W4A16) for large language models with minimal tuning overhead. The integration supports sequential calibration, produces compressed tensors compatible with vLLM serving, and works across Intel hardware (Xeon, Gaudi, Arc GPUs) and CUDA devices. The workflow requires just 128 calibration samples and ~200 iterations, with future plans to support FP8, MXFP4/8, mixed-bit search, and MoE models.