Beyond Shadow Weights: Quantization-Aware Training as Quantized-Endpoint Descent
Quantization-aware training (QAT) updates a full-precision shadow weight \(\mathbf{x}\) but deploys the quantized endpoint \(Q(\mathbf{x})\). Existing explanations for QAT largely view its success through the lens of shadow weights: QAT can move \(\mathbf{x}\) toward flatter basins, gain robustness from quantization-induced oscillations, or balance the shadow loss \(f(\mathbf{x})\) against the quantization error \(\|\mathbf{x}-Q(\mathbf{x})\|_2\). These perspectives do … Read more