LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

Reading Notes

Personal reading notes. Original paper: arXiv:2606.10531v1.

Paper Basic Information

• Paper Title: LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization
• Paper ID: 2606.10531v1
• Authors: Haoyu Wang, Xingyu Yu, Haiyan Zhao, Fengxiang Wang, Xu Han
• Conference: ICML 2026 (Proceedings of the 43rd International Conference on Machine Learning)
• Core Field: Large Language Models (LLM), Model Quantization, Quantization-Aware Training (QAT)

Research Background and Challenges

The paper points out that deploying Large Language Models (LLMs) on resource-constrained devices faces significant memory and computational challenges, making model quantization a key technology. Particularly in extremely low-bit (1-2 bit) scenarios, Quantization-Aware Training (QAT) generally outperforms Post-Training Quantization (PTQ) because it can adaptively adjust parameters to compensate for quantization errors.

However, existing QAT methods are primarily based on Scalar Quantization (SQ). The authors argue that while SQ has high optimization efficiency, it suffers from severe information loss at 2-bit precision, leading to performance degradation and requiring large amounts of training data to recover accuracy.

On the other hand, Vector Quantization (VQ) has higher representation capacity and can retain more information. However, integrating VQ into end-to-end QAT faces a core difficulty: traditional VQ involves time-consuming nearest neighbor search and discrete codebook lookup, making the quantization process non-differentiable and hindering gradient-based end-to-end optimization. Existing attempts rely on expensive coordinate descent or beam search, resulting in inefficient and asynchronous parameter updates.

Core Method: LC-QAT

To address the above issues, the authors propose a framework named LC-QAT. This is an end-to-end Vector Quantization-Aware Training framework specifically designed for 2-bit weight-only quantization.

Linear-Constrained Codebook Parameterization

The core innovation of LC-QAT lies in replacing unconstrained codebooks with linear-constrained parameterization. The paper proposes that codewords in each weight matrix can be generated by applying a shared linear mapping to discrete quaternary vectors.

Specifically, the codeword $c$ is generated by the following formula: $c = Az + B$ where $z \in \{0, 1, 2, 3\}^d$ is a discrete quaternary vector, and $A \in \mathbb{R}^{d \times d}$ and $B \in \mathbb{R}^d$ are floating-point parameters. This form defines a structured codebook where all codewords lie within an affine subspace determined by $A$ and $B$.

Differentiable Training Flow

Through this parameterization, LC-QAT converts discrete index selection into simple rounding and clipping operations, followed by linear projection. This allows gradients to propagate through the quantization process without explicit index search.

In Figure 1 provided in the paper, the authors compare two training flows:

• Traditional Unconstrained Codebook Training: Involves time-consuming Nearest Neighbor Search and Codebook Lookup; this process is Non-differentiable.
• LC-QAT Linear-Constrained Training: Uses Quaternary Vectors to generate weights via Direct Rounding and Linear Projection. This process is Differentiable, and no explicit codebook lookup is required during forward propagation.

This design enables vector-quantized weights to be trained under standard backpropagation, solving the fundamental limitation of VQ-QAT being difficult to optimize end-to-end.

Initialization Strategy and Optimization Analysis

The success of LC-QAT largely depends on high-quality initialization. The authors propose an initialization method based on linear-constrained codebooks, combined with the LDLQ algorithm and Hadamard transform to eliminate outliers.

Parameter Initialization

The initialization formulas for parameters $A$ and $B$ are as follows: $A = sG, \quad B = -sG\mu\mathbf{1}$ where $G$ is a random orthogonal matrix, $\mu$ is a centering constant, and $s$ is a scaling factor. This initialization ensures the codebook has approximately zero mean, unit variance, and weak inter-dimensional correlation.

Loss Landscape Analysis

To verify the quality of initialization, the authors analyzed the Loss Landscape. In Figure 2, the paper displays the cross-entropy loss landscape of the Qwen-3-1.7B model on WikiText-2:

• Figure 2a shows that the initialization point of LC-QAT is very close to the FP16 baseline, indicating significantly lower performance degradation. In contrast, initialization points based on SQ deviate significantly from the global minimum.
• Figure 2b shows that the LC-QAT initialization lies within a low-loss basin and exhibits a saddle point structure similar to that of the full-precision model.
• Figure 2c shows that SQ-based initialization deviates from the optimal region and lacks local minima nearby.

The authors believe this is because Vector Quantization retains more information during the post-training compression process. Therefore, LC-QAT starts optimization from a more favorable region in the parameter space, reducing the difficulty of subsequent fine-tuning and improving data efficiency.

Experimental Results and Conclusions

The paper conducted experiments on multiple large-scale LLMs and benchmarks. The main conclusions are as follows:

1. Data Efficiency: LC-QAT has extremely high data efficiency. Experiments show that the method can match or surpass state-of-the-art SQ and VQ-based QAT methods using only 0.1% to 10% of the training data.
2. Performance: LC-QAT outperforms previous VQ-QAT methods in final accuracy and significantly outperforms SQ-QAT methods in data efficiency. This advantage remains consistent across different benchmarks as the training budget increases.
3. Scalability: The method is established as a practical and scalable solution for extremely low-bit model deployment.

Summary

LC-QAT successfully solves the non-differentiable problem of Vector Quantization in QAT by introducing linear-constrained parameterization. This method not only provides high-quality PTQ initialization but also achieves end-to-end optimization without explicit codebook lookup. Experimental results confirm that LC-QAT can achieve high performance with minimal data at 2-bit precision, providing new insights for extreme low-bit deployment of LLMs.