Research & academia
Train Where the Quantized Model Goes: On-Policy Distillation for Low-Bit Reasoning
The authors target a reasoning failure in sub-3-bit models: quantization makes the model’s own generation paths diverge from the fixed prefixes used in QAD training.
Why it matters: For low-bit reasoning, training on the model’s actual quantized generation paths may address failures that continued teacher-forced QAD leaves behind.