Hanyang Li
@hanyang-li · 1 works
Recent posts by Hanyang Li
- Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization - Quantizing AdamW's optimizer states reduces persistent storage, but quantization errors propagate through the moment recurrences and perturb subsequent adaptive updates. We redesign 4-bit… · Paper · arxiv.org