A fascinating new paper by @GoogleResearch argues that language models need sleep. Instead of remaining frozen after tr…
A fascinating new paper by @GoogleResearch argues that language models need sleep. Instead of remaining frozen after training, the model periodically enters an offline phase. It consolidates fragile in-context memories into long-term parameters, expands its capacity, then "dreams" up synthetic training data to rehearse knowledge and improve itself. The proof of concept beats SFT and GRPO on several math benchmarks, reaches 80% on few-shot abstract reasoning versus 72.5% for SEAL, and achieves…