Latent Compaction with Test-Time Training
By Yingfa Chen · Paper · cs.CL
Long contexts enlarge key--value (KV) caches, increasing memory use and attention costs during generation. Compressing these caches must preserve information for unseen questions. We introduce **Latent Compaction**, which learns a compact document-specific cache through test-time
Tags: Cs.cl