Proteus: Incremental Memory Activation for Long-Context Sequence Modeling
By Reza Bayat · Paper · cs.LG
The quadratic cost of attention-based sequence models for long contexts has motivated a growing line of research on memory-based models that can compress context into a compact state. However, most existing memory models expose a static memory throughout the entire sequence. Beca