Structured Memory for Edge Language Models: Persistent Context and Corpus Retrieval via O(1) SSM State Injection
By Anusha Madan Gopal · Paper · cs.LG
Retrieval-augmented generation (RAG) imposes a prefill cost proportional to retrieved context length, and -- with Transformer backbones -- a KV-cache that grows with each generated token. State-Space Models (SSMs) avoid the second cost by construction; we eliminate the first, col