TokTier: Exact Stateful Tokenization for Agentic LLM Serving
By Zhenyu Zhang · Paper · cs.CL
LLM serving systems cache prompt KV state, yet most front ends still re-tokenize the full request text on every call. The cost lands on coding agents, which resubmit a long transcript after each small tool result, and reuse is hard because even a short append can change token bou