I'm kinda surprised that in 2026 we're still stuck with tokenization for AI models. For a while, it looked like byte-le…
I'm kinda surprised that in 2026 we're still stuck with tokenization for AI models. For a while, it looked like byte-level designs such as ByT5 or Meta's BLT might replace tokens altogether, letting models ingest raw bytes directly. But apparently tokens aren't going anywhere anytime soon. For better or worse, they seem here to stay.