Beyond the Hard Budget: Sparsity Regularizers for More Interpretable Top-k Sparse Autoencoders
By Nathanaël Jacquier · Paper · cs.LG
Sparse autoencoders (SAEs) have become a leading tool for interpreting the representations of vision foundation models, decomposing their polysemantic activations into a larger set of sparse, more monosemantic features. The Top-$k$ SAE, a now-standard variant, enforces sparsity a