SLORR: Simple and Efficient In-Training Low-Rank Regularization
By David González-Martínez · Paper · cs.LG
Low-rank factorization is widely used to compress neural networks, but modern models are often not naturally amenable to aggressive factorization without significant accuracy loss. Existing training-time low-rank regularizers can improve compressibility, but they often require SV