Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining
By Yuto Nishida · Paper · cs.CL
Measuring training data influence consistently across language model pretraining is challenging. It is difficult to select downstream tasks or validation sets representative of a model's general capabilities, and reliance on task performance at intermediate checkpoints complicate