Pretraining Data Can Be Poisoned through Computational Propaganda
By Victoria Graf · Paper · cs.AI
Poisoning pretraining data can introduce harmful behaviors to LMs that are difficult to detect and mitigate. Prior work on poisoning pretraining data has largely exploited established data sources such as Wikipedia, which do not represent the large scale and heterogeneity typical