On the estimation and validity of AI time horizons---a statistical look at the METR plot
By Drew T. Nguyen · Paper · cs.AI
METR's 50\% time horizon measures the human completion time of software tasks that an AI solves with 50\% probability, allowing AI capabilities to be expressed in interpretable units. On 228 tasks and 26 AIs, we recompute the time horizons using splines and item-response theory t
Tags: Cs.ai