Pandora's AI Model Routing Box: Efficient Allocation with Costly Value Estimation
By Adam Fisch · Paper · cs.AI
Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing queries to the specialist who can answer most effectively at the lowest cost. Routing requires estimating each specialist's exp