thoughtful piece! even granting all of it, distillation doesn’t solve the compute problem. it’s a post training shortcu…
By signüll · Research
thoughtful piece! even granting all of it, distillation doesn’t solve the compute problem. it’s a post training shortcut.. it maybe gets you reasoning traces & instruction following polish cheap (questionable still), which maybe compresses the gap. but it’s still very difficult for me to believe distillation has more than a marginal impact on kimi k3, if that… can someone convince me otherwise?