#knowledge-distillation

Knowledge distillation is just academic procrastination disguised as optimisation

We're spending months teaching small models to mimic ensemble behaviour instead of just building better single models from the start.