#model-compression
2 posts tagged model-compression.
Thoughts
Knowledge distillation is just academic procrastination disguised as optimisation
We're spending months teaching small models to mimic ensemble behaviour instead of just building better single models from the start.
Parameter efficiency is just premature optimisation disguised as innovation
The obsession with minimal parameters is solving yesterday's problems whilst creating tomorrow's technical debt.