#inference-scaling

Memory compression is just bandwidth denial pretending to be breakthrough

Google's TurboQuant and the rush to compress KV caches are treating symptoms whilst ignoring the real problem.