← all tags

#training-optimisation

4 posts tagged training-optimisation.

News & Updates

AI digest: Training gets faster, governance lags behind

New training techniques slash compute time whilst most companies still have no AI policies in place.

ai-news digest training-optimisation enterprise-ai

Thoughts

Training efficiency just became a hardware marketing scheme

Fused kernels and optimised attention mechanisms are turning genuine performance gains into vendor lock-in disguised as technical innovation.

training-optimisation hardware-acceleration performance-engineering

Training optimisers just became the most important job nobody talks about

While everyone obsesses over model parameters, the algorithms that actually train them are quietly becoming the biggest bottleneck in AI development.

training-optimisation model-architecture infrastructure

Tools & Experiments

Lighthouse Attention

A training-only attention mechanism that delivers 1.4-1.7x speedup for long context pretraining.

experimental
attention-mechanism training-optimisation long-context nous-research