← all tags

#inference

4 posts tagged inference.

Thoughts

Reasoning phases are just expensive preprocessing with delusions of intelligence

Adding a 'thinking step' before generation is just prompt engineering disguised as architectural innovation.

reasoning inference performance

On-device inference just became the only game worth playing

Google killing TensorFlow Lite for LiteRT proves the industry has finally picked a side in the deployment wars.

edge-computing inference privacy mobile-ai

Tools & Experiments

LiteRT

Google's new universal on-device inference framework that replaces TensorFlow Lite and now supports PyTorch models.

experimental
edge-deployment mobile-ai inference tensorflow

NVIDIA AITune

Open-source toolkit that automatically finds the fastest inference backend for any PyTorch model.

experimental
pytorch inference optimization nvidia tensorrt