5 Ultimate On-Device Inference Frameworks with Liquid AI
Executive Summary (TL;DR): Liquid AI just shipped LFM2.5-230M , a liquid neural network designed explicitly for on-device inference . The model comes with first‑class support for five battle‑tested runtimes: llama.cpp , MLX , vLLM , SGLang , and ONNX . We put each framework through a real‑world wringer on an M2 MacBook Air and a Jetson Orin Nano. Below you’ll find concrete YAML snippets , CLI commands , and war stories that cut through the marketing fluff. We still remember the day a deployment to a fleet of edge cameras failed because the “lightweight” transformer choked on 512 tokens. That experience taught us to respect on-device inference not as a buzzword but as a brutal hardware constraint. When Liquid AI dropped LFM2.5-230M with simultaneous support for five inference frameworks, we knew we had to validate it immediately. Not on a cloud GPU with infinite memory – but on the very same M2 MacBook Air and Jetson Orin Nano sitting on our desks. The model itself is a liq...