Liquid AI LFM2.5 QAD Review 2026: 4-Bit Quantization Keeps 97% Quality — Edge AI Breakthrough
Verdict first: Liquid AI's LFM2.5 QAD checkpoints (released August 20, 2026) are the most interesting edge-AI release of the month. By building quantization into training rather than bolting it on after, the QAD technique keeps 96.5%–97.4% of the BF16 teacher's quality in 4-bit form — the old "aggressive quantization costs visible quality" trade-off largely disappears at small scale. The 2.6B model is only ~1.3GB after quantization, runs on a phone at 213 tokens/s and a Raspberry Pi at 42 tokens/s. If you deploy LLMs on the edge, this is a genuine breakthrough worth your attention.
LFM2.5 QAD — Liquid AI's quantization-aware distillation brings near-full precision to edge devices
What Is LFM2.5 QAD?
Released on August 20, 2026, the LFM2.5 QAD Q4_0 checkpoints cover four models — 230M, 350M, 1.2B-Instruct, and 2.6B — all published as GGUF on Hugging Face for direct use with llama.cpp. They join a series that also includes the LFM2.5-8B-A1B (BenchLM's top-rated model) and VL vision-language variants.
The positioning is clear: this is Liquid AI's play for edge and on-device inference — small models that run where big models can't.
Specifications at a Glance
| Item | Value |
|---|---|
| Vendor | Liquid AI |
| Release date | August 20, 2026 (QAD Q4_0 checkpoints) |
| Positioning | Edge / on-device AI deployment |
| Models | 230M · 350M · 1.2B-Instruct · 2.6B |
| Core tech | QAD — Quantization-Aware Distillation |
| Context window | 32K (230M/350M/1.2B) · 128K (2.6B) |
| Format | GGUF (llama.cpp compatible) |
| Open weights | Hugging Face, $0/$0 hosted pricing |
| 2.6B quantized size | ~1.3GB |
The QAD Breakthrough
Traditional post-training quantization (PTQ) compresses a finished model and loses precision. QAD instead bakes the quantization constraint into training: it distills a high-precision BF16 teacher directly into a Q4_0 student. The result — quality retention across the four models:
| Model | Quality retained | Matches quality of |
|---|---|---|
| 230M | 97.1% | Q5_K_M |
| 350M | 96.5% | Q5_K_M |
| 1.2B | 97.4% | Q4_K_M |
| 2.6B | 96.6% | Q4_K_M |
The 4-bit checkpoints match or beat standard Q4_K_M, and match Unsloth's UD-Q4_K_XL external baseline — while being 4–33% faster than the equivalent-higher-precision quantizations at native Q4_0 memory and speed. The era where "4-bit means visibly worse" may be ending for small models.
Edge Deployment Tests (Real Hardware)
| Device | Measured speed |
|---|---|
| Samsung Galaxy S25 Ultra (phone) | 213 tokens/s (230M) |
| Raspberry Pi 5 | 42 tokens/s (230M) |
| MacBook Pro (Apple M5 Max) | Tested OK |
| NucBox (AMD Ryzen AI Max+) | Tested OK |
From phones to Raspberry Pi, the models genuinely run on edge hardware. Benchmarks cover reasoning, instruction-following, tool use, and agentic ability (GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, BFCLv4, plus GSM8K/AIME25 math).
Pros & Cons
Pros
- ✅ QAD keeps 96.5–97.4% quality at 4-bit — a real breakthrough
- ✅ 2.6B only ~1.3GB after quantization, no dedicated GPU needed
- ✅ Runs on phones and Raspberry Pi — true edge deployment
- ✅ Fully open GGUF weights, llama.cpp ready, $0/$0 API
- ✅ Small models punch above their weight on data-extraction tasks
Cons
- ❌ Small models (230M–8B) — not frontier intelligence
- ❌ Smaller ecosystem and market influence vs big labs
- ❌ 2.6B-Base has no inference provider yet — you self-host
- ❌ Some figures vendor-reported, awaiting independent validation
Who Should Use It (And Who Shouldn't)
Great fit
- Edge / on-device AI developers (mobile, IoT, embedded)
- Teams needing local small-model deployment
- Researchers exploring quantization distillation
- Makers running AI on Raspberry Pi or old hardware
Poor fit
- Users needing frontier-level intelligence (choose GPT-5.6 / Fable 5)
- Multimodal / very-long-context workloads
- Large-scale cloud production (limited ecosystem/providers)
Frequently Asked Questions
What is LFM2.5 QAD?
Liquid AI's edge-AI model series; the QAD Q4_0 checkpoints (230M–2.6B) released Aug 20, 2026 keep 96.5–97.4% of BF16 quality at 4-bit.
What is QAD?
Quantization-Aware Distillation — builds quantization into training, distilling a BF16 teacher into a Q4_0 student to recover most precision loss.
How big is LFM2.5-2.6B?
2.69B params, 128K context, ~1.3GB after 4-bit quantization — runs without a dedicated GPU.
Is it open source?
Yes — GGUF on Hugging Face, llama.cpp ready, no official API pricing ($0/$0), self-host at hardware cost.
How fast on edge?
213 tokens/s on Galaxy S25 Ultra, 42 tokens/s on Raspberry Pi 5 (230M model).
Is it a frontier model?
No — small models (230M–8B) for edge deployment, not flagship intelligence.
Sources
- Liquid AI official blog — LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment
- The Agent Times — Liquid AI Ships 4-Bit Quantized Models Claiming 97% of BF16 Quality
- BenchLM — LiquidAI AI Models: Benchmarks & Pricing
- AIHOT Daily 2026-08-20 — Liquid AI releases LFM2.5 QAD Q4_0 checkpoints
- 2libra AI Daily — 2026-08-20
没有评论:
发表评论