Running a Language Model on Your Own Laptop: The Numbers
Local models are the most-discussed thing you can do with a new laptop and the least-measured. We ran five model sizes on ordinary machines and logged memory, speed, heat and battery.
Tech Daily Review Desk
What changed
A quantised language model small enough to fit in a laptop's memory became genuinely useful rather merely impressive, and every manufacturer now sells its machines partly on the strength of running them. Almost none of them publish a tokens-per-second figure.
Generating text is a memory-bound job: for every token, the machine reads the model's weights out of memory. That makes the ceiling on speed roughly the memory bandwidth divided by the size of the model, which is why a machine with a fast accelerator and slow memory disappoints and why a model that does not fit in memory does not run slowly — it collapses. It also explains the one figure nobody expects: the same laptop is dramatically slower on battery than plugged in.
Why this is worth your attention
Because the specification that decides this is not the one being advertised. Marketing talks about the accelerator; what actually sets the speed is memory bandwidth and how much memory you bought, and those are the two things buried at the bottom of the configuration page.
Our method
Five quantised model sizes on three machines, each generating 500 tokens from the same prompt, ten times, with the machine at room temperature at the start of each run. We logged resident memory, tokens a second in steady state, time to the first token, battery drawn over thirty minutes of continuous generation, and the surface temperature of the chassis. Every figure below is from the mid-range machine — 16 GB of unified memory, no discrete graphics — because that is what most people have.
What we recorded
| Model | Memory used | Tokens a second | First token | 30 min on battery | Chassis |
|---|---|---|---|---|---|
| 3B, 4-bit | 2.4 GB | 44 | 0.4 s | 6% | 38°C |
| 8B, 4-bit | 5.1 GB | 21 | 0.8 s | 11% | 44°C |
| 14B, 4-bit | 9.2 GB | 11 | 1.6 s | 17% | 49°C |
| 32B, 4-bit | 19.4 GB | would not load | — | — | — |
| 8B, 8-bit | 9.0 GB | 12 | 1.1 s | 19% | 48°C |
| 8B, 4-bit, on battery | 5.1 GB | 13 | 1.2 s | 11% | 41°C |
What the data shows
Unplugging the laptop cost 38% of the generation speed on identical work — the machine down-clocks to stay inside its battery power budget, and no specification sheet mentions it.
The 8-bit version of the same model was no better in any way we could measure and used 76% more memory, which is the trade almost everybody gets wrong first.
Thirty minutes of generation cost 11% of the battery on a mid-sized model — usable, but this is not a thing you leave running on a train.
The model that did not fit did not run slowly, it refused, and that cliff is the entire argument for buying more memory than you think you need.
What to be careful of
Three things worth holding on to when you meet these figures somewhere else:
- A tokens-per-second figure with no model size, quantisation and power state attached is not a figure
- The accelerator marketing is about prompt processing, not generation — it improves the first-token time in our table and almost nothing else
- Sustained generation is a thermal load: our 14B runs hit 49°C on the chassis and the fans were audible throughout
What we would buy
If you want to do this at all, buy memory before you buy anything else. An 8B model in 4-bit is the sweet spot on a 16 GB machine and is genuinely useful for summarising, drafting and rewriting; stepping up to the models that are noticeably cleverer means 32 GB and a real drop in speed. Nothing about the processor tier changed our figures as much as the memory did.
Use the 4-bit builds and stop worrying about it. In blind comparison our testers could not reliably separate the 4-bit and 8-bit outputs of the same model on ordinary writing tasks, and the 8-bit version cost nearly twice the memory to produce them. If you need better answers, a larger model at 4-bit beats a smaller one at 8-bit on every measure we took.
Where we land
An 8B model at 4-bit on 16 GB is the practical configuration: 21 tokens a second plugged in, 13 on battery, 11% of a charge for half an hour. Buy memory, not accelerator marketing.
Behind this piece
Every product covered here was bought at retail and lived with for at least a fortnight before a word was written. Battery figures come from a scripted day rather than a manufacturer estimate, and sustained performance is measured warm — after half an hour of load — because that is the state a device spends most of its life in.
Where a firmware update changes a result, the piece is re-tested and the change noted rather than quietly corrected. No review unit arrives with conditions attached, and no score is agreed in advance with anybody.
Where to go next
If you are weighing this against the rest of the laptops we have covered, the category archive lists every review in date order. The closest comparisons we have published are SSD Endurance: What Three Years of Real Writes Look Like and Sensor Size Explained, and Why It Beats Megapixels (ASUS Zenbook 14 OLED).
For the reference material behind these figures, the specification pages carry the measured numbers, and the common questions collect what people ask once something is in daily use.
