02 - The cheapest local AI box that actually works

The parts, the honest numbers, what fought me, and where it falls short

[drafted by a project agent, edited by me, by hand]

I wanted a local AI box the way I’d want any tool in the shop: owned, understood, serviceable by me. No subscription, no cloud, nothing phoning home. So I built the cheapest one I could that still does real work all day. Not the fastest box… the cheapest one that’s actually viable. Here’s the whole thing, with the real numbers and the parts that fought me.

The shape of it

A Ryzen 9 mini-PC (mine’s a Minisforum UM890 Pro) which stays a normal everyday computer, and an Arc Pro B60 24GB rides alongside it on an open-air eGPU dock with its own 650W Gold PSU, connected over OCuLink. Two machines wearing one job. The mini-PC never gives up its own silicon to the model, and if the dock ever unplugs you’ve still got a first-rate desktop. If your host has no native OCuLink port, you can enumerate the card off an M.2-to-OCuLink adapter, just keep the cable short.

The part that fought me

Getting the card to enumerate cleanly meant sorting a small-BAR issue during bring-up. Once that was handled it comes up clean every boot. The link is Gen4-capable and I’ve watched it train there; this boot is sitting at Gen3 x4 under load, and every number in this post reproduces at Gen3. the weights live in VRAM, so the link isn’t your bottleneck either way.

The real lesson: it’s the model, not the card

I came from a dense 32B doing 15.5 tok/s and it was a slog. I swapped to a 35B-A3B MoE and the same card runs it fully offloaded: about 42 tok/s at low context on my daily serving config (q8 KV cache + flash attention), and around 8K depth it holds 29–32 tok/s. (A shallow-KV bench config reads higher, ~45 I’m quoting the as-served number on purpose, because a tok/s figure without its config is half a number.) At 8K the gap gets embarrassing: about 30 versus the dense 32B’s 7.15. better than 4x, because only ~3B parameters are active per token. If you’re budget-building, the MoE is the upgrade; the GPU is just the enabler.

First-hand Arc gotcha while you’re benching: a q8 KV cache makes flash-attention mandatory on SYCL, and -fa auto will silently switch it on. I briefly believed I had “flash-attn-off” numbers that turned out not to exist. Check what actually resolved.

The surprise

The bigger model holds more context on the same card. The 35B MoE fits 32K and still moves at ~16 tok/s, while the dense 32B flat OOMs at 32K. That’s measured, its memory footprint at depth is far smaller. I’m telling you what the meter showed, not theorizing why.

It does three jobs at once

The 35B MoE lives on the B60, a vision model rides the Ryzen’s iGPU for paper and photos, and a voice model runs on two CPU cores with zero VRAM. On the vision. the honest limit, it does not read values off a chart or a dense number grid. That hit an unimplemented op on my Arc/SYCL stack, so the box hands the chart back to you instead of guessing at it. Nothing shares, nothing waits.

Thermals, measured because I don’t trust vibes

Half an hour flat out: 60 to 62°C, never throttled, on the open-air dock.

Power, honestly

Across the whole box, wall draw runs 55 to 190W idle to load, and it tops out around 145W-ish full tilt. The detailed efficiency picture, what a single answer actually costs, is its own measurement. I’d rather run the wall meter properly and show the method than ballpark it here. That writeup’s coming.

The honest limits

I won’t pretend Intel cards are competitive for gaming, and the software stack is younger than the green team’s, you’ll read more docs getting to the same places, and work harder through issues that are novel. But for the money, 24GB of VRAM from a $650ish GPU that runs a 30B-class brain all day is a hard argument to lose.

Money

If you already own an OCuLink mini-PC, the card, dock, PSU and cable land under a grand. Buying everything new including the host, figure $1,500 to $1,800 in parts at June 2026 prices. One thing I want to be straight about, that number does not include RAM or an SSD. Those ride whatever memory and storage prices are doing the day you buy, and they swing. So, budget them on top rather than getting surprised at checkout. The dedicated AI appliances in this class run $4,000+ and only do the AI half.

For a box meant to run continuously, I cap self-scheduled background work at 80% of idle… that’s a warranty-and-margin call, not a thermal one. Nothing in this build overheats at full load; 80% just keeps it unambiguously inside “normal use” and leaves headroom to catch up.

Owned, understood, serviceable, and nothing leaves the building. That was the whole point.