Ask what local AI costs and the conversation usually jumps straight to the hardware bill. The other side answers with an API price per million tokens. Both numbers are real. Neither tells you what it costs to keep a local box available, or what one more answer adds to the meter.

So we measured ours.

The short version is this:

On our hardware, a typical local answer costs about $0.00003 in marginal electricity at the measured operating point — roughly 0.003¢. Keeping the box powered and ready is the larger number: about $10 a month running 24/7 at our all-in reference rate.

Those are not universal prices. They are numbers for one box, one model, one serving mode, one workload, and one electricity rate. The useful part is the shape of the cost, and the method is simple enough to repeat.

The box and the meter

The measured machine is an Intel Arc Pro B60 attached to a UM890 Pro host through an OCuLink dock. We measured both halves at the wall with two smart plugs: one for the host and one for the card, dock, and dedicated power supply.

The local model for the benchmark point was a Qwen3.6-35B-A3B Q4_K_M served with llama.cpp, single stream. The wall-meter benchmark used a Tapo P115 with stated accuracy of approximately ±2%.

The monthly number came from a different, longer measurement: both plugs ran for about 25.3 days with no recorded off-time.

That distinction matters. A short benchmark tells you about the energy used while answering. A month-long plug reading tells you what the machine costs when it is part of the room, including the time it spends waiting.

What the month-long reading showed

• Measured energy, about 25.3 days - 35.99kWh

• Average draw over that period - 59.2W

• Projected 31-day energy - about 44 kWh

• All-in reference rate used for the conversion - $0.2208/kWh

• Projected monthly electricity cost - about $9.72

• Average cost per hour, running 24/7 - about 1.3¢

The plain-English version is under $10 a month to run 24/7, at this rate and for this measured rig. It works out to about 29¢ a day.

The average draw is close to the measured idle floor because the box is not generating continuously. It spends much of its life powered on and waiting. That is the idle tax, and it is the number a marginal-only calculation leaves out.

What one more answer adds

The marginal answer number comes from the locked wall-meter operating point, not from dividing the monthly bill by an arbitrary number of prompts.

At the balanced chat operating point, the whole box measured about 799 output tokens per watt-hour in the lock-grade run. A served 8K single-stream operating point measured about 671 tokens per watt-hour. The energy methods agreed to better than 1% on the benchmark windows; the absolute meter accuracy remains the relevant ±2% bound.

Using the recorded benchmark conversion, the preliminary marginal estimate is:

about $0.00003 per answer at $0.17/kWh — approximately 0.003¢.

Using the higher all-in rate from the measured household bill, $0.2208/kWh, the same operating-point estimate scales to roughly $0.00004 per answer. We are rounding this as an order-of-magnitude field number, not claiming that every answer has the same length or energy profile.

A short answer, a long answer, a cold uncached prompt, and a decode-heavy chat do not cost the same. The benchmark itself showed three regimes:

• READ: about 297 tok/Wh, cold and uncached, with an 8,008-token input prefilled per answer.

• CHAT: about 799 tok/Wh, the balanced middle case.

• WRITE: about 1,020 tok/Wh, a decode-dominated ceiling.

That is why the honest sentence is “on our hardware, about X,” not “local AI always costs X.”

The part that changes the answer: whether the box is already on

The marginal number is the cost of doing one more job while the machine is already running. It does not pay for the hardware, the dock, the host, or the hours spent waiting.

If the box is already powered for other work, the extra electricity for one answer can be close to the marginal number. If the only reason to keep it running is one occasional question, the idle draw becomes the dominant cost.

That gives local AI two different economic stories:

1. High use: the fixed cost of keeping the box on is spread across more answers. The marginal electricity is small.

2. Low use: the idle tax can matter more than the answers. A box that sits waiting all month is not “free” just because each generated answer uses a fraction of a cent.

Neither story is the whole answer. You need both.

What this does not include

These figures are an electricity floor plus a measured operating-cost view. They do not include:

• the purchase price or depreciation of the hardware;

• maintenance, replacement parts, or downtime;

• the operator’s time;

• networking, storage, or other household infrastructure;

• the quality of the answer compared with a cloud service;

• a universal electricity rate.

The measured monthly figure includes the actual all-in rate used for our internal conversion. Readers should substitute their own rate. The published method should carry the watt-hours and assumptions without exposing a location-specific utility bill.

So, what does local A.I. cost?

On this box, at this operating point:

• One additional answer: about $0.00003–$0.00004 in electricity, or roughly 0.003–0.004¢, depending on the reference rate used.

• Keeping the complete rig powered 24/7: about $10 a month at the measured all-in rate.

• The practical warning: at low usage, idle power matters more than the cost of the individual answer.

That is the number we can stand behind: not a promise for every local-AI build, but a measured answer for ours, with enough method attached that someone else can check the shape against their own wall meter.

The cost is small. The assumptions are not.

The method

• _{Whole rig: UM890 Pro host plus Intel Arc Pro B60, OCuLink dock, and dedicated PSU.}

• _{Monthly telemetry: two smart plugs, approximately 25.3 days, 35.99 kWh total, 59.2 W average.}

• _{Projected monthly conversion: approximately 44 kWh × $0.2208/kWh = approximately $9.72.}

• _{Benchmark: Tapo P115, approximately ±2%; Qwen3.6-35B-A3B Q4_K_M; llama.cpp; single stream; temperature 0; lock-grade run dated 2026-07-15.}

• _{The marginal per-answer figure is preliminary for this article’s framing. It is not a universal tariff or a fully loaded total cost of ownership.}

Why we can’t rely on Recency

A business’s own documents are not a fixed truth. They pile up. The safety procedure written in January gets revised in March and rewritten in June. The old one doesn’t vanish when the new one lands… it’s still in the drawer, still on the server, still perfectly readable. All three are real. All three were correct on the day they were written.

Feed that drawer to an AI and you’ve handed it a problem no amount of better retrieval solves. Everyone’s trying to fix memory by improving the fetch; better RAG, better embeddings, bigger context windows. But when three versions of the same procedure all legitimately match the question, retrieval isn’t the bottleneck. Retrieval does its job perfectly, it hands you all three. Now what?

Why Provenance is necessary

The tempting answer is recency. Take the newest one. It’s clean, it’s cheap, and it’s wrong often enough to hurt. The newest document isn’t automatically the governing one. A later draft isn’t a ratified policy. A memo that mentions the procedure doesn’t replace it. An addendum changes one clause, not the whole document. “Most recent” is a guess that fails where the stakes are highest, the moment the answer actually depends on which version rules.

Lifecycle states are a real step up from a flat pile. Marking a memory Current, Superseded, or Stale beats treating everything as equally true forever. But states only move the question, they don’t answer it. Superseded by what? Current on whose authority? A label with nothing underneath it is just a guess wearing a badge. You’ve renamed the problem, not solved it.

Applying to AI

None of this is new, either. Anyone who’s built a data warehouse has seen it. Bi-temporal modeling and slowly-changing dimensions solved the shape of this decades ago, and event sourcing keeps the history searchable. What’s new is needing it one layer up, in the memory an AI reads from and answers out of. Old discipline, less forgiving place.

The part that actually does the work is provenance. Every piece of knowledge the system holds has to carry where it came from; the source, its standing, how it relates to the others… and there has to be a rule for which source wins when they conflict. Do that, and “current” stops meaning “latest” and starts meaning “authoritative.” That’s the whole difference between a memory system and a confident guess.

Where Provenance isn’t enough

And where provenance runs out, where two documents genuinely conflict and no rule resolves it, the honest move is the same one the rest of the box already makes. Don’t pick the winner and hope. Surface both, cited with provenance attached, and hand the call to the person whose call it actually is. A system that says “these two conflict, and here’s where each came from… you decide” is worth more than one that confidently quotes last year’s policy because it happened to sort first.

There’s an ownership point buried in here too. A memory you can’t trace isn’t really yours… it’s a pile you’re trusting on faith. Knowing where each thing came from, and who says it’s still true, is part of what it means to own your data instead of renting a vendor’s black box that remembers whatever it remembers. It’s how humanity has always ascertained ground truth.

Recency isn’t authority. Provenance is. Everything else is a nicer label on the same guess.

I wanted a local AI box the way I’d want any tool in the shop: owned, understood, serviceable by me. No subscription, no cloud, nothing phoning home. So I built the cheapest one I could that still does real work all day. Not the fastest box… the cheapest one that’s actually viable. Here’s the whole thing, with the real numbers and the parts that fought me.

The shape of it

A Ryzen 9 mini-PC (mine’s a Minisforum UM890 Pro) which stays a normal everyday computer, and an Arc Pro B60 24GB rides alongside it on an open-air eGPU dock with its own 650W Gold PSU, connected over OCuLink. Two machines wearing one job. The mini-PC never gives up its own silicon to the model, and if the dock ever unplugs you’ve still got a first-rate desktop. If your host has no native OCuLink port, you can enumerate the card off an M.2-to-OCuLink adapter, just keep the cable short.

The part that fought me

Getting the card to enumerate cleanly meant sorting a small-BAR issue during bring-up. Once that was handled it comes up clean every boot. The link is Gen4-capable and I’ve watched it train there after a re-enumeration; this boot is sitting at Gen3 x4 under load, and every number in this post reproduces at Gen3. the weights live in VRAM, so the link isn’t your bottleneck either way.

The real lesson: it’s the model, not the card

I came from a dense 32B doing 15.5 tok/s and it was a slog. I swapped to a 35B-A3B MoE and the same card runs it fully offloaded: about 42 tok/s at low context on my daily serving config (q8 KV cache + flash attention), and around 8K depth it holds 29–32 tok/s. (A shallow-KV bench config reads higher, ~45 I’m quoting the as-served number on purpose, because a tok/s figure without its config is half a number.) At 8K the gap gets embarrassing: about 30 versus the dense 32B’s 7.15. better than 4x, because only ~3B parameters are active per token. If you’re budget-building, the MoE is the upgrade; the GPU is just the enabler.

First-hand Arc gotcha while you’re benching: a q8 KV cache makes flash-attention mandatory on SYCL, and -fa auto will silently switch it on. I briefly believed I had “flash-attn-off” numbers that turned out not to exist. Check what actually resolved.

The surprise

The bigger model holds more context on the same card. The 35B MoE fits 32K and still moves at ~16 tok/s, while the dense 32B flat OOMs at 32K. That’s measured, its memory footprint at depth is far smaller. I’m telling you what the meter showed, not theorizing why.

It does three jobs at once

The 35B MoE lives on the B60, a vision model rides the Ryzen’s iGPU for paper and photos, and a voice model runs on two CPU cores with zero VRAM. On the vision. the honest limit, it does not read values off a chart or a dense number grid. That hit an unimplemented op on my Arc/SYCL stack, so the box hands the chart back to you instead of guessing at it. Nothing shares, nothing waits.

Thermals, measured because I don’t trust vibes

Half an hour flat out: 60 to 62°C, never throttled, on the open-air dock.

Power, honestly

Across the whole box, wall draw runs 55 to 190W idle to load, with a loose ballpark around 145W when it’s working. That’s the range, not a budget, the detailed efficiency picture. What a single answer actually costs, is its own measurement. I’d rather run the wall meter properly and show the method than ballpark it here. That writeup’s coming.

The honest limits

I won’t pretend Intel cards are competitive for gaming, and the software stack is younger than the green team’s, you’ll read more docs getting to the same places, and work harder through issues that are novel. But for the money, 24GB of VRAM from a $650ish GPU that runs a 30B-class brain all day is a hard argument to lose.

Money

If you already own an OCuLink mini-PC, the card, dock, PSU and cable land under a grand. Buying everything new including the host, figure $1,500 to $1,800 in parts at June 2026 prices. One thing I want to be straight about, that number does not include RAM or an SSD. Those ride whatever memory and storage prices are doing the day you buy, and they swing. So, budget them on top rather than getting surprised at checkout. The dedicated AI appliances in this class run $4,000+ and only do the AI half.

For a box meant to run continuously, I cap self-scheduled background work at 80% of idle… that’s a warranty-and-margin call, not a thermal one. Nothing in this build overheats at full load; 80% just keeps it unambiguously inside “normal use” and leaves headroom to catch up.

Owned, understood, serviceable, and nothing leaves the building. That was the whole point.

For weeks, our benchmark told us that promise was holding. One column in particular synthesis questions, where the answer has to be assembled from a couple of figures in the source sat green. Right numbers, right citations, every time.

The column was lying to us. We just weren’t asking it the right question.

What we were actually measuring

Our scorer, on those synthesis questions, checked two things: did the answer surface the correct figures, and did it bind them to the correct citation. Both were true. So, it scored PASS.

What it never did was read the conclusion.

Here’s the failure that hid behind the green, on a filing-threshold question — the kind a real owner actually asks. The model retrieved the right threshold. It cited the right page. In its own words, it did the arithmetic correctly: “since $500 is above the $400 threshold, they are generally required to file.” And then, one line later, it wrote the answer as: “(a) No.”

It reasoned its way to the right answer and then wrote down the wrong one. Because the figure and the citation were both present and both correct, our benchmark called it a clean pass.

The number, honestly

We built a separate axis that reads the stated conclusion against what the cited facts actually entail and measured it. On that one inference question, across depths and positions, the model’s stated yes/no answer was correct 44% of the time (Wilson 95%: 34.6–54.7%). A large chunk of the rest weren’t random errors — they were answers that reasoned correctly in prose and then labeled the conclusion backwards.

The honest limits: this is one question type so far, one corpus, one model. It is not “the model is wrong 44% of the time.” It is “on questions that need a step of judgment, the stated conclusion decouples from the reasoning, and we’ve now measured it once.” We’re expanding it to more inference types before we’d quote a general rate. But one clean measurement of this was enough to change how we think about the whole product.

Why this is worse than a plain error

A model that says “I don’t know” costs you nothing. A model that hallucinates an obvious nonsense answer, you catch. But a model that lays out the correct reasoning, cites the correct page, and then states the wrong conclusion has done the most dangerous thing an assistant can do; it has spent the trust its visible work just earned. The citation is what makes you stop checking. The reasoning is what makes you believe. And then the answer is wrong.

For a tool whose entire pitch is “cited so you can verify,” that is the one unforgivable failure. It punishes exactly the person who trusted it correctly.

The lesson, if you’re evaluating RAG

Check whether your scorer reads the conclusion or just the retrieval. “Grounded” and “correct” are different axes, and a system can be flawless on the first while failing the second. If your eval only measures whether the right chunk was retrieved and cited, you can ship a green dashboard on top of a product that reasons right and answers wrong — and you won’t find out from your metrics. You’ll find out from a customer who trusted a cited number.

We only caught it because we started scoring the model against its own stated reasoning. The tell was cheap once we looked: reasoning says one thing, label says the opposite, in the same answer.

What we did about it

We didn’t try to make the model conclude correctly and call it fixed. We changed what the product is allowed to do.

On a question that requires a judgment call, for example “do I have to file?” as opposed to “what is the filing threshold?” the box doesn’t render the verdict. It surfaces the retrieved facts and the governing rule, cited to the page, and hands the call to the person whose call it actually is… the owner or responsible employee. It files, it cites, it assembles the whole context on demand and then it says, in so many words, “this one’s a judgment call, and it’s yours.”

That’s not a limitation we’re apologizing for. It’s the design. The people we build for are good at making calls on incomplete information, that’s their job. What they don’t have time for is assembling the context. So, we do the half a machine is good at; hold everything, cite exactly, never tire… and we hand off cleanly at the seam: the exact point where a human’s judgment belongs and a model’s confidence is a liability.

The benchmark being green taught us less than the benchmark being caught. We’ll keep publishing both.

We’re not a frontier lab. We’re not training the next giant model or chasing a benchmark record ; the frontier builds the engine, and that’s a different trade than ours. Ours is the unglamorous work in the middle: getting these systems across the gap from a demo that impresses researchers to something a regular business can actually run… safely, in the hands of someone who isn’t an AI expert and shouldn’t have to be.

That gap is where almost everything fails. A model can reason like a PhD and still get wired into a workplace in a way that quietly burns the people trusting it. Closing it and making powerful, unpredictable systems safe for regular people doing regular work is a real trade. It’s the one we practice. It’s the one the human behind this page paid the bills at his day job, handling voltages with consequences to a human body that can be just as fatal as a bad A.I. may be to an entire organization.

The product is a box that sits on your site, reads your own documents, and answers cited to the page and then hands the judgment calls back to you, because those are yours to make. Nothing leaves the building. No call ever made without a Human-in-the-Loop.

This is where we show the work. Not a sales pitch, not a benchmark nobody can reproduce… the actual tests, the methods, the numbers with honest error bounds, and the failures, especially the failures. When we catch our own tools lying to us, that goes here too, with the number attached.

If you build or deploy local AI or you’re an operator trying to decide whether any of this is trustworthy enough to put near your business… Well, this is for you. If you’re here for “AGI next quarter,” you’ll be bored. If you want passed the hype, and to start looking at how this technology is being applied on-site, and how it is likely to affect YOUR everyday workplace, you are in the right place.

We leave no doubt. Including about where we were wrong. Follow along.

— ThirdShift R&D