What Consumer Hardware Actually Does

Twenty-nine local-model labs, May to August 2026, across three consumer GPUs: an RTX 4070 Ti, an RTX 5070, and a pair of Intel Arc Pro B70s. Every number below is measured, dated, and attributed to the lab that produced it.

Sixty-four gigabytes of VRAM for near or less than the price of one flagship card is a trade almost nobody writes up. Here is what it actually returns, where it stops, and the one constraint that turns out to matter more than any of it.

Six things this hardware buys you

Each tile is a claim with a measured number behind it. Follow it to the lab that produced it.

All figures Q4-class quantization except the 120B (MXFP4) and the Mixtral (Q2_K). The two eras are not directly comparable — see the note below. Full tables in each section.

Two rows are worth pausing on. The Mixtral is a 140-billion-parameter sparse model, 52 GB on disk, running on a workstation with a 12 GB card — 128 GB of system memory does the carrying while the GPU contributes a slice, roughly 81% CPU to 19% GPU. Four tokens a second is useless for chat and perfectly adequate for an overnight reviewer that reads a diff while you sleep. And the 70B only appears on this chart at all because of a single configuration flag; without it, that row reads 21.5 tok/s of prompt processing instead of 151.4.

How the rig got here

None of this was bought as a lab. It accreted, one constraint at a time, and the sequence explains most of the numbers below — including the one that keeps breaking.

Step 2 is the whole story of this hardware in one line: two large GPUs bolted onto a machine whose system memory was specified for a single one. That mismatch is not a footnote — it is the binding constraint in three separate labs, and it is why the most useful finding here is about DDR4 rather than VRAM. Step 5 is the fix, and it hasn't happened yet.

Six findings that survived every lab

1. System RAM is the binding constraint, not VRAM

This is the one that costs newcomers the most time. The B70 box has 64 GB of VRAM and 32 GB of DDR4, and the 32 GB is what breaks — repeatedly, in three independent labs. A stack sweep found the box sitting at 24 GiB used with one 30B model resident and swap 100% saturated. A judging run OOM-killed until the critic model's context came down from 32k to 8k, which freed about 7 GiB. And on 30 July three large models enabled at once produced 81 crash-loop restarts and roughly 18 GiB of memory that went missing entirely — trapped by killed driver allocations, and only recoverable by rebooting.

If you are speccing a local-inference box: buy the system RAM. The VRAM number on the box is not the number that will stop you.

2. MoE is the architecture that fits consumer hardware

At equal size — roughly 32B total parameters, Q4, one card — a 3B-active mixture-of-experts decoded 5.7× faster than a dense 32B (130 vs 23 tok/s) and prefilled 2.9× faster. Decode streams only the active parameters, so sparsity buys back exactly the memory bandwidth consumer cards don't have. Every practical result in this whole corpus rests on that.

3. Backend and kernel choice swing results more than the GPU does

Same model, same card, same prompt: CUDA processed the prompt at 2276 tok/s and Vulkan at 68.5. A 33× spread on identical silicon. And on Vulkan / Arc, flash attention turns out to be a pessimization — about 4× slower than the default attention path at depth. Since quantized KV cache forces flash attention on, "save memory with q8 KV" is a trap on that stack: it measured 2–4× slower, not faster. That one began as a wrong prediction and had to be corrected against the data — which is how it became a finding rather than an assumption.

The most extreme case is a single flag. Running a 70B across both B70s, turning memory-mapping off took prompt processing from 21.5 to 151.4 tok/s — a 7× win, and the difference between "unusable" and "usable". Nothing about the hardware changed. If you are benchmarking a local stack and the number looks absurd, suspect the configuration before the card.

4. Context depth is the real cliff, not model size

A 30B MoE that fits comfortably on one B70 decodes at 86.9 tok/s empty and 5.3 tok/s at 64k context. On the Windows/Vulkan side the same shape showed 132.8 tok/s empty falling to 11.5 at 64k — roughly linear to 32k, then a superlinear cliff. Meanwhile prompt ingest dominates the wall clock: a 57,000-token request spent 207 of its 213 seconds just reading the prompt.

Which means the thing worth engineering is not placement. It's caching. An identical 8k prompt sent twice went from 11.8 s to 0.17 s — 68× — on cache hit. And restoring a saved KV cache beat re-computing it by 29× on real bytes, with the advantage growing with context: 22× at 1k, 75× at 14k.

5. Local output held up against frontier cloud

On a scored documentation and decision-record task, the local 120B model scored 93.8 against Gemini 3.1 Pro's 90.6 and 3.5 Flash's 89.2. The cloud's measurable advantage was latency — about 4–5× faster — and that was the entire advantage. One task family, fifteen cells — not a claim that local wins everywhere. The narrower claim: "local is the cheap option you settle for" did not survive contact with a rubric.

6. And you can train on it, not just serve

The one most likely to be assumed impossible. A LoRA fine-tune of a 7B coder model, running on one Arc Pro B70 on Windows via PyTorch's XPU backend, took held-out loss from 2.96 to 1.13 in 108 seconds of training — then merged, quantized, and served on the same card in about six seconds, generating correct code. Full detail in section 4. Local inference is well-trodden ground by now. Local weight-changing, on a non-NVIDIA consumer card, is not.

All sections

1 · NVIDIA consumer — RTX 5070 and 4070 Ti, 12 GB each

Backend lab — CUDA vs Vulkan vs the integrated GPU

The same model and prompt driven through four isolated servers on separate ports, to find out what the backend choice alone is worth.

• The integrated GPU really can run models — it loaded 49.6 GiB of the Mixtral into Intel shared memory and ran at 0.83 tok/s. A capability proof, not a lane: it left under 5 GB of free system RAM.
• "Both GPUs visible" is not a useful split. Mixed Vulkan enumerated both devices, then placed the small model entirely on the RTX and the large one on RTX plus CPU-mapped RAM. The Intel part was never materially used.
• Running a second model concurrently is survivable but expensive: the 14B fell from ~61.9 to ~21.5 tok/s, and a 30B coder model fell to 4.12 tok/s.

Source: Ollama backend lab — results summary, raw CSV, per-profile JSON and runner logs. 2 July 2026.

The 52 GB sparse model, off system RAM

Cold load 39.6 s, warm load 0.03 s — residency is the whole cost. The machine stayed responsive throughout and never obviously paged.

Source: local MoE findings note. 2 July 2026.

The capability matrix and the baseline protocol

Two more documents from the same day turn those numbers into decisions rather than trivia. A capability matrix assigns each model class an operational role — resident controller, interactive specialist, batch specialist, fallback worker, overnight critic — and tags every cell measured, published, or inferred, so you can see which claims came off this rig and which came off a model card. Its operating conclusion: a 12 GB GPU strongly favours a control-plane and specialist split, and the constraint is residency, not active parameters. A 30B MoE behaves like a small model per token, but its total weights still drive placement.

The other is a dated baseline snapshot plus a written rerun protocol, built deliberately so a future Arc Pro B70 rerun would be comparable. Warm load for both test models collapses to about 0.10 s. Worth noting for anyone building on Ollama: same-service "concurrency" there behaved as queueing against one daemon, not true batched serving.

Sources: hardware capability matrix; RTX 5070 baseline snapshot and baseline protocol. 2 July 2026.

What the raw logs said that the summaries didn't

Going back through the raw per-call CSVs — which log GPU utilization, memory and power draw before and after every request — turned up two things no summary of mine had stated.

• The 30B draws a third of the power the 14B does — at the same token rate. 66.7 W against 203.4 W, for 57.4 vs 61.8 tok/s. A 12 GB card running a 30B mixture-of-experts is memory-bandwidth-bound and barely touching its compute; the dense 14B saturates it. That's a performance-per-watt argument for sparse models, and it matters if the box runs all day.
• Warm prompt throughput is 7× the cold headline number — 7154 vs 1015 tok/s. Quote the cold figure and the steady state is badly understated.

Source: raw capability-bench and cross-service CSVs, RTX 5070. 2 July 2026.

The model matrix — on the 4070 Ti

Worth separating, because it's a different question and a different card. Where the 5070 work is two models deep and many scenarios wide, this one is model-wide: a planner-and-critic loop run across different model pairings to find where it breaks. It broke in three distinct places, each run isolating one.

• An 8B planner with a 3B critic produced a false approval. The critic signed off on a plan in which five of six file paths did not exist. Cause: the planner had been given only a README excerpt and a top-level directory listing, so it invented a layout — and the critic had no repository context at all, so it couldn't catch the invention. Fix: give both roles the actual tracked-file tree.
• After the fix, paths were correct and the loop stalled anyway — on critic noise. The 3B critic engaged but degraded to bare noun-phrases by round three and missed concrete bugs. The grounding fix was model-agnostic; the critic's capacity was not.
• A 20B critic was transformative and still didn't converge. Seven specific, grounded issues in round one — real compile errors, undefined symbols. Round two nearly converged; round three regressed, with the planner emitting a corrupted path. Still an open question: was that the 8B planner's weakness, or VRAM contention from co-running a 13.6 GB critic alongside it on a 12 GB card?

That question is unresolved — and it is the single best argument for the dual-B70 box that follows: 12 GB forces a choice between a good planner and a good critic, and the choice is what breaks.

Source: local-inference research series and rehearsal run log, RTX 4070 Ti on Ollama/CUDA, from 5 June 2026.

2 · Intel Arc Pro B70 — the Windows / Vulkan era

b70tools — is Task Manager telling you the truth?

It wasn't. That's why the tool exists. It reconciles four independent telemetry sources — kernel-mode adapter counters, per-process heap budgets, the Vulkan memory-budget extension, and Intel's own power/thermal API — and flags where they disagree in real time.

Look at the middle two rows: the 30B MoE generates four times faster than the 32B dense, while the dense model prompts eight times faster. Different bottlenecks, same box.

The other finding was about the cards themselves. They're physically identical and they are not identical in software: the top-slot card runs 10–15 °C hotter under identical load and its telemetry reports physically impossible values — 5.117 V and 8.55 GHz at idle. That's a driver bug, not a hardware one. It took several experiments to conclude the cards deliver identical inference throughput, which is the whole argument for building the observation tool before trusting the dashboard.

Source: github.com/djcdevelopment/b70tools — Windows-native telemetry workbench, May 2026. Observation cost: 16.6 MiB RSS, under 130 ms init.

The overnight campaign — 14B, 32B, and a 70B across both cards

A phased ladder run overnight: each card alone on a 14B, then a 32B, then Llama-3.3-70B tensor-split across both, plus a context curve and a KV-precision comparison. About a hundred files of raw output. This is the run the rest of the Windows era was built on.

• The mmap flag is the whole story on the 70B. 21.5 to 151.4 tok/s of prompt processing — 7× — from turning memory-mapping off. With it on, a 70B on this box is a curiosity. With it off, it's a tool. Eleven tokens a second of generation is slow, but it is a 70-billion-parameter model running on two cards that together cost less than one flagship.
• Prompt throughput peaks mid-batch, not at maximum. On the 14B: 903 tok/s at a 128-token prompt, 1254 at 512, then back down to 1160 at 2048. Generation stayed flat at ~45 regardless.
• The quantized-KV run produced nothing at all. It ran for 221 seconds and emitted only a loader banner; the deep-context variants are error dumps. Logged at the time as a failed run. A month later the flash-attention finding explained it — the trap was already in the data, unread.
• Where the fp16 curve did complete on the 70B: 186 tok/s of prompt processing at both 1k and 4k, falling to 121 at 16k.

Source: overnight bench campaign, dual Arc Pro B70 on Windows Vulkan, 24 May 2026. Roughly 100 raw JSONL and log files, retained.

denning — treating the KV cache as an OS-managed resource

The deepest lab in the set, and the one to hand a skeptic. The premise: on a box with no GPU fabric, less system RAM than VRAM, and an operating system arbitrating video memory, correct model-state management is bandwidth-roofline admission control — and you have to coexist with an OS memory manager that will act against you. Named for Peter J. Denning's working-set model. Predictions were committed to git before the data that confirmed them.

• The demotion cliff. Once another app pushes the GPU past its memory budget, the OS involuntarily evicts a model that fits — decode collapses 5× with about 5 GB spilled across PCIe. The live budget drops from 31 GB to 15 GB under a co-tenant. Reproduced twice. You cannot pin against it: the residency-priority API is an intra-process hint only.
• Decode roofline for the 30B MoE: 132.8 tok/s empty, 74.3 at 8k, 58.2 at 16k, 38.0 at 32k, 11.5 at 64k.
• Restore beats re-prefill by 29× on real KV bytes (84 ms vs 2422 ms), and the gap widens with context — 75× at 14k, where re-prefill costs 24.7 s and restore costs 0.33 s.
• The admission knee: goodput under a latency target peaks at 8 concurrent sessions (229 tok/s, all targets met) and collapses to zero by 10. Over-admission is worse on every axis.
• Card-to-card transfer is 6.48 GB/s — about 0.47× host-to-VRAM speed, because it is host-bounced. There is no peer-to-peer on this platform. Hold that number; it comes back.

Source: github.com/djcdevelopment/denning — on-rig validation 19–21 June 2026, dual Arc Pro B70, Windows / Vulkan.

The single-card battery — PCIe width turns out not to matter

After a thermal pull left the box on one card, a six-part battery ran at full PCIe ×16: prefill, model load, session rotation, restore-versus-re-prefill, an engine probe, and a 70B deliberately overcommitted. Predictions were locked to a git tag before the data was collected.

• PCIe link width was irrelevant to everything measured. That's the useful negative result: if you're agonizing over slot bifurcation, or whether your board drops a card to ×8, this says stop. The binding constraints are compute (the decode cliff), host RAM and commit (the memory inversion), and your storage tier — not the link.
• The restore-versus-re-prefill advantage held between 133× and 475× in this run.
• The overcommitted 70B spilled 7.5 GiB into shared system RAM — the OS demoting a model that no longer fits, confirmed again.

Source: pre-registered single-card ×16 battery, one Arc Pro B70 on Windows, 21 June 2026.

battlemage — where it started

The earliest of the three: single- and dual-card Windows 10 benchmarking, including the dual-card 70B-class run on Vulkan that first proved 64 GB of shared inference was real on these cards. Superseded by the two repos above for anything load-bearing, and worth knowing it exists.

Source: github.com/djcdevelopment/battlemage — from May 2026.

3 · Intel Arc Pro B70 — the Linux / SYCL era

Does splitting a model across two cards help? (29 June 2026)

The first Linux-native benchmark asked two things: does multi-card work on Ubuntu the way it didn't on Windows, and is splitting actually better than using one card?

Multi-card layer split works, and buys nothing for a single stream. Parity through 32k, slightly worse at 64k. The second card is capacity, not speed. The more aggressive split modes weren't even candidates: row-split segfaulted on both variants, and tensor-split timed out and then segfaulted.

Source: SYCL context-ladder lab, dual Arc Pro B70 on Ubuntu, 29 June 2026.

The same question through a real server

Benchmarks measure the model. This measured what a client experiences — a resident server at 128k context behind an OpenAI-compatible facade, timing readiness, time-to-first-token, streaming and cache reuse.

Then the concurrency ladder, at about 14,300 prompt tokens: 1 client 22.9 s, 2 clients 45.3 s, 4 clients 91.0 s, 8 clients 181.5 s. Almost perfectly linear — which is the bad news. Time-to-first-token is nearly identical to full completion time at 1 through 4 clients, meaning streaming stops being useful the moment concurrent prompt ingest dominates. The conclusion there is architectural, not hardware: what needs work is request scheduling, prompt-cache strategy and admission control.

Source: service-path placement matrix, depth ladder and concurrency ladder, 29 June 2026.

What the two cards can actually say to each other (18 July 2026)

Remember the 6.48 GB/s host-bounced number from Windows. On Linux, with the modern driver and Level Zero stack, the peer-transfer benchmark was built from source and run — 50 iterations, 8 bytes to 256 MB.

14.29 GB/s is the practical ceiling of PCIe 4.0 ×8. Going through the root complex costs nothing. That is the same traffic that was host-bounced at 6.48 GB/s on Windows — and it is the single cleanest measurement of what the Linux migration was worth on this hardware. Transfers are perfectly symmetric in both directions, and the dedicated copy engine matches the compute engine exactly, so bulk weight and cache movement can ride the blitter without stealing compute.

Source: peer-transfer microbenchmark, dual Arc Pro B70 on Ubuntu, 18 July 2026.

A 63 GB model, resident, on two consumer cards (18 July 2026)

The question was whether a 120-billion-parameter MoE — 63.4 GB in a single file — can be an always-warm service on hardware like this.

It fits only because about four layers of experts ride host memory rather than VRAM — that flag is required, not tuning, since 63.4 GB of weights exceed roughly 60 GiB of usable VRAM. Sparse architecture plus a little CPU offload is the whole trick.

Then the attempt to break it

The brief was blunt: find out if it breaks before we build things with it. Six sweeps — concurrency ladder, context depth, decode-heavy, sustained soak, overload burst, and memory pressure. 453 requests, 453 successes, zero restarts.

• Goodput saturates at ~26 tok/s at four concurrent and holds flat to sixteen. Oversubscription costs queue latency, never throughput. The queue is stable and fair.
• 32 concurrent requests against 4 slots: all completed, zero errors. A textbook linear queue drain, 105 s max wall.
• Memory pressure barely registered. A hog process stepped up to 18 GiB — about 48 GB of demand against 30 GB physical — and decode lost 4% or less. Hot expert pages won the page-cache fight and compressed swap absorbed the rest. Caveat stated in the lab: the hog's pages compress near-perfectly, so that's a best case.
• Thermals are a non-issue: 78 °C peak GPU die at full sustained load, against a proven 92 °C envelope for image generation on the same box.

Source: stress characterization of the resident 120B model, 18 July 2026. Instruments and raw per-request data retained so the campaign is rerunnable after any config change.

The standoff, 30 July 2026

Not a planned benchmark — an incident, and the most instructive record here for anyone about to run several local models at once. Three large models were all enabled as boot-persistent services on one box. They starved each other.

• 81 crash-loop restarts of the 120B service, each failing to allocate 26 GiB, because a 30B model was holding 17.3 GiB and left only 13.7 GiB free.
• Host RAM peaked at 14.7 GiB before an out-of-memory kill, and roughly 18 GiB went phantom — unaccounted memory trapped by 81 killed driver allocations. A reboot was the only thing that cleared it.
• An open port is not a ready model. Health checks that only probed the socket reported success while the model was still loading, or already dying.
• A 120B model leaves roughly 4–5 GiB per card. There is no room for a second tenant. Residency on consumer cards is exclusive and has to be scheduled that way.

Fixed by making the services mutually exclusive at the init-system level, adding start-rate limits and alerting. The eventual topology decision cut the other two models entirely: one model serves every role.

Source: incident trace, dual Arc Pro B70 on Ubuntu, 30 July 2026.

4 · Training on it — LoRA fine-tuning on an Intel card

Everything above is about serving models. llama.cpp serves a model; it cannot change one. This is the other half — and it is widely assumed to be unavailable on this hardware.

It got found by accident. A probe meant to assess a different inference engine refuted its own premise: the environment already had PyTorch's XPU build working on native Windows. Not WSL, not a container — native. torch.xpu saw the B70, and matrix multiply ran at 150 TFLOPS fp16 and 158 TFLOPS bf16, with the matrix engines engaged and bf16 available — a numeric format the Vulkan serving path doesn't have. The engine under test was blocked on Windows by packaging, not by any absence of hardware support. That reframed the whole question and opened the door to training.

The memory ceiling, mapped before trusting it

Before any real run, peak VRAM was swept against sequence length, batch size and gradient checkpointing — to find the envelope deliberately rather than discover it by crashing.

Gradient checkpointing isn't a tuning knob here, it's a requirement. Without it the 7B runs out of memory by a 2048-token sequence. With it, 4096 tokens at batch size 1 fits in 27.3 GiB with room to spare. Past that you need 4-bit quantized training. Note also that the over-budget runs didn't fail cleanly — they climbed past 32 GiB by spilling into shared system RAM first, the same OS demotion behaviour that haunts the serving side.

The actual fine-tune, and closing the loop

Then a real run inside that envelope: 40.4 million trainable parameters — 0.53% of the model — held-out loss 2.96 to 1.13, training loss 2.33 to 0.91, peak 17.0 GiB, sixty optimizer updates in 108 seconds, producing a 157 MB adapter.

Then the part that makes it a loop rather than a demo. Merge the adapter into the base weights, convert to the serving format (14.2 GB at half precision), quantize to 4-bit (4.36 GB), and hand it to the ordinary Vulkan serving path. It loaded in about six seconds and generated a correct Fibonacci function.

Two honest limits. This is LoRA, not full fine-tuning — half a percent of the parameters, not all of them. And a 7B is a small model; 14B and up, or longer sequences, need 4-bit training that has not been run. What is proven is that the path exists end to end and the envelope is mapped.

Source: on-rig fine-tuning project, phases 0–4, one Arc Pro B70 on Windows with PyTorch XPU, 22 June 2026.

The serving substrate underneath

Worth a line because it is the connective tissue: partway through, Ollama came off that box in favour of something leaner — its memory overhead was the motivation. A Windows-native Vulkan dual-card substrate that owns config, the exact launch recipe, process lifecycle and structured status, behind a minimal OpenAI-compatible surface. It ran a 30B mixture-of-experts dual-split at 128k context alongside a 14B critic, from June 2026, and it's the direct ancestor of the Linux serving work in section 3.

Its most-earned design decision is one line: readiness means "can serve", not "process is up." Ignoring it produced a production incident on the Linux side five weeks later, when health checks that only probed the socket cheerfully reported success while a model was still loading — or already dying.

Source: dual-B70 Windows Vulkan serving substrate, June 2026. A dozen dated run directories with per-run manifests and telemetry, largely unmined.

5 · Does the output hold up?

Throughput is the easy half. These labs asked whether local models produce good enough output — and ran into a harder problem, which is that when a language model grades the work, you may be measuring the ruler.

The matrix wind tunnel — does self-refinement help?

Six rounds of overnight experiments on idle hardware, planner and critic deliberately running on different physical boxes: the 30B MoE on the dual-B70 machine, a 30B coder model on the RTX 5070. Each cell was a combination of planner, critic, prompt archetype, refinement laps and ordering, scored 0–100 by a held-out judge panel. 192 cells in the largest round.

Read those two columns side by side. Round 4 tells a dramatic story — two laps is the sweet spot, four laps collapses. Round 6 says lap count barely matters, and the collapse was largely an artifact of the first judge's bias toward brevity. The same judge inflated the one real effect in the data — a concise author prompt, "shortest complete answer, lead with the decision" — from +2.8 points to +8.0.

The follow-on work quantified it: judges are near-deterministic on repeat (0.40 point sampling deviation) and wildly variable across rubrics (6.5 points average disagreement, up to 47). So resolving a genuine ~3-point effect takes roughly 19 lens-diverse votes but only about 1 repeat vote. Spend the evaluation budget on different rubrics, not on asking the same judge twice.

The value here wasn't the answer about laps — that's genuinely "meh". It was that the instrument flagged an effect, corrected it, explained it, and then caught its own measurement bias. Entirely on idle local hardware, overnight, at no marginal cost.

Source: github.com/djcdevelopment/windtunnel — dual Arc Pro B70 plus RTX 5070, 5–10 July 2026.

6 · Local vs cloud — quality, latency, money

The head-to-head (21 July 2026)

A scored documentation and decision-record task, run identically across the local model and two frontier cloud models. Fifteen cells, all successful.

The local model won on quality, on two consumer GPUs, at zero marginal cost. The cloud won on latency by 4–5×, and that was the whole of its advantage on this task.

What cloud actually costs (23 July 2026)

A billing audit after standing up a cloud agent that reaches back into the local GPUs across an audited network boundary. The finding was not about tokens. Idle standing infrastructure produced an unexpected $36–38 bill — two managed agent runtimes at roughly $3.50/day each plus a small VM — while actual inference spend was trivial. If you are comparing local against cloud, compare the standing cost, not the per-token rate.

The ledger

Not a lab — the production record. Every model call routed through the local gateway is logged with backend, model, token counts and outcome, so the economics are measured rather than argued.

618 of those calls landed on already-owned hardware. Every backend listed ran at a 100% success rate.

7 · What is not measured

Stated plainly so nobody cites this for something it doesn't cover.

• A dense 32B on the RTX 5070. Named as an open cell in the capability matrix, never run. The 12 GB verdict for 32B dense is inferred, not measured.
• The 30B MoE on the NVIDIA card. Measured extensively on the B70s, never on the 5070. There is no direct same-model cross-vendor comparison in this corpus.
• Windows against Linux on identical workloads. The two eras used different harnesses. The peer-transfer contrast — 6.48 vs 14.29 GB/s — is the one clean cross-platform number. The throughput figures are not directly comparable.
• Two-card symmetric scaling on Windows. Retracted, not resolved. The honest number waits on an asymmetric, watchdog-monitored re-run.
• The gaming-degradation curve for running worker loads on a gaming rig. Protocol written, human-gated by design — an agent can't play the game.
• No formal output-quality rubric on the NVIDIA lane. Those labs measured throughput and residency only; all the quality scoring ran through the B70 box or the cloud.
• The NPU. Present in the workstation, used by nothing in any stack tested here.
• Full fine-tuning, and anything above 7B. The training result is LoRA — half a percent of the parameters. 14B-and-up, or sequences past 4096, need 4-bit quantized training not yet run.
• Whether the two cards are actually equal. Two runs four days apart disagree by 18% on an identical model, and nothing reconciles them.
• A dozen serving-substrate run directories with per-run manifests and telemetry, across three model families at up to 128k context, sitting unread on a drive.

If you're deciding whether to try this

The honest summary is that a 30B-class mixture-of-experts model runs comfortably fast on a single consumer card — 80 to 87 tokens a second — and that a 120B model is genuinely serviceable on two of them, at 26 to 29. A 70B works too, once you find the flag. The output quality is competitive with frontier cloud on real work, and you can fine-tune on the same hardware you serve from. What will actually stop you is none of those things: it's system RAM, context depth, and the fact that a local inference server will silently truncate your prompt and tell you everything went fine.

Buy the RAM. Cache your prefixes. Check your own payload sizes. And build the observation tool before you trust the dashboard — because on at least one of these platforms, the dashboard was reporting 5.117 volts and 8.55 gigahertz at idle, and cheerfully meant it.