Moonshot open-weighted 2.8 trillion parameters. I priced out every way I could find to run it. Here's where it lands.
Prices verified 28 July 2026. GPU rental moves weekly, so every price in the model is yours to override. Updated 29 July: a measured throughput number landed and it corrects this piece downward — see §02. You can now also price against Claude and GPT instead of Moonshot.
My first thought was the same as yours: 2.8 trillion parameters, open weights, I can run that. So I built a model to price it. If you want K3 for sovereignty or because you're going to fine-tune it, everything below is for you. If you want it because you assume it's cheaper than paying Moonshot, I'd read on before you book a node.
It comes down to cached input. Moonshot charges $0.30 per million tokens on a prompt cache hit and reports around 90% hit rates on coding traffic. I could not find a rental configuration that beats that, and I went looking.
Before you take that as the whole comparison: for a lot of buyers Moonshot's API was never on the menu. The question isn't really "K3 on my hardware against K3 on Moonshot's" — it's "K3 on my hardware against the Anthropic and OpenAI bill I'm already paying." People are markedly more comfortable running a Chinese model on their own silicon than routing their traffic through a Chinese company's private API, wherever it happens to be hosted. That's apples to oranges on capability, and it's still the trade most people are actually weighing.
So the calculator lets you pick which price you're measuring against. Two things fall out. Claude Sonnet 5 lists at $3.00 in, $0.30 cached, $15.00 out[32] — identical to Moonshot's, to the cent — so the cached-input wall is the same wall, and you can buy it from a US vendor. Move up to Claude Opus 5 or GPT-5.6 Sol[33] and cached input is $0.50 with output at $25 to $30, which drops every bar for self-hosting by a third or more.
Output tokens go the other way. At $15 per million you clear the API's price at about 1,100 tokens a second, which is easy. Against Opus 5's $25 it's 660, and against GPT-5.6's $30 it's 550.
So: saturated and output-heavy, self-host. Prompt-heavy with caching, don't. Interactive coding agents — the case most people are actually asking about — don't either, and §02 is where that turned out to be true.
And there's a floor under all of it. One 8× B300 node, because the weights are 1.4 TB and natively four-bit, so there's no smaller configuration and no quantizing your way down to one.
Seven situations cover most of it.
| Your situation | What to do |
|---|---|
| Output-heavy, saturated batch | Self-host. You beat frontier output prices by 7× against Sonnet 5, 15× against GPT-5.6. |
| Interactive coding agents | Use the API. The one measured run lands near the unbatched worst case. Read §02. |
| Prompt-heavy with cache hits | Use the API. At $0.30 cached you will not win this one. |
| Bursty, under 200 hr/mo | Rent where local NVMe is included. Not RunPod. |
| 20–50% duty cycle | Reserve. Nebius or Lambda, multi-month. |
| Over 4,300 tok/s sustained, six months running | Buy. 2,600 if what you're displacing is Opus 5. Line up colo before the box ships. |
| Needs to work this week | 16× H200 on Instant Clusters. Read §05 first. |
It all hangs on one number: how busy you keep the box. An unbatched K3 endpoint runs about $148 per million tokens, because vLLM measures 111 tok/s at batch 1 and a $59/hr node divided by 0.4M tokens an hour is not a business. Saturate it at the throughput vLLM implies and you land near $2. So every argument for self-hosting is really an argument about utilization, and I think most of the people making it right now haven't checked theirs. Mine included — §02 has a measured number that is nowhere near saturation.
Everything in this section is quoted at the calculator's opening position — a coding agent, 8% output, 85% cache hits — so you can reproduce it before you move it.
Against Moonshot's own API, which blends to $1.85/M at that mix, renting an 8× B300 node needs roughly 8,900 tokens a second sustained to break even, and buying one at $470k needs 4,300 tokens a second, around the clock, for three years. Swap the reference price for Claude Opus 5 at $3.08/M blended and those become 5,300 and 2,600. For GPT-5.6 Sol, 4,700 and 2,300. Go look up your own volume before you get attached to the idea. That's the number that ends most of these conversations.
Owning is the only configuration that wins outright, and only if you fill it — and that part doesn't move when you change the API you're comparing against, because it's hardware against hardware. Against renting on-demand you need 30–48% duty before the capex pays back, depending on the node; against a dedicated endpoint, 18–30%. Land in between and you should reserve. Nobody likes that answer. I didn't either.
And if you're picturing one in a rack at the office: a B300 node draws 14.5 kW continuously and needs four tons of cooling. It's going in a colo.
Four ways to put K3 into production. Set your workload, hardware and facility, and the four routes reprice together against the API. Defaults are July 2026 list prices.
The bars above spread the hardware over its life, which is the accountant's view of it. This is the money actually leaving the account, and the month the cumulative lines finally cross.
Where the lines cross is where your decision changes. The dashed rule is whichever API you picked, priced at your mix, so anything above it costs more than just buying the tokens. Log on both axes.
What each commitment needs before it pays for itself.
Everything the calculator does is arithmetic on the figures below. If one of them looks wrong for your situation, override it up there and watch the answer move.
Everything published, all of it measured on GB300 NVL72:[3] 111 tok/s at batch 1 on TP8, 118 on TP16, and 370 with DSpark speculative decode, a 3.14× gain. High-throughput serving is quoted at 2,000+ TPGS.
Treat that last one carefully, because it drives every cost-per-token number here. GB300 NVL72 is a 72-GPU NVLink domain, close to an ideal expert-parallel fabric, and a discrete pod or a two-node cluster over InfiniBand will not match it. vLLM never defines TPGS in the post either, and it almost certainly counts prefill and decode together. Prefill is cheap in bulk and decode isn't. Plan on 1,000.
Two days after this went up, the team at imec's aistack published a measured K3 run, and it is nothing like 1,000 TPGS.[31] They served K3 on an 8× B300 node with SGLang and drove it with real coding-agent traffic: 64 tasks from SWE-Bench Pro, run through Claude Code as the harness. At 16 concurrent sessions the box produced 122 output tokens per second in aggregate, with a median task time of 38 minutes.
That is about 15 tokens per GPU-second, not 1,000. I had the shape of the argument right and the magnitude badly wrong, in the direction that flatters self-hosting.
Work it through and it is worse than a missed slider. 64 tasks at 16 concurrent is four waves of 38 minutes, call it 2.5 hours of a $59.12/hr node, so roughly $150 — against 2.5 hours × 122 tok/s ≈ 1.1M output tokens. That's about $135 per million output tokens DERIVED, which lands next to the $148 unbatched worst case at the top of §Why, not the $2 saturated case. Nine times what Moonshot charges for output. Five times Opus 5.
So the honest split is not output-heavy against prompt-heavy. It's saturated batch against interactive agents. A batch pipeline you can keep full still behaves the way the rest of this page describes. A coding agent does not: agents spend most of their wall-clock compiling, reading files, running tests and waiting on a human, and an endpoint sized for the peak sits idle through all of it. The utilization argument I made against everyone else turns out to bite hardest here.
Two caveats in the other direction, and I don't think either rescues it. Their run was not a throughput benchmark — the setup and evaluation phases idle the endpoint, so the box was never asked for its ceiling. And they say inference-engine tuning is a follow-up post and expect to roughly double what one box serves. Double 122 and you are still more than thirty times under the number this calculator opens on. If you have measured better on this hardware, I would genuinely like to see it.
The slider now goes down to 10 TPGS so you can dial in what was actually measured rather than what was advertised. The default is still 1,000, because that is the figure the vendor publishes and I would rather you see the gap than have me quietly pick a side.
RunPod on-demand, verified in July:[7] 8× B300 at $7.39/GPU-hr secure or $6.94 community, 16× B200 at $5.89, 16× H200 at $4.39. Network volume runs $0.05/GB/mo above a terabyte, or $0.14 for the high-performance tier.
Nebius has B200 at $7.15 on-demand and $3.95 preemptible, H200 at $4.50 and $2.45, and up to 35% off for multi-month reservations.[8] Lambda runs B200 around $6.69–6.99 and includes 22 TiB of local SSD on multi-GPU nodes, which matters more than the hourly rate if you stop and start.[9] Fireworks dedicated is $12/GPU-hr on B300 and $10 on B200.[10]
Don't take the preemptible rate here. A preemption on a model that needs 15 to 45 minutes to reload costs more than the discount saves.
An HGX B300 8-GPU server quotes at $430k to $550k depending on OEM.[14][15] Add NVMe, a 100 GbE uplink, rack and PDU for $15k to $30k, and $5k to $13k to install into liquid-capable colo. Roughly $470k racked and serving. Lead times run 2–4 weeks at Supermicro out to 6–10 at HPE, assuming a single-unit order from a small buyer gets quoted at all.
The quote leaves out the power, which is the line the Power & cooling tab exists to make visible. An HGX B300 caps at 1,100 W per GPU, so 8.8 kW of GPU, and NVIDIA measures the full system at 14.5 kW.[16][17] Multiply by PUE for what the facility actually pulls, then pay for it twice: once as energy and once as the cooling plant and space that high-density colo bills at $175–225 per kW per month.[18]
I went into this assuming the hard part was the money. It's the facility. That 14.5 kW is a dedicated 80 A circuit carrying nothing else, plus 49,500 BTU/hr of heat that has to leave the room every hour the thing runs, and the HGX board wants direct liquid or a hybrid loop. You don't retrofit that into an office.
Then depreciation, in a market that's falling. B200 rental has already come down from about $10/hr to $5.89, so the 25% residual I used is the optimistic case and I'd treat it that way.
And you get the pager. HBM failures, firmware, driver regressions, the liquid loop, and a 1.4 TB model that takes minutes to come back after any of them.
2.8T parameters, 104B active, 16 of 896 experts per token across 93 layers. The weights ship in MXFP4 and were trained that way, so four bits is where it starts and where it ends. That's 1.4 TB of weight tensors, 1.56 TB on disk. If your plan involved pulling a Q4 GGUF down onto a couple of consumer cards, that plan is over.[1][2]
69 of the 93 layers are Kimi Delta Attention, a linear attention that carries fixed state. Only the 24 MLA layers keep a KV cache that grows with the prompt, so long context costs less here than it would on a dense model this size. It's the weights that hurt.
Moonshot doesn't publish per-token cache figures. You can derive them from config.json.[29] MLA caches the compressed latent plus the decoupled RoPE key, shared across heads, so (512 + 64) × 24 layers gives 13.5 KiB/token at fp8 and 27.0 at bf16. The same formula on DeepSeek-V3 returns 68.6 KiB/token against its published ~70 KB, so the method holds.
KDA is the other half. Each head carries a fixed d_k × d_v matrix that gets overwritten on every step instead of appended to, so it doesn't grow with context at all. 128² × 96 heads × 69 layers is 108,527,616 elements, which is 434 MB per sequence at fp32 and 217 at bf16. You pay it in full the moment a slot opens.
So one KDA slot costs what 31,400 tokens of KV cache costs. Under roughly 31k context your memory goes to concurrency instead of context length, which is backwards from how most models size. vLLM says the same thing in passing, that a KDA state is much larger than one ordinary token's KV entry, and it's why they moved to large physical state blocks with a separate prefix-match unit.[30] SGLang ships --mamba-full-memory-ratio as a knob you are expected to turn.[4] Size the deployment by slot count first.
Everything computed from config.json. Both dtypes are engine flags you set at launch, so move them and watch this react.
Driven by the context, concurrency and precision settings on the Memory tab of the calculator.
MIT-derived for most of its length, with two triggers.[23] Model-as-a-Service needs a separate agreement with Moonshot once revenue across you and your affiliates passes $20M over any 12 months. Anything above 100M MAU or $20M monthly revenue has to display "Kimi K3" in the interface. Serving it inside your own company is unencumbered. It's reselling the inference that trips it.
The 8× B300 floor is vLLM's, stated flatly in the day-0 post, with 16× B200 as the other supported shape.[3] SGLang's list runs wider, adding H200, H100 and the MI355X.[4] That gap between the two lists is what §05 is about.
| Configuration | Aggregate HBM | Holds 1.4 TB? |
|---|---|---|
| 8× B300 (288 GB) | 2.30 TB | yes, ~900 GB spare |
| 8× B200 (180 GB) | 1.44 TB | ~40 GB spare, so no |
| 16× B200 | 2.88 TB | yes, 2 nodes |
| 16× H200 (141 GB) | 2.26 TB | yes, see §05 |
| 8× H100 (80 GB) | 0.64 TB | no |
Appearing in a supported-GPU list gets you a codebase that compiles. Neither engine publishes a validated 16× H200 config for K3, so you'd be the one finding the OOMs.
Ratios use the math path each architecture actually runs. FP4 tensor cores on Blackwell, BF16 on Hopper.
It fits and it runs. You pay about 17% more than an 8× B300 node for roughly a seventh of the compute, and it falls down exactly in the high-batch regime that justifies self-hosting.
Hopper has no FP4 tensor cores. Blackwell has dedicated FP4 matrix instructions with hardware microscaling and SM90 doesn't, so MXFP4 on H100 and H200 runs as W4A16.[19][20] Reports have vLLM pinning the Marlin backend, the same path gpt-oss took.[21]
Same bytes in memory on both. What changes is where dequantization happens and what runs afterward.
| 8× B300 | 16× H200 | |
|---|---|---|
| Aggregate HBM | 2.30 TB | 2.26 TB |
| Math path | native MXFP4 | W4A16 to BF16 |
| Cluster compute | 120 PFLOPS | 15.8 PFLOPS, 7.6× less |
| Aggregate bandwidth | 64 TB/s | 76.8 TB/s, 1.2× more |
| RunPod $/hr | $59.12 | $70.24 |
Decode at low batch is bandwidth-bound and the H200 cluster has more of it, so it holds up there. Prefill and high-batch decode are compute-bound, and that's where the 7.6× lands.
H200 leads at batch 1, loses by about 2× at batch 128, and is down the full 7.6× at saturation.
Calibrated to the one published batch-1 point, 111 tok/s on TP8. It ignores prefill, attention cost, all-to-all and scheduler overhead, so the ceilings sit well above anything you will actually measure. vLLM's real number is 2,000+ TPGS, far under the line drawn here. The shape is the part I'd trust. ESTIMATED
Nobody has published K3 throughput on H200, so call it parity at batch 1 and 3–5× worse at serving batch sizes, which lands around $4–8 per million tokens against $1–2 on the B300 node. It still clears the $15/M output API. It loses against $1.85 blended, which was the hard one.
There's one good reason to pick it anyway, which is that you can actually get H200s. RunPod's Instant Clusters offer H200 two-node with 3200 Gbps as a normal product and B300 isn't in the cluster lineup at all.[11] If you've got short prompts, low concurrency, and you need something running this week, I think that's a defensible call.
Two things will cost you money before the model answers anything: the download, and the layout you pick across nodes. Both ship with a default that looks fine and will bill you for it.
Both vllm serve moonshotai/Kimi-K3 and the SkyPilot template pull from Hugging Face at runtime, on the GPU node. At 1 Gbps that's 3.5 hours of a $59/hr box, so about $205 before you serve a token, and more if the first attempt dies and still bills. Hydrate on a CPU pod at $0.20/hr instead and the one-time cost is about a dollar.
The recurring part doesn't go away that easily. A 2 TB network volume is $100/mo whether you serve or not, because you cannot power down and keep the weights for free. Every cold start reloads 1.4 TB into HBM, which is $12 to $46 a time. The usual RunPod answer of baking the model into the image inverts at this size, since a 1.5 TB image will choke the registry and the image cache.
So the tax doesn't kill RunPod generally. It kills RunPod for bursty work, which is exactly the case you'd have picked a cheap on-demand provider for in the first place. Saturated 24/7 it rounds to nothing. If you're bursty, take the high-performance storage tier or move to a provider that includes local NVMe.
From the vLLM day-0 post and the SGLang cookbook. Flags move fast, so check the sources before pasting.
Multi-node K3 has two traps that cost throughput without telling you. Build a layout and see which one you just walked into.
I wanted this to come out the other way. Open weights ought to mean cheaper inference, and I've paid enough API bills to want that badly. At this size, for most workloads, it just doesn't. Our house rule is to instrument the spend before believing any claim about it, and I've found that rule bites hardest on the claims I'd like to be true.
It bit here. I published arithmetic; imec's aistack team published a measurement, and the measurement moved the arithmetic by two orders of magnitude on the one input everything else hangs from.[31] Their post is the more comprehensive piece of work and you should read it — they ran four hardware tiers against real coding tasks, and they measured the thing I care most about and never touched, which is whether the model is any good. K3 resolved 86.4% of their task set against 62.5% for both GLM-5.2 and Opus 4.8, with the honest caveat that SWE-Bench Pro may sit in K3's training data. What they deliberately leave out is power, cooling and maintenance, on the grounds that it varies too much by organisation — which is what §02's facility section and the Power & cooling tab exist to put a number on. The two pieces fit together; neither replaces the other.
So I published the arithmetic instead of the conclusion. Every input in that calculator is yours to change. Put your own prices in and it will cheerfully tell you something different from what it told me. If it does, I'd like to hear about it: hello@3dl.dev.
--
baron
All accessed 28 July 2026. GPU rental prices move weekly, so re-check before committing capital. Figures are tagged MEASURED for a vendor spec or published benchmark, DERIVED for arithmetic on those, and ESTIMATED where nothing is published and we had to model it.