Moonshot open-weighted 2.8 trillion parameters. Here is what it costs to run and when that's worth doing.
Prices verified 28 July 2026. GPU rental moves weekly, so every price in the model is yours to override.
Self-hosting K3 is a sovereignty or fine-tuning decision. It is almost never a cost decision.
The reason is cached input. Moonshot charges $0.30 per million tokens on a prompt cache hit and reports around 90% hit rates on coding traffic. Nothing you rent beats that.
Output tokens go the other way. At $15 per million you clear the API's price at about 1,100 tokens a second, which is easy.
So: output-heavy, self-host. Prompt-heavy with caching, don't.
And there is a floor. One 8× B300 node, because the weights are 1.4 TB and natively four-bit, so there is no smaller configuration and no quantizing your way down to one. Everything below is the arithmetic behind that.
Six situations cover most of it.
| Your situation | What to do |
|---|---|
| Output-heavy, any volume | Self-host. You beat the $15/M output price by 4–15×. |
| Prompt-heavy with cache hits | Use the API. You will not win this one. |
| Bursty, under 200 hr/mo | Rent where local NVMe is included. Not RunPod. |
| 20–50% duty cycle | Reserve. Nebius or Lambda, multi-month. |
| Over 2,600 tok/s sustained, six months running | Buy. Budget the colo, not the garage. |
| Needs to work this week | 16× H200 on Instant Clusters. Read §05 first. |
One number decides it: how busy you keep the box. An unbatched K3 endpoint runs about $148 per million tokens, because vLLM measures 111 tok/s at batch 1 and a $59/hr node divided by 0.4M tokens an hour is not a business. Saturated it lands near $2. Every argument for self-hosting is an argument that you can keep it busy.
Rent an 8× B300 node and you need roughly 7,100 tokens a second sustained to beat the API's blended price. Buy one at $470k and the bar is 2,600 tokens a second, around the clock, for three years. That is 224M tokens a day, every day. Most people who want this are nowhere near it.
Owning is the only configuration that wins outright, and only if you fill it. Against renting on-demand you need 36–48% duty before the capex pays back, against a dedicated endpoint 22–29%. Between those, reserve instead, which nobody finds satisfying and is usually correct.
And if you are picturing one in a rack at the office: a B300 node draws 14.5 kW continuously and needs four tons of cooling. Colo from day one.
Four ways to put K3 into production. Set your workload, hardware and facility, and the four routes reprice together against the API. Defaults are July 2026 list prices.
The bars above amortize the hardware over its life. This is the money actually leaving the account: what you pay on day one, what you pay every month after, and where the cumulative lines cross.
Where the lines cross is where your decision changes. The dashed rule is the API at your mix, so anything above it costs more than paying Moonshot. Log on both axes.
What each commitment needs before it pays for itself.
Everything the calculator does is arithmetic on the figures below. If one of them looks wrong for your situation, override it up there and watch the answer move.
Everything published, all of it measured on GB300 NVL72:[3] 111 tok/s at batch 1 on TP8, 118 on TP16, and 370 with DSpark speculative decode, a 3.14× gain. High-throughput serving is quoted at 2,000+ TPGS.
Treat that last one carefully, because it drives every cost-per-token number here. GB300 NVL72 is a 72-GPU NVLink domain, close to an ideal expert-parallel fabric, and a discrete pod or a two-node cluster over InfiniBand will not match it. vLLM never defines TPGS in the post either, and it almost certainly counts prefill and decode together. Prefill is cheap in bulk and decode isn't. Plan on 1,000.
RunPod on-demand, verified in July:[7] 8× B300 at $7.39/GPU-hr secure or $6.94 community, 16× B200 at $5.89, 16× H200 at $4.39. Network volume runs $0.05/GB/mo above a terabyte, or $0.14 for the high-performance tier.
Nebius has B200 at $7.15 on-demand and $3.95 preemptible, H200 at $4.50 and $2.45, and up to 35% off for multi-month reservations.[8] Lambda runs B200 around $6.69–6.99 and includes 22 TiB of local SSD on multi-GPU nodes, which matters more than the hourly rate if you stop and start.[9] Fireworks dedicated is $12/GPU-hr on B300 and $10 on B200.[10]
Don't take the preemptible rate here. A preemption on a model that needs 15 to 45 minutes to reload costs more than the discount saves.
An HGX B300 8-GPU server quotes at $430k to $550k depending on OEM.[14][15] Add NVMe, a 100 GbE uplink, rack and PDU for $15k to $30k, and $5k to $13k to install into liquid-capable colo. Roughly $470k racked and serving. Lead times run 2–4 weeks at Supermicro out to 6–10 at HPE, assuming a single-unit order from a small buyer gets quoted at all.
The quote leaves out the power, which is the line the Power & cooling tab exists to make visible. An HGX B300 caps at 1,100 W per GPU, so 8.8 kW of GPU, and NVIDIA measures the full system at 14.5 kW.[16][17] Multiply by PUE for what the facility actually pulls, then pay for it twice: once as energy and once as the cooling plant and space that high-density colo bills at $175–225 per kW per month.[18]
It cannot go in your house. That 14.5 kW is a dedicated 80 A circuit carrying nothing else, and 49,500 BTU/hr of heat that has to leave the room every hour it runs. The HGX board wants direct liquid or a hybrid loop, which is not something you retrofit into an office.
It depreciates in a falling market. B200 rental has already come down from about $10/hr to $5.89. The 25% residual is the optimistic case.
You get the pager. HBM failures, firmware, driver regressions, the liquid loop, and a 1.4 TB model that takes minutes to come back after any of them.
2.8T parameters, 104B active, 16 of 896 experts per token across 93 layers. The weights ship in MXFP4 and were trained that way, so four bits is the floor rather than a starting point. That is 1.4 TB of weight tensors, 1.56 TB on disk. Every instinct about pulling a Q4 GGUF onto consumer cards dies here.[1][2]
69 of the 93 layers are Kimi Delta Attention, a linear attention that carries fixed state. Only the 24 MLA layers keep a KV cache that grows with the prompt. Long context costs less than it would on a dense model this size. The weights are what bind.
Moonshot doesn't publish per-token cache figures. You can derive them from config.json.[28] MLA caches the compressed latent plus the decoupled RoPE key, shared across heads, so (512 + 64) × 24 layers gives 13.5 KiB/token at fp8 and 27.0 at bf16. The same formula on DeepSeek-V3 returns 68.6 KiB/token against its published ~70 KB, so the method holds.
KDA is the other half. Each head carries a fixed d_k × d_v matrix that gets updated rather than appended to, so it doesn't grow with context at all. 128² × 96 heads × 69 layers is 108,527,616 elements, which is 434 MB per sequence at fp32 and 217 at bf16. You pay it in full the moment a slot opens.
So one KDA slot costs what 31,400 tokens of KV cache costs. Under roughly 31k context your memory goes to concurrency rather than context length, which is backwards from most models. vLLM says the same thing in passing, that a KDA state is much larger than one ordinary token's KV entry, and it's why they moved to large physical state blocks with a separate prefix-match unit.[29] SGLang ships --mamba-full-memory-ratio as a knob you are expected to turn.[4] Size the deployment by slot count first.
Everything computed from config.json. The two dtypes are engine flags, not model properties.
Driven by the context, concurrency and precision settings on the Memory tab of the calculator.
MIT-derived for most of its length, with two triggers.[23] Model-as-a-Service needs a separate agreement with Moonshot once revenue across you and your affiliates passes $20M over any 12 months. Anything above 100M MAU or $20M monthly revenue has to display "Kimi K3" in the interface. Serving it internally is unencumbered. Reselling inference is the trigger.
The 8× B300 floor is vLLM's, stated flatly in the day-0 post, with 16× B200 as the other supported shape.[3] SGLang's list runs wider — H200, H100 and the MI355X.[4] The difference between those two lists is the whole of §05.
| Configuration | Aggregate HBM | Holds 1.4 TB? |
|---|---|---|
| 8× B300 (288 GB) | 2.30 TB | yes, ~900 GB spare |
| 8× B200 (180 GB) | 1.44 TB | ~40 GB spare, so no |
| 16× B200 | 2.88 TB | yes, 2 nodes |
| 16× H200 (141 GB) | 2.26 TB | yes, see §05 |
| 8× H100 (80 GB) | 0.64 TB | no |
Sitting in a supported-GPU list is not the same as having a tested recipe. Neither engine publishes a validated 16× H200 config for K3, so you would be the one finding the OOMs.
Ratios use the math path each architecture actually runs. FP4 tensor cores on Blackwell, BF16 on Hopper.
It fits and it runs. You pay about 17% more than an 8× B300 node for roughly a seventh of the compute, and it falls down exactly in the high-batch regime that justifies self-hosting.
Hopper has no FP4 tensor cores. Blackwell has dedicated FP4 matrix instructions with hardware microscaling and SM90 doesn't, so MXFP4 on H100 and H200 runs as W4A16.[19][20] Reports have vLLM pinning the Marlin backend, the same path gpt-oss took.[21]
Same bytes in memory on both. What changes is where dequantization happens and what runs afterward.
| 8× B300 | 16× H200 | |
|---|---|---|
| Aggregate HBM | 2.30 TB | 2.26 TB |
| Math path | native MXFP4 | W4A16 to BF16 |
| Cluster compute | 120 PFLOPS | 15.8 PFLOPS, 7.6× less |
| Aggregate bandwidth | 64 TB/s | 76.8 TB/s, 1.2× more |
| RunPod $/hr | $59.12 | $70.24 |
Decode at low batch is bandwidth-bound and the H200 cluster has more of it, so it holds up there. Prefill and high-batch decode are compute-bound, and that's where the 7.6× lands.
H200 leads at batch 1, loses by about 2× at batch 128, and is down the full 7.6× at saturation.
Calibrated to the one published batch-1 point, 111 tok/s on TP8. It ignores prefill, attention cost, all-to-all and scheduler overhead, so the ceilings sit well above anything you will measure. vLLM's real number is 2,000+ TPGS, far under the ceiling drawn here. What it gets right is the shape. ESTIMATED
Nobody has published K3 throughput on H200, so call it parity at batch 1 and 3–5× worse at serving batch sizes, which lands around $4–8 per million tokens against $1–2 on the B300 node. It still clears the $15/M output API. It loses against $2.31 blended, which was the hard one.
One good reason to pick it anyway: you can actually get H200s. RunPod's Instant Clusters offer H200 two-node with 3200 Gbps as a normal product and B300 isn't in the cluster lineup at all.[11] Short prompts, low concurrency, needed this week, that's a defensible call.
Two things will cost you money before the model answers anything: the download, and the layout you pick across nodes. Both have a wrong default that looks correct.
Both vllm serve moonshotai/Kimi-K3 and the SkyPilot template pull from Hugging Face at runtime, on the GPU node. At 1 Gbps that's 3.5 hours of a $59/hr box, so about $205 before you serve a token, and more if the first attempt dies and still bills. Hydrate on a CPU pod at $0.20/hr instead and the one-time cost is about a dollar.
The recurring part doesn't go away that easily. A 2 TB network volume is $100/mo whether you serve or not, because you cannot power down and keep the weights for free. Every cold start reloads 1.4 TB into HBM, which is $12 to $46 a time. The usual RunPod answer of baking the model into the image inverts at this size, since a 1.5 TB image will choke the registry and the image cache.
So the tax doesn't kill RunPod generally, it kills RunPod for bursty work, which is the case you'd pick a cheap on-demand provider for in the first place. Saturated 24/7 it rounds to nothing. If you're bursty, take the high-performance storage tier or move to a provider that includes local NVMe.
From the vLLM day-0 post and the SGLang cookbook. Flags move fast, so check the sources before pasting.
Multi-node K3 has two traps that cost throughput without telling you. Build a layout and see which one you just walked into.
Our house rule is to instrument the spend before trusting any claim about it, and that applies hardest to claims we would like to be true. Open weights did not make self-hosting cheap. So rather than publish the conclusion, we published the arithmetic, with the inputs exposed: disagree with our prices and the model still works, it just answers differently.
If your numbers land somewhere other than ours, we would rather hear it than not.
All accessed 28 July 2026. GPU rental prices move weekly, so re-check before committing capital. Figures are tagged MEASURED for a vendor spec or published benchmark, DERIVED for arithmetic on those, and ESTIMATED where nothing is published and we had to model it.