Every major frontier model released in the last two years — Kimi K3, DeepSeek V4, GLM 5.3, GPT-5.6, Llama 4 — shares one architectural secret: they are Mixture-of-Experts (MoE) models. The term gets thrown around in every launch announcement ("Kimi K3 is a 1T-parameter MoE"), but few explanations go beyond "it has experts." This guide explains what MoE actually is, why it slashed inference costs across the industry, and what it means for the API pricing you see as a developer.
The problem MoE solves: parameters aren't the same as compute
For years, the rule was brutal: to make a model smarter, you make it bigger, and a bigger model costs proportionally more to run. A "dense" transformer uses every one of its parameters for every single token it processes. A 70B dense model burns all 70 billion parameters to predict each word — even if the word is "the".
But research consistently showed that most of those parameters aren't needed for any given token. Deciding whether "bank" means a riverbank or a financial institution doesn't require the model's knowledge of protein folding. The parameters are mostly redundant per token — just not redundant overall, because you never know which knowledge a given token will need. That mismatch — enormous knowledge, tiny per-token need — is exactly the inefficiency MoE was designed to eliminate.
The core idea: a router and a switchboard
A Mixture-of-Experts model splits its feed-forward layers (where most parameters live) into many separate blocks called experts. A small neural network called the router looks at each token and decides which experts should handle it.
The flow looks like this:
- A token enters the layer and is converted to an embedding.
- The router scores every expert — say there are 256 of them.
- The token is sent to only the top-k scoring experts (commonly k = 8, sometimes as low as 1).
- Each selected expert processes the token, and their outputs are combined, weighted by the router's scores.
So a model advertised as "1 trillion parameters" might only activate 30–50 billion per token. That ratio — total parameters vs. active parameters — is the single most important number in an MoE spec sheet. Kimi K2's famous architecture, which Kimi K3 builds on, runs 384 experts with 8 active per token plus one shared expert. Qwen's MoE line uses far smaller expert counts. There is no single right answer; it's a trade-off between specialization and routing overhead.
A pizza-restaurant analogy
Imagine a restaurant with one chef who can cook every cuisine on Earth (a dense model). Every order — pizza, sushi, tacos — goes through that one chef, who is equally expensive to employ no matter what you order.
Now imagine a food hall: 200 specialist stalls and a host at the door who looks at your order and routes you to the 8 stalls best suited for it (an MoE model). The food hall contains vastly more culinary knowledge than the single chef, but each dish only pays for 8 stalls' worth of cooking. That's MoE: more total knowledge, less compute per request.
Why this matters for training too
MoE isn't only an inference trick — it also changes training economics. Because each token only flows through a subset of experts, you can train a much larger total model for roughly the compute cost of a much smaller dense one. DeepSeek's V3/V4 line made headlines precisely because it trained a 671B-parameter model on a compute budget rivals spent on models a quarter of the size. Smaller labs and nations suddenly became frontier-competitive because the architecture, not just the budget, determined what was reachable.
The catch is a problem called load balancing: if the router sends too many tokens to the same experts, those experts become bottlenecks while others sit idle. Modern MoE models add auxiliary losses during training to push the router toward even utilization. Get the balancing wrong and the model wastes its own capacity — a quiet failure that shows up as mysterious quality gaps between checkpoints.
The pricing connection: what MoE means for your API bill
Here's the practical takeaway. Because active parameters, not total parameters, drive inference cost, MoE models deliver capability per dollar that dense models simply can't match. This is visible in real marketplace pricing:
| Model | Architecture | Approx. active params | Typical price (in/out per 1M tokens) |
|---|---|---|---|
| Kimi K3 | MoE | ~32B of 1T total | $3 / $15 retail |
| DeepSeek V4 Pro | MoE | ~37B of 671B total | ~$1.6 / $3.96 retail |
| GPT-5.6 Sol | Dense-class flagship | full | $5 / $30 retail |
| Claude Opus 5 | Dense-class flagship | full | $5 / $25 retail |
On an open marketplace like Qubax AI, where compute providers compete on price, MoE models from DeepSeek, GLM, and Moonshot routinely trade at 5–20x below flagship dense models while being competitive on coding and reasoning benchmarks. That gap is not discounting — it's the architectural advantage of MoE showing up directly in your bill.
There's a second-order effect too: MoE models are cheaper to serve at scale because their memory-to-compute ratio favors high batch sizes on modern accelerators. That's why the wholesale prices for these models on open marketplaces keep falling — more providers can serve them efficiently, and competition passes the savings to you.
Common misconceptions
- "Experts are like departments (one for code, one for math)." Mostly false. Experts specialize in ways that don't map to human categories — token-level patterns, syntax, obscure associations. Routing is learned, not designed, and interpretability research shows experts splitting and recombining in surprising ways.
- "MoE models are always cheaper to run." Not always. The router itself adds compute, and all experts must sit in memory even when inactive. For small models on limited hardware, a dense model can be more efficient — which is why Gemma-class small models remain dense.
- "More parameters = more knowledge per response." Capability comes from the active pathway. But total parameters still matter: they give the router a bigger menu of specialists to draw from across different inputs.
- "MoE is new." The idea dates back to 1991 (the Jacobs and Jordan "Adaptive Mixtures of Local Experts" paper). Google's Switch Transformer and GShard made it practical for transformers at scale; the frontier labs made it the default.
The future: MoE everywhere
MoE has essentially won. Every 2026 frontier release of consequence — Kimi K3, GLM 5.3, DeepSeek V4, Qwen 3.8 — uses some form of sparse activation, and research into finer-grained experts (many tiny experts instead of few large ones) keeps pushing the efficiency curve. The next frontier is making the routing itself cheaper and more reliable, with ideas like shared experts, expert offloading for local deployment, and hardware-aware routing.
Try it yourself
The fastest way to feel the MoE difference is to run the same prompt through a flagship dense model and a frontier MoE model back-to-back and compare quality against cost. You can do exactly that on Qubax AI — one API key, all the major MoE models side by side, priced on an open market. The docs include drop-in OpenAI-compatible examples so the swap takes one line.
FAQ
What does Mixture-of-Experts mean in simple terms?
An MoE model contains many separate sub-networks ("experts") and a router that sends each token to only a few of them. You get the knowledge of a huge model while only paying the compute cost of a small fraction of it.
What does "active parameters" mean?
The subset of a model's parameters actually used to process a given token. A 1T-parameter MoE with 32B active parameters uses 3% of its weights per token.
Are MoE models less accurate than dense models?
No. Frontier MoE models match or beat dense models of equal compute. Quality depends on training, not architecture alone.
Which cheap MoE models should I try first?
DeepSeek V4 Pro, GLM 5.3, and Kimi K3 are the strongest price-to-capability ratios right now — all available on Qubax.
