How to Build a Cost-Saving AI Model Router in Node.js (Cut Your API Bill 80%+)

Published: August 30, 2026 | Category: AI Tutorial

Here's an uncomfortable truth about AI apps in 2026: most of them overpay for intelligence. Your support chatbot doesn't need a frontier model to say "your refund was processed." Your summarizer doesn't need Opus-class reasoning to condense a newsletter.

The fix isn't a cheaper single model — it's a model router: a thin layer in your backend that inspects each request and sends it to the cheapest model that can handle it, with automatic failover to a stronger model when needed.

In this tutorial you'll build one in Node.js in about 20 minutes. It routinely cuts AI spend by 60–90% with no visible quality loss.

What You'll Build

A routing layer that:

1. Classifies request difficulty (trivial / standard / hard)
2. Picks the cheapest capable model per tier
3. Falls back automatically if a model errors or times out
4. Logs cost per request so you can prove the savings

We'll use the OpenAI SDK format — the same one used by Qubax's OpenAI-compatible API — so the code works with any compatible endpoint.

Prerequisites

• Node.js 18+
• An API key from your provider (get a Qubax key here)
• npm install openai

Step 1: Define Your Model Tiers

The core idea: pick two or three models across a price/quality spectrum. Here's a realistic tier setup with current Qubax pricing:

// config.js
const TIERS = {
  // ~$0.015 in / $0.09 out per 1M tokens — tiny classification, routing, extraction
  budget:  { model: "gpt-5.6-luna",       maxTokens: 1024 },
  // ~$0.075 in / $0.473 out per 1M tokens — everyday chat, summaries, RAG answers
  mid:     { model: "glm-5.2",            maxTokens: 4096 },
  // ~$0.375 in / $1.875 out per 1M tokens — complex reasoning, agentic coding
  premium: { model: "claude-opus-5",      maxTokens: 8192 },
};
module.exports = { TIERS };

Why these three? GPT-5.6 Luna costs roughly 25x less than Claude Opus 5 — but it can't plan a multi-step refactor. GLM 5.2 sits in between and handles 80% of production traffic well. Matching task to tier is where the money is.

Step 2: Create a Shared Client

// client.js
const OpenAI = require("openai");

const client = new OpenAI({
  apiKey: process.env.API_KEY,
  baseURL: process.env.BASE_URL, // your OpenAI-compatible endpoint
});

module.exports = client;

Set API_KEY and BASE_URL in your environment. Because we use the OpenAI SDK shape, swapping providers later is a config change, not a rewrite.

Step 3: Classify Request Difficulty

The cheapest classifier is often the model itself — but don't burn premium tokens on routing. Use the budget model with a strict prompt, plus cheap heuristics first:

// classifier.js
const client = require("./client");

const HARD_SIGNALS = [
  /debug|stack trace|why does this (fail|break)/i,
  /architect|refactor|migration/i,
  /math|proof|derive|calculate/i,
];

async function classify(messages) {
  const lastUser = [...messages].reverse().find(m => m.role === "user")?.content ?? "";
  const text = typeof lastUser === "string" ? lastUser : JSON.stringify(lastUser);

  // Fast path: obvious hard tasks skip the classifier call entirely
  if (HARD_SIGNALS.some(r => r.test(text))) return "premium";
  if (text.length < 200 && /hi|hello|thanks|ok\b/i.test(text)) return "budget";

  // Otherwise, let the budget model judge
  const res = await client.chat.completions.create({
    model: "gpt-5.6-luna",
    temperature: 0,
    max_tokens: 10,
    messages: [
      { role: "system", content:
        "Classify the user request as one word: trivial (simple lookup, rephrase, short answer) " +
        "or hard (reasoning, code, multi-step). Respond with one word only." },
      { role: "user", content: text.slice(0, 1000) },
    ],
  });

  const verdict = res.choices[0].message.content.trim().toLowerCase();
  if (verdict.startsWith("hard")) return "premium";
  if (verdict.startsWith("trivial")) return "budget";
  return "mid";
}

module.exports = { classify };

Notes on the design:

• Regex fast paths avoid a classifier call for the clearest cases — that's free routing.
• The classifier itself runs on the budget tier, costing a fraction of a cent per request.
• `temperature: 0` keeps classification deterministic.

Step 4: Route With Automatic Failover

// router.js
const client = require("./client");
const { TIERS } = require("./config");
const { classify } = require("./classifier");

async function complete(messages, { forceTier } = {}) {
  const tierName = forceTier ?? await classify(messages);
  const chain = buildChain(tierName); // e.g. premium -> mid -> budget fallbacks

  for (const tier of chain) {
    const t0 = Date.now();
    try {
      const res = await client.chat.completions.create({
        model: TIERS[tier].model,
        messages,
        max_tokens: TIERS[tier].maxTokens,
      });
      logUsage(tier, TIERS[tier].model, res.usage, Date.now() - t0);
      return res.choices[0].message.content;
    } catch (err) {
      console.error(`[${tier}] ${TIERS[tier].model} failed: ${err.message}`);
      // fall through to the next model in the chain
    }
  }
  throw new Error("All models in fallback chain failed");
}

function buildChain(tier) {
  const order = ["premium", "mid", "budget"];
  // Start at the chosen tier, then fall back DOWN (cheaper) — quality is capped, cost is minimized
  return order.slice(order.indexOf(tier));
}

function logUsage(tier, model, usage, ms) {
  console.log(JSON.stringify({
    tier, model,
    in: usage?.prompt_tokens, out: usage?.completion_tokens, ms,
  }));
}

module.exports = { complete };

Two deliberate choices:

• Fallbacks go down the price ladder, not up. If Claude Opus 5 times out, do you really want to retry on another premium model? Usually a mid-tier model answers fine — and if the user complains, you escalate.
• Every response logs tokens. This is how you'll measure savings in Step 6.

Step 5: Use It Like Any Other Chat Call

// server.js
const express = require("express");
const { complete } = require("./router");

const app = express();
app.use(express.json());

app.post("/chat", async (req, res) => {
  try {
    const reply = await complete(req.body.messages);
    res.json({ reply });
  } catch (e) {
    res.status(502).json({ error: "upstream failure" });
  }
});

app.listen(3000, () => console.log("router up on :3000"));

Your app code never knows or cares which model answered.

Step 6: Measure the Savings

After a week, aggregate your logs and compute what the same traffic would have cost at a flat premium model vs. your routed mix. A typical distribution looks like:

That's roughly an 89% reduction on output tokens — usually the dominant cost — before any discount pricing. And if you route through a marketplace like Qubax, each tier's base price is already discounted from retail (Claude Opus 5 is 92% off retail there), so the savings stack.

Hardening Tips Before You Ship

• Add a timeout per tier (AbortController + setTimeout) so a slow premium model falls back fast.
• Cache classifier verdicts by request hash — identical prompts don't need re-classifying.
• Let users opt out. A forceTier: "premium" flag in your API keeps power users happy.
• Track quality, not just cost. Log thumbs-up/down per tier; if the budget tier's satisfaction drops, move that intent up a tier.
• Pin model versions in config, not in code, so you can A/B new models without deploys.

FAQ

Does routing hurt response quality?

Not if your tiers are honest. Route conservatively at first (start everything at "mid"), watch your quality signals, then push trivial traffic down. Most teams find 50–70% of traffic is genuinely trivial.

Can I use this pattern with non-OpenAI models?

Yes. The OpenAI SDK format is a de facto standard — most providers, including Qubax, expose OpenAI-compatible endpoints, so one client handles Claude, Gemini, GLM, DeepSeek, and more via model name alone.

How do I handle long conversations with cheap models?

Cheap models often have shorter context windows. Add a guard: if the token count of messages exceeds the tier's window, promote the request to a bigger tier.

What's the fastest way to get API keys for all these models?

A single Qubax account gives you one key and one endpoint for 100+ models, with per-model pricing listed transparently — see the docs for the base URL and quickstart.

Wrap-up: a ~100-line router, three well-chosen models, and honest classification can cut your AI bill by an order of magnitude. Compare model prices side by side at qubax.ai/models and pick your three tiers today.