Methodology

Quality Scores — LiveBench

We use LiveBench for model quality scores. LiveBench is a contamination-resistant benchmark that:

• Releases new questions monthly to prevent test-set contamination
• Uses objective, judge-free evaluation (no LLM-as-judge bias)
• Covers 7 categories: Reasoning, Coding, Agentic Coding, Mathematics, Data Analysis, Language, and Instruction Following
• Per-task scores are averaged within each category, then categories are averaged for an overall score (0–100)

We sync every LiveBench release (11 historical releases from June 2024 to June 2026) and map their model IDs to ours via an alias table. Models without a LiveBench entry are not shown — we never self-report quality.

Latency — Time to First Token (TTFT)

TTFT is measured from request send to the first streamed content token. We report two sources, combined:

• Real traffic — every chat and API streaming request through Qubax, recorded in our telemetry system
• Synthetic probes — a 16-token request (“Reply with exactly: pong”) sent to every benchmarked model every 15 minutes, so low-traffic models still have fresh latency data

The leaderboard shows p95 (95th percentile) from a 7-day rolling window. Probe p95 is preferred when available (consistent load); real-traffic p95 is used as a fallback. Synthetic probes are excluded from user billing entirely.

Success Rate

Success rate is the percentage of requests that returned a usable response. We exclude client-side errors that are not the model's or network's fault:

• Client aborts (user closed the connection)
• Insufficient credits (user-side billing)
• Rate limiting (user-side quota)
• Bad requests (user-side parameter errors)

Both real traffic and synthetic probes contribute to the success rate shown. The sample count (real + probe) is displayed next to the percentage.

Price — Blended $/1M Tokens

Price is our sell price (what users pay), not upstream cost. The blended price uses a 3:1 input-to-output ratio (typical chat workload):

blended = (input_price × 3 + output_price) / 4

Prices are pulled from our live pricing engine (the same prices shown on our model pages and price calculator).

Quality per Dollar (Q/$)

Q/$ is a simple value metric: LiveBench overall score divided by blended price.

Q/$ = LiveBench score ÷ blended $/1M

A model with a score of 80 at $1/1M has Q/$ = 80. A model with a score of 70 at $0.50/1M has Q/$ = 140. Higher is better — you get more quality per dollar spent.

Data Freshness

What We Don't Show

• Upstream cost or profit margins — never exposed
• Individual user request data — all reliability stats are aggregated
• Self-reported or vendor-published benchmarks — we only use independently verified scores