Most LLM APIs don't return JSON. They return prose — and if your app needs structured data, you're left regexing model output and praying. There's a better way: build a bulletproof structured-output extractor that combines JSON mode, schema validation, and a smart repair loop. In this tutorial you'll build exactly that in Python, in under 100 lines.
What We're Building
A reusable extract() function that takes any text and a Pydantic schema, and returns a validated, typed object. The pipeline:
- Prompt the model with the schema inline (JSON mode where supported)
- Parse the raw response, stripping markdown fences and stray text
- Validate against a Pydantic model
- Repair: if validation fails, send the error back to the model for a targeted fix
- Cap the repair loop so a broken model can't burn your budget
This works with any OpenAI-compatible API. We'll point it at Qubax, which gives you one endpoint for models from OpenAI, Anthropic, Google, DeepSeek, Zhipu and more — useful, because structured-output reliability varies a lot between models, and you want to swap cheaply.
Step 1: Setup
pip install pydantic openai
export QB_API_KEY="sk-..." # your key from qubax.ai
export QB_BASE_URL="https://api.qubax.ai/v1"Step 2: Define Your Schema
Pydantic is the contract. Everything the model produces gets checked against it:
from pydantic import BaseModel, Field
from typing import Optional
from enum import Enum
class Sentiment(str, Enum):
positive = "positive"
negative = "negative"
neutral = "neutral"
class ReviewInsight(BaseModel):
sentiment: Sentiment
score: float = Field(ge=0, le=1, description="0.0 = awful, 1.0 = perfect")
products: list[str] = Field(description="Products mentioned, lowercase")
summary: str = Field(max_length=200)
needs_human_review: bool = False
escalation_reason: Optional[str] = None
class ReviewInsights(BaseModel):
results: list[ReviewInsight]Step 3: The Robust Extractor
The core trick: never trust the raw string. Strip fences, find the outermost JSON object, and validate:
import json, os, re
from openai import OpenAI
from pydantic import BaseModel, ValidationError
client = OpenAI(
api_key=os.environ["QB_API_KEY"],
base_url=os.environ["QB_BASE_URL"],
)
def extract_json(text: str) -> str:
"""Strip code fences and pull out the outermost JSON object."""
text = re.sub(r"(json)?", "", text).strip() start = text.find("{") if start == -1: raise ValueError("No JSON object found in response") depth = 0 for i, ch in enumerate(text[start:], start): if ch == "{": depth += 1 elif ch == "}": depth -= 1 if depth == 0: return text[start:i + 1] raise ValueError("Unbalanced JSON in response")
def extract(text: str, schema: type[BaseModel], model: str, maxrepairattempts: int = 2) -> BaseModel: schemajson = json.dumps(schema.modeljsonschema(), indent=2) messages = [ {"role": "system", "content": "Extract structured data from the user's text. Respond ONLY with " "JSON that validates against this schema:\n" + schemajson}, {"role": "user", "content": text}, ] for attempt in range(maxrepairattempts + 1): resp = client.chat.completions.create( model=model, messages=messages, temperature=0, ) raw = resp.choices[0].message.content try: return schema.modelvalidatejson(extractjson(raw)) except (ValueError, ValidationError) as e: if attempt == maxrepair_attempts: raise # give up — caller decides fallback messages.append({"role": "assistant", "content": raw}) messages.append({"role": "user", "content": f"Your JSON failed validation:\n{e}\n" "Return ONLY corrected JSON conforming to the schema."})
Note what the repair loop does: it shows the model **its own invalid output plus the exact validation error**. That's far more effective than retrying blind — validation errors like `score: Input should be less than or equal to 1` tell the model exactly what to fix.
## Step 4: Use It
python review = """ Bought the AeroPress Go last month and it's been great for travel, though the included scoop cracked in week one. My partner's original AeroPress is still going strong after 3 years. """
insights = extract(review, ReviewInsights, model="gpt-5.6-luna") for r in insights.results: print(r.sentiment, r.score, r.products, "-", r.summary)
Because the result is a Pydantic model, downstream code gets type-checked fields, sensible errors, and IDE autocompletion — not a dict of strings you hope are right.
## Step 5: Production Hardening
Three additions that separate a demo from a system:
**Cost control.** Repair loops consume tokens. Cap spend per call:
python resp = client.chat.completions.create( model=model, messages=messages, temperature=0, max_tokens=2000, # hard ceiling on the expensive direction ) ```
Model fallback. Structured-output failure rates differ wildly by model. Route extraction failures to a stronger model automatically — a weak model at $0.01/M tokens for the first attempt, a strong one only when needed. On Qubax you can compare per-model input/output prices on the models page and pick the pair that minimizes your expected cost per successful extraction.
Telemetry. Log model, repair_attempts, and latency per call. When your extraction failure rate crosses even 2%, that number tells you which model to replace — before your users notice.
Common Pitfalls
- Don't ask for JSON in prose. "Return a JSON object" is weaker than an inline schema plus "respond ONLY with JSON."
- Beware of nested schemas. Deeply nested models multiply validation failures. Flatten where you can.
- temperature=0 for extraction. Creativity is the enemy of schema compliance.
- Validate enums. Free-text fields like "positive/negative/neutral" drift ("Positive", "POSITIVE", "mostly positive"). Enum types catch this for free.
FAQ
Does every model support structured outputs / JSON mode?
Native JSON modes differ across providers, which is why the extractor above works at the prompt-and-validate level — it functions with any chat model, and repair loops cover the ones without native JSON mode.
Why Pydantic instead of raw JSON parsing?
Raw parsing gives you a dict with unchecked fields. Pydantic gives you type validation, range constraints (ge, le), enums, and machine-readable error messages that power the repair loop.
How many repair attempts should I allow?
Two is a good default. Beyond that you're usually better off failing over to a stronger model than retrying the same one.
Which cheap models are good at structured output?
It changes monthly — check live pricing on Qubax models and benchmark extraction accuracy on your own data before committing. The full API reference is in the Qubax docs.
One API key, every major model, wholesale pricing — test your extractor against a dozen models on [Qubax](https://qubax.ai/models).
