How to Build an AI Text Summarizer API in Python (With Length Control and Cost Tracking)

Summarization is the "hello world" of production AI — simple enough to build in an afternoon, nuanced enough to teach you almost every lesson that matters: prompt design, length control, structured output, streaming, and cost management. In this tutorial, we'll build a complete summarization API in Python with FastAPI, including three summarization modes, token-aware length control, and real-time cost tracking.

By the end you'll have a service you can actually deploy — and a template you can reuse for any AI feature.

What We're Building

A REST API with one main endpoint:

POST /summarize
{
  "text": "...your long document...",
  "style": "brief | detailed | bullets",
  "max_words": 100
}

Response:

{
  "summary": "...",
  "input_tokens": 8421,
  "output_tokens": 137,
  "cost_usd": 0.001642
}

Prerequisites

• Python 3.11+
• An API key for any OpenAI-compatible endpoint (we'll make the base URL configurable so you can point it at any provider or aggregator)

Install dependencies:

pip install fastapi uvicorn httpx pydantic tiktoken

Step 1: Config That Works With Any Provider

Keep your base URL and key in environment variables so you can swap providers without touching code:

# config.py
import os

API_BASE_URL = os.environ.get("API_BASE_URL", "https://api.example-ai.com/v1")
API_KEY = os.environ["API_KEY"]
MODEL = os.environ.get("SUMMARIZER_MODEL", "a-capable-cheap-model")

Cost tip before we write any code: summarization has a huge input-to-output token ratio (often 50:1), so input pricing dominates your bill. A mid-tier model at $0.05/M input often produces summaries indistinguishable from a $1/M flagship. You can compare real per-token prices across models on Qubax to pick the right tier — and set the model per-request later if you want.

Step 2: The Prompt With Length Control

The most common failure in summarization APIs is length — you ask for "brief" and get 400 words. The fix is to specify the constraint in two places: the system prompt and a soft budget computed from the target length.

STYLE_PROMPTS = {
    "brief": "Write a tight executive summary.",
    "detailed": "Write a comprehensive summary that preserves key numbers, names, and conclusions.",
    "bullets": "Write a summary as 5-8 bullet points, each under 25 words.",
}

SYSTEM_TEMPLATE = """You are a professional summarizer.
{style_instruction}
Hard limit: approximately {max_words} words. Do not exceed it.
Do not add information that is not in the source text.
Do not start with phrases like "This document discusses" — lead with the substance."""

Why both belt and suspenders? Because models treat word counts as suggestions. The downstream truncation guard in Step 4 is the actual guarantee.

Step 3: The Core Client With Retries

# summarizer.py
import httpx, asyncio, tiktoken
from config import API_BASE_URL, API_KEY

_client = httpx.AsyncClient(
    base_url=API_BASE_URL,
    headers={"Authorization": f"Bearer {API_KEY}"},
    timeout=60.0,
)

async def call_model(messages: list, model: str, max_retries: int = 3) -> dict:
    for attempt in range(max_retries):
        try:
            r = await _client.post("/chat/completions", json={
                "model": model,
                "messages": messages,
                "temperature": 0.2,   # low = faithful summaries, less invention
            })
            r.raise_for_status()
            return r.json()
        except (httpx.HTTPStatusError, httpx.TransportError) as e:
            if attempt == max_retries - 1:
                raise
            await asyncio.sleep(2 ** attempt)  # 1s, 2s, 4s backoff

Note the temperature: 0.2. Summarization is a fidelity task, not a creativity task — low temperature measurably reduces hallucinated details.

Step 4: Cost Tracking and the Length Guard

The API returns usage in every response. Capture it, price it, and enforce your length cap:

# pricing per million tokens — swap in your model's real numbers
PRICES = {  # model: (input $/M, output $/M)
    "a-capable-cheap-model": (0.05, 0.20),
}

def cost_usd(usage: dict, model: str) -> float:
    in_p, out_p = PRICES[model]
    return (usage["prompt_tokens"] / 1e6 * in_p
          + usage["completion_tokens"] / 1e6 * out_p)

def enforce_word_limit(summary: str, max_words: int) -> str:
    words = summary.split()
    if len(words) <= max_words:
        return summary
    # cut at sentence boundary, not mid-word
    trimmed = " ".join(words[:max_words])
    for sep in (". ", "! ", "? "):
        idx = trimmed.rfind(sep)
        if idx > max_words * 5:  # keep a sensible minimum
            return trimmed[:idx + 1]
    return trimmed + "…"

Step 5: The FastAPI Endpoint

# main.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
import tiktoken
from summarizer import call_model
from config import MODEL
from prompts import STYLE_PROMPTS, SYSTEM_TEMPLATE
from pricing import cost_usd, enforce_word_limit

app = FastAPI(title="Summarizer API")

class SummarizeRequest(BaseModel):
    text: str = Field(min_length=200, max_length=400_000)
    style: str = Field(default="brief", pattern="^(brief|detailed|bullets)$")
    max_words: int = Field(default=100, ge=20, le=1000)

@app.post("/summarize")
async def summarize(req: SummarizeRequest):
    system = SYSTEM_TEMPLATE.format(
        style_instruction=STYLE_PROMPTS[req.style],
        max_words=req.max_words,
    )
    messages = [
        {"role": "system", "content": system},
        {"role": "user", "content": f"Summarize the following text:\n\n{req.text}"},
    ]
    try:
        resp = await call_model(messages, model=MODEL)
    except Exception:
        raise HTTPException(status_code=502, detail="Upstream model call failed")

    summary = resp["choices"][0]["message"]["content"]
    usage = resp["usage"]

    return {
        "summary": enforce_word_limit(summary, req.max_words),
        "input_tokens": usage["prompt_tokens"],
        "output_tokens": usage["completion_tokens"],
        "cost_usd": round(cost_usd(usage, MODEL), 6),
    }

Run it:

uvicorn main:app --reload --port 8000

Test:

curl -X POST localhost:8000/summarize -H 'Content-Type: application/json' -d '{
  "text": "<your 5000-word document here>",
  "style": "bullets",
  "max_words": 150
}'

Leveling Up: Three Improvements Worth Making

1. Map-reduce for very long documents. Above ~100K tokens, send chunks to the model in parallel, summarize each chunk, then summarize the summaries. With asyncio.gather this is both faster and cheaper per document than one giant context call — and it sidesteps context limits entirely.

2. Cache repeated inputs. Support docs, news feeds, and changelogs get re-summarized constantly. Hash the input text; return the cached summary for identical inputs. For partially-shared prefixes (same system prompt, similar docs), provider-level prompt caching discounts repeated input tokens automatically.

3. Route by document size. Small docs (under 2K tokens) can go to an ultra-cheap model with no quality loss; only route long, complex documents to premium tiers. A simple size-based router cut one team's summarization bill by 80%. Our model router tutorial shows the pattern in depth.

What This Tutorial Teaches Beyond Summarization

Every pattern here — low temperature for fidelity tasks, retry with backoff, usage-based cost tracking, output guards, provider-agnostic config — transfers directly to extraction, translation, classification, and moderation pipelines. Build it once, reuse the skeleton forever.

Ready to pick the right model for your workload? Browse live pricing across dozens of models on Qubax's marketplace, and check the Qubax docs for more integration guides.

FAQ

Which model should I use for summarization?

Start with a cheap mid-tier model and eval against a premium one on your documents. Because summarization is input-heavy, the input price dominates cost — and mid-tier models are often within a few percentage points on summary quality. Compare real prices on Qubax.

How do I stop the model from exceeding my word limit?

Combine three layers: explicit word limits in the system prompt, low temperature, and a programmatic sentence-boundary trim as a hard guarantee (shown in Step 4).

How do I summarize documents longer than the context window?

Use map-reduce: chunk the document, summarize chunks in parallel, then summarize the combined chunk-summaries. It's cheaper and faster than one massive context call.

How much does it cost to summarize 1 million words?

Roughly 1.33M input tokens. At $0.05/M input that's about $0.07; at $1/M it's about $1.33 — a 20× spread, which is why model choice matters so much for this workload.