Streaming is the difference between a chatbot that feels alive and one that feels broken. When a user sends a message, they want to see words appear immediately — not stare at a spinner for 15 seconds while the entire response generates in the background.
This tutorial walks you through building a streaming AI chatbot using Server-Sent Events (SSE) and any OpenAI-compatible API endpoint, including Qubax. We will cover the server, the client, error handling, and production hardening.
Why Server-Sent Events?
SSE is the industry standard for streaming text from an AI API to a browser. Here is why it beats the alternatives:
- Simplest protocol: Built on HTTP, no special server or protocol needed
- One-way is enough: AI responses flow server→client; you do not need bidirectional WebSockets for text generation
- Auto-reconnect: The browser's EventSource API automatically reconnects if the connection drops
- Works everywhere: Supported by all modern browsers, curl, and every HTTP library
Prerequisites
- Node.js 20+ (or Python 3.11+)
- A Qubax API key (sign up at qubax.ai)
- Basic familiarity with Express or FastAPI
Part 1: The Server (Node.js + Express)
Step 1: Install Dependencies
npm init -y
npm install express cors dotenvStep 2: Create the Streaming Endpoint
Create server.js:
import express from "express";
import cors from "cors";
import "dotenv/config";
const app = express();
app.use(cors());
app.use(express.json());
const QUBAX_API_URL = "https://api.qubax.ai/v1/chat/completions";
const API_KEY = process.env.QUBAX_API_KEY;
app.post("/api/chat", async (req, res) => {
const { messages, model = "gpt-5.6-luna" } = req.body;
// --- Set SSE headers ---
res.writeHead(200, {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
Connection: "keep-alive",
"X-Accel-Buffering": "no", // Critical for nginx proxies
});
try {
const upstream = await fetch(QUBAX_API_URL, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${API_KEY}`,
},
body: JSON.stringify({
model,
messages,
stream: true,
max_tokens: 2000,
}),
});
if (!upstream.ok) {
const errBody = await upstream.text();
res.write(`event: error\n`);
res.write(`data: ${JSON.stringify({ error: errBody })}\n\n`);
res.end();
return;
}
// --- Parse and forward SSE chunks ---
const reader = upstream.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split("\n");
buffer = lines.pop(); // Keep incomplete line in buffer
for (const line of lines) {
if (!line.startsWith("data: ")) continue;
const data = line.slice(6);
if (data === "[DONE]") {
res.write("event: done\n");
res.write("data: {}\n\n");
continue;
}
// Forward the chunk to the client
res.write(`data: ${data}\n\n`);
}
}
res.end();
} catch (err) {
console.error("Stream error:", err);
res.write(`event: error\n`);
res.write(`data: ${JSON.stringify({ error: err.message })}\n\n`);
res.end();
}
});
// --- Heartbeat to keep connection alive ---
setInterval(() => {
// Prevents proxy/load-balancer timeouts on idle connections
}, 15000);
app.listen(3000, () => {
console.log("Server running on http://localhost:3000");
});The key patterns here:
- Set SSE headers before any data —
Content-Type: text/event-streamtells the browser to treat the response as a stream - Forward chunks line by line — the upstream API sends
data: {...}lines; we parse and relay them - Handle the `[DONE]` sentinel — OpenAI-compatible APIs use this to signal completion
- Always call `res.end()` — even on errors, to avoid hanging connections
Step 3: Add Rate Limiting
For production, add basic rate limiting to prevent abuse:
const rateLimit = new Map();
function rateLimiter(req, res, next) {
const ip = req.ip;
const now = Date.now();
const windowMs = 60_000; // 1 minute
const maxRequests = 20;
if (!rateLimit.has(ip)) {
rateLimit.set(ip, []);
}
const timestamps = rateLimit.get(ip).filter((t) => now - t < windowMs);
if (timestamps.length >= maxRequests) {
return res.status(429).json({ error: "Rate limit exceeded" });
}
timestamps.push(now);
rateLimit.set(ip, timestamps);
next();
}
app.post("/api/chat", rateLimiter, async (req, res) => {
// ... streaming logic
});Part 2: The Client (Vanilla JavaScript)
The HTML
<div id="chat">
<div id="messages"></div>
<form id="form">
<input id="input" placeholder="Ask anything..." autocomplete="off" />
<button type="submit">Send</button>
</form>
</div>The JavaScript
const form = document.getElementById("form");
const input = document.getElementById("input");
const messages = document.getElementById("messages");
form.addEventListener("submit", async (e) => {
e.preventDefault();
const userText = input.value.trim();
if (!userText) return;
// Show user message
messages.innerHTML += `<div class="msg user">${userText}</div>`;
input.value = "";
// Create assistant message bubble
const bubble = document.createElement("div");
bubble.className = "msg assistant";
messages.appendChild(bubble);
messages.scrollTop = messages.scrollHeight;
// --- Stream via fetch + ReadableStream ---
const response = await fetch("/api/chat", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
messages: [{ role: "user", content: userText }],
model: "gpt-5.6-luna",
}),
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
let buffer = "";
while (true) {
const { done, value } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split("\n");
buffer = lines.pop();
for (const line of lines) {
if (line.startsWith("data: ")) {
const data = line.slice(6);
if (data === "[DONE]") continue;
try {
const json = JSON.parse(data);
const delta = json.choices?.[0]?.delta?.content;
if (delta) {
bubble.textContent += delta;
messages.scrollTop = messages.scrollHeight;
}
} catch (e) {
// Ignore malformed lines
}
}
}
}
});We use fetch with a ReadableStream rather than the EventSource API because EventSource only supports GET requests — and we need POST to send the message body.
Part 3: Python Alternative (FastAPI)
If you prefer Python, here is the equivalent server using FastAPI:
from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
import httpx
import os
import json
app = FastAPI()
QUBAX_API_URL = "https://api.qubax.ai/v1/chat/completions"
API_KEY = os.environ["QUBAX_API_KEY"]
@app.post("/api/chat")
async def chat(request: Request):
body = await request.json()
messages = body["messages"]
model = body.get("model", "gpt-5.6-luna")
async def stream():
async with httpx.AsyncClient() as client:
async with client.stream(
"POST",
QUBAX_API_URL,
json={
"model": model,
"messages": messages,
"stream": True,
"max_tokens": 2000,
},
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
},
timeout=120.0,
) as response:
async for line in response.aiter_lines():
if line.startswith("data: "):
data = line[6:]
if data == "[DONE]":
yield "event: done\ndata: {}\n\n"
break
yield f"data: {data}\n\n"
return StreamingResponse(
stream(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
"X-Accel-Buffering": "no",
},
)Run it with:
pip install fastapi uvicorn httpx
uvicorn server:app --reload --port 3000Production Checklist
Before shipping your streaming chatbot, make sure you handle these common issues:
Nginx Proxy Buffering
If you run behind nginx, it will buffer SSE responses by default, breaking streaming. Add this to your nginx config:
location /api/chat {
proxy_pass http://localhost:3000;
proxy_buffering off;
proxy_cache off;
proxy_set_header Connection '';
proxy_http_version 1.1;
chunked_transfer_encoding on;
}Connection Timeouts
Long conversations can exceed proxy or load-balancer timeout limits. Either:
- Set generous timeouts (120s+)
- Send periodic heartbeat comments (
: heartbeat\n\n) to keep the connection alive
Error Recovery on the Client
Handle mid-stream failures gracefully:
if (!response.ok) {
bubble.textContent = "Sorry, something went wrong. Please try again.";
return;
}Token Management
For multi-turn conversations, manage context window limits:
- Keep a rolling history of the last N messages
- Summarize older messages to compress context
- Set
max_tokensto prevent runaway responses
Model Selection
Different models have different cost/quality trade-offs:
| Model | Best For | Cost (per 1M tokens) |
|---|---|---|
| GPT-5.6 Luna | General chat, fast responses | See Qubax pricing |
| DeepSeek V4 Flash | Cost-sensitive high-volume | Budget-friendly |
| Claude Opus 5 | Complex reasoning, long context | Premium tier |
| GLM 5.2 | Ultra-cheap multilingual | Lowest cost |
Browse all available models and their real-time pricing at qubax.ai/models.
Testing Your Stream
Verify your endpoint works with curl before building the UI:
curl -N -X POST http://localhost:3000/api/chat \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":"Write a haiku about streaming"}]}'The -N flag disables curl's output buffering, so you will see tokens appear one at a time.
Wrapping Up
Streaming transforms the user experience of any AI-powered application. With SSE and the OpenAI-compatible API format, you can build a production-grade streaming chatbot in under 100 lines of code.
The patterns in this tutorial — SSE headers, chunk forwarding, error handling, and rate limiting — scale from a weekend project to a production system serving thousands of concurrent users.
Ready to build? Get your API key at [Qubax](https://qubax.ai/models) and access 200+ AI models with a single OpenAI-compatible endpoint. Check the [docs](https://qubax.ai/docs) for full API reference.
FAQ
What is the difference between SSE and WebSockets for AI streaming?
SSE is simpler, uses standard HTTP, and has built-in auto-reconnect. WebSockets are bidirectional and lower-overhead for high-frequency two-way communication. For AI text streaming (server→client only), SSE is the standard choice.
Why use fetch instead of EventSource for the client?
The browser's EventSource API only supports GET requests, but AI chat APIs require POST to send the message body. Using fetch with a ReadableStream gives you POST support while still parsing SSE format.
How do I handle connection drops during streaming?
On the client, catch fetch errors and offer a "retry" button. On the server, ensure all code paths call res.end() — even error paths — to prevent hanging connections. Consider implementing resumable streams with the last-received token ID.
Can I stream multiple models at the same time?
Yes. You can open parallel SSE connections to different models and merge the streams client-side. This is useful for A/B testing model quality or building "model routing" features.
How much does it cost to run a streaming chatbot?
Cost depends on the model and token volume. Budget models like GLM 5.2 cost fractions of a cent per conversation. Frontier models like GPT-5.6 Sol cost more but deliver higher quality. See Qubax pricing for details.
