How to Build a Semantic Search Engine with Embeddings: A Complete Developer Tutorial

Keyword search fails exactly when you need it most. A user searches your docs for "how do I stop getting charged twice" and gets nothing — because the page that answers them is titled "Duplicate Billing Prevention." The words don't match; the meaning does.

Semantic search fixes this by searching with meaning instead of strings, and it has become one of the highest-ROI features a development team can ship: better docs, better search, better chatbots, better recommendations. In this tutorial, you'll build a production-shaped semantic search engine in about 150 lines of Python — embeddings, vector database, hybrid ranking, and all.

(This article is a hands-on companion to our explainer on embeddings — but everything you need is included below.)

What You're Building

By the end of this tutorial you will have:

1. A chunking pipeline that turns documents into embeddable pieces.
2. An embedding pipeline that converts chunks into vectors via an OpenAI-compatible API.
3. A vector index (pgvector — Postgres you already know) for fast similarity search.
4. Hybrid search: semantic + keyword results fused with Reciprocal Rank Fusion.
5. A small query API you can drop behind FastAPI.

Stack: Python 3.11+, psycopg, httpx, and any OpenAI-compatible embeddings endpoint (we'll use Qubax AI's, which works with any provider on the platform).

Prerequisites

# Project setup
python -m venv .venv && source .venv/bin/activate
pip install httpx psycopg pgvector numpy

# Postgres with the pgvector extension (Docker)
docker run -d --name vecdb -p 5432:5432 \
  -e POSTGRES_PASSWORD=dev \
  pgvector/pgvector:pg16

You'll need an API key from qubax.ai (start with the embeddings-capable models there) and the Qubax docs at qubax.ai/docs for endpoint specifics.

Step 1: Chunk Your Documents

Embedding models have a maximum input size (a token limit), and similarity degrades when you cram unrelated content into one vector. The standard approach: split documents into overlapping chunks of roughly 200–500 tokens.

# chunker.py
def chunk_text(text: str, max_chars: int = 1600, overlap: int = 200) -> list[str]:
    """Split text into overlapping chunks, respecting paragraph boundaries."""
    chunks = []
    start = 0
    while start < len(text):
        end = min(start + max_chars, len(text))
        # Try to break at a paragraph or sentence boundary
        if end < len(text):
            for sep in ("\n\n", ". ", " "):
                cut = text.rfind(sep, start, end)
                if cut > start + max_chars // 2:
                    end = cut + len(sep)
                    break
        chunks.append(text[start:end].strip())
        start = end - overlap
    return [c for c in chunks if c]

Why overlap? If an answer spans a chunk boundary, overlapping windows ensure at least one chunk contains the full idea. 10–15% overlap is a good default.

Step 2: Embed the Chunks

An embedding model maps text to a fixed-length vector (typically 768–3072 dimensions) where cosine similarity ≈ semantic similarity. We'll call an OpenAI-compatible endpoint so the same code works against any provider:

# embed.py
import httpx

API_BASE = "https://api.qubax.ai/v1"
API_KEY = "sk-..."

def embed(texts: list[str], model: str = "text-embedding-3-small") -> list[list[float]]:
    """Embed a batch of texts via an OpenAI-compatible endpoint."""
    resp = httpx.post(
        f"{API_BASE}/embeddings",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json={"model": model, "input": texts},
        timeout=60,
    )
    resp.raise_for_status()
    # Preserve input order — APIs return objects keyed by index
    data = sorted(resp.json()["data"], key=lambda d: d["index"])
    return [d["embedding"] for d in data]

Two rules that will save you hours:

• Never mix models. Vectors from different embedding models live in different coordinate systems — comparing them produces garbage. Pick one model per table and store its name in a column.
• Batch your requests. Embedding 1,000 chunks one at a time is slow and expensive; batches of 64–128 are the sweet spot.

Step 3: Store Vectors in Postgres with pgvector

You don't need a dedicated vector database for your first million vectors — Postgres plus the pgvector extension is battle-tested and keeps your data in one place.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE IF NOT EXISTS chunks (
    id         BIGSERIAL PRIMARY KEY,
    doc_id     TEXT NOT NULL,
    chunk_idx  INT  NOT NULL,
    content    TEXT NOT NULL,
    model      TEXT NOT NULL,          -- embedding model used!
    embedding  VECTOR(1536)            -- match your model's dimensionality
);

-- cosine index for fast approximate search
CREATE INDEX IF NOT EXISTS chunks_emb_idx
    ON chunks USING hnsw (embedding vector_cosine_ops);

And the Python side of the insert:

# store.py
import psycopg
from pgvector.psycopg import register_vector

def save_chunks(doc_id: str, chunks: list[str], vectors: list[list[float]], model: str):
    with psycopg.connect("postgresql://postgres:dev@localhost:5432/postgres") as conn:
        register_vector(conn)
        with conn.cursor() as cur:
            cur.executemany(
                """INSERT INTO chunks (doc_id, chunk_idx, content, model, embedding)
                   VALUES (%s, %s, %s, %s, %s)""",
                [(doc_id, i, c, model, v) for i, (c, v) in enumerate(zip(chunks, vectors))],
            )
        conn.commit()

Step 4: Semantic Search in One Query

With data loaded, searching is a single SQL statement. pgvector computes cosine similarity between your query vector and every stored chunk (using the HNSW index to stay fast at scale):

# search.py — pure semantic search
def semantic_search(query: str, cur, top_k: int = 5):
    qvec = embed([query])[0]
    cur.execute(
        """SELECT content, doc_id,
                  1 - (embedding <=> %s::vector) AS score
           FROM chunks
           WHERE model = %s
           ORDER BY embedding <=> %s::vector
           LIMIT %s""",
        (qvec, MODEL, qvec, top_k),
    )
    return cur.fetchall()

The <=> operator is cosine distance; 1 - distance gives a similarity score where 1.0 is identical meaning. The query "how do I stop getting charged twice" will now surface "Duplicate Billing Prevention" — mission accomplished.

Step 5: Upgrade to Hybrid Search with RRF

Pure semantic search has a weakness: it can miss exact matches — error codes, function names, SKUs. Classic keyword search (Postgres full-text) is great at those. The industry-standard fix is to run both and fuse the rankings with Reciprocal Rank Fusion:

# search.py — hybrid semantic + keyword with Reciprocal Rank Fusion
def hybrid_search(query: str, cur, top_k: int = 5, k: int = 60):
    # Two independent result lists
    semantic = semantic_search(query, cur, top_k=20)
    keyword = keyword_search(query, cur, top_k=20)   # via ts_rank + to_tsquery

    # Fuse: score = sum over lists of 1 / (k + rank)
    scores: dict[int, float] = {}
    rows: dict[int, tuple] = {}
    for results in (semantic, keyword):
        for rank, row in enumerate(results):
            key = hash(row[0])  # in production, dedupe on a stable content hash
            scores[key] = scores.get(key, 0.0) + 1.0 / (k + rank + 1)
            rows[key] = row

    ranked = sorted(scores.items(), key=lambda x: -x[1])
    return [rows[key] for key, _ in ranked[:top_k]]

RRF needs no score normalization — it only uses ranks — which is why it became the default fusion method. Documents that appear high in both lists shoot to the top. Expect a measurable lift over either method alone.

Step 6: Wrap It in an API

# app.py
from fastapi import FastAPI
import psycopg

app = FastAPI()

@app.get("/search")
def search(q: str, top_k: int = 5):
    with psycopg.connect(DB_URL) as conn, conn.cursor() as cur:
        return {"results": [
            {"content": r[0], "doc_id": r[1], "score": round(float(r[2]), 4)}
            for r in hybrid_search(q, cur, top_k)
        ]}

Production Checklist

Before you ship, address these — each one bites in production:

• Re-embedding on model change. Locked by the never-mix-models rule: when you switch embedding models, re-embed everything and version the index (that's why we stored model per row).
• Normalization. Some providers return normalized vectors, some don't. Normalize on write (v / |v|) so cosine similarity behaves consistently.
• Dimension mismatch errors. VECTOR(1536) must equal your model's output dimension exactly; check the model card before creating the table.
• Chunk metadata. Store titles, URLs, and timestamps alongside chunks — you'll need them for citation and staleness handling.
• Evaluation. Keep a held-out set of (query → expected doc) pairs and measure recall@5 before and after every change. Hybrid search should win; verify it does on your data.
• Cost control. Embeddings are cheap but not free — cache by content hash so re-indexing an unchanged doc costs nothing.

FAQ

What are embeddings in AI?

Embeddings are numeric vectors that represent the meaning of text. Similar meanings map to nearby points in vector space, so "cancel my subscription" and "end my plan" land close together even though they share no words. See our full explainer on embeddings for a deeper dive.

Do I need a dedicated vector database?

Not at first. pgvector handles millions of vectors comfortably inside ordinary Postgres, with HNSW indexes for approximate nearest-neighbor search. Reach for dedicated vector stores (or pgvector's newer versions with quantization) when you're into the tens of millions of vectors or need sub-10ms p99.

What embedding model should I use?

Any modern 768–3072 dimension text embedding model works with this tutorial — just pick one model and stay consistent across your whole index. Browse available models and pricing at qubax.ai/models.

Why hybrid search instead of pure semantic search?

Semantic search misses exact-match queries (error codes, identifiers, function names), while keyword search misses meaning. Fusing both with Reciprocal Rank Fusion (RRF) consistently outperforms either alone and costs almost nothing extra to compute.

How much does it cost to embed a documentation site?

Rule of thumb: 1 million tokens of input embeds for a few cents to a few dollars depending on the model — a 500-page docs site typically costs well under $1 to index end-to-end. The Qubax platform discounts many models below retail; check current pricing at qubax.ai/models.

Can I use this for RAG?

Yes — this is the retrieval half of retrieval-augmented generation. Take the top chunks from hybrid_search, drop them into a prompt for any chat model via the same API, and you have a RAG pipeline. The Qubax docs cover the chat-completion side.