Semantic Search

Semantic search finds results by meaning, not exact keywords: it embeds the query and documents as vectors and returns the nearest ones.

Published Updated

On this page

Definition

Semantic search finds results by meaning rather than by exact words. It turns your query and every document into a list of numbers — an embedding vector — and returns the documents whose vectors sit closest to the query's. That is why a search for "how to fix a flat tyre" can surface a guide titled "repairing a punctured wheel" even though the two share no words: the phrases mean nearly the same thing, so an embedding model places their vectors near each other.

This is the opposite of traditional keyword search, also called lexical search, which matches literal strings. Lexical search would score "repairing a punctured wheel" as a poor match for "flat tyre" because none of the query words appear in it, and it would rank a page that merely repeats the word "tyre" many times above a page that actually explains the repair. Semantic search inverts that: word overlap is irrelevant, closeness of meaning is everything. The shift is from "find documents that contain these words" to "find documents that are about what I asked."

How It Works

The whole method rests on one idea from information retrieval: if you can place text at points in a high-dimensional space so that similar meanings land near each other, then "search" becomes "find the nearest points to the query point." Three steps make that concrete.

First, embed everything once, ahead of time. An embedding model reads each document and outputs a fixed-length vector. The length is a convention of the model — commonly somewhere between 384 and 1536 dimensions. A 768-dimensional vector stored as 32-bit floats takes 768 × 4 = 3,072 bytes, about 3 KB; a 1536-dimensional vector takes about 6 KB. These vectors are computed once and saved, so indexing a million documents is a one-time cost, not something you repeat per query.

Second, at query time, embed the query with the same model into the same space, then measure closeness. The standard measure is cosine similarity — the cosine of the angle between two vectors — which ranges from 1 (pointing the same way, nearly identical meaning) through 0 (unrelated) to −1 (opposite). Cosine ignores vector length and looks only at direction, which is why a short query and a long document can still score as a strong match. Computing one cosine score is a dot product over the dimensions: for 768-dimensional vectors, that is 768 multiply-add operations per document.

Third, rank by that score and return the top few. The catch is scale. Comparing the query against every document — an exact, brute-force scan — costs one dot product per document. Over 10 million documents at 768 dimensions that is 7.68 billion multiply-add operations for a single query, which is too slow for an interactive search box. This is why real systems use approximate nearest-neighbour (ANN) indexes such as HNSW, which trade a small, tunable amount of accuracy for a query cost that grows roughly with the logarithm of the collection size: on the order of log₂(10,000,000) ≈ 23 rather than 10 million comparisons. That index — the data structure, the distance metric, the memory layout — is the subject of vector search; semantic search is the idea it serves.

An embedding model is what makes the meaning land in the geometry. It is trained so that paraphrases, synonyms and translations end up close together, which is where the "semantic" comes from — see semantic understanding for how models represent meaning in the first place.

Real-World Applications

The clearest use today is the retrieval stage of retrieval-augmented generation: before a language model answers a question, semantic search pulls the handful of passages most relevant to it and hands them to the model as context. Because users phrase questions in their own words rather than the document's, meaning-based retrieval matters more here than exact-word matching — this is the reason vector databases became standard infrastructure for AI assistants.

Enterprise and product documentation search is another: a support system where a customer types "the app keeps logging me out" needs to reach a help article titled "resolving session expiry errors," which shares no keywords with the query. E-commerce search uses the same trick so that "warm jacket for hiking" surfaces insulated shells whose product copy never used those exact words. In each case the value is specifically that the query and the target text describe the same thing differently, which is exactly what lexical search cannot bridge and semantic search is built for.

Key Concepts

  • Lexical (keyword) search: the baseline that matches literal strings, typically with an algorithm like BM25. It is precise for exact terms and blind to synonyms — the complement of semantic search, not merely an inferior version of it.
  • Cosine similarity: the direction-based closeness score most semantic search uses; length-independent, so a short query matches a long passage on meaning alone.
  • Approximate nearest-neighbour (ANN) search: the reason semantic search stays fast at scale, accepting a slightly imperfect nearest-neighbour list in exchange for far fewer comparisons.
  • Hybrid search: running lexical and semantic search together and merging their scores, so exact identifiers and meaning-based matches are both covered.
  • Reranking: an optional second pass where a slower, more accurate model reorders the top candidates that the fast vector search returned.

Challenges

The defining failure mode is that semantic search always returns something. Nearest-neighbour search reports the closest vectors that exist, and there is no built-in signal for "nothing here is actually relevant." Lexical search has one — if no document contains the query terms, it returns nothing, and that emptiness is informative. Semantic search instead hands back the least-distant documents with confident-looking similarity scores even when the true answer is absent, which is how a retrieval system feeds a plausible-but-wrong passage into a downstream model and produces a fluent, sourced, incorrect answer. Setting a similarity threshold below which results are discarded is a partial defence, but thresholds are hard to tune because the score scale shifts with the model and the domain.

The second failure mode is the mirror image: semantic search is bad at exact identifiers. A part number like MJ-4471, an error code 0x80070005, a specific case citation or a function name carries almost no distributed "meaning" for an embedding model to place, so the query vector for 0x80070005 may sit closer to unrelated hexadecimal-looking text than to the one document that resolves that exact code. Lexical search gets these right instantly. This is the single most common reason teams are disappointed by a pure semantic system, and it is why hybrid search exists — it lets the lexical side catch the exact tokens while the semantic side catches the paraphrases, and neither alone is enough.

Beyond those, quality depends heavily on the embedding model matching the domain: a model trained on general web text can misjudge closeness in medical, legal or code corpora, where the same word means something specific. And long documents have to be split into passages before embedding, because a single vector for a 20-page document averages away the one paragraph that answers the query — how you chunk the text quietly determines what the system can ever retrieve.

Code Example

The ranking step is the concrete heart of semantic search. In a real system the vectors come from an embedding model; here they are placed by hand on three interpretable "meaning axes" so the result is reproducible without downloading a model. The query "how to fix a flat tyre" ranks a same-meaning, no-shared-words document essentially level with a word-sharing one, and pushes unrelated topics far down:

import numpy as np

# Real vectors come from an embedding model. These are hand-placed on
# three meaning axes so the ranking is reproducible without a model:
#            [ vehicle-repair, cooking, finance ]
docs = {
    "Repairing a punctured wheel":       np.array([0.95, 0.02, 0.05]),
    "Changing a car tyre on the road":   np.array([0.90, 0.05, 0.10]),
    "The best slow-cooked pasta sauce":  np.array([0.03, 0.97, 0.02]),
    "How compound interest grows money": np.array([0.04, 0.03, 0.96]),
}
query = np.array([0.92, 0.04, 0.08])   # "how to fix a flat tyre"

def cosine(a, b):
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

ranked = sorted(docs.items(), key=lambda kv: cosine(query, kv[1]), reverse=True)
for title, vec in ranked:
    print(f"{cosine(query, vec):.3f}  {title}")

Running this prints:

1.000  Changing a car tyre on the road
0.999  Repairing a punctured wheel
0.129  How compound interest grows money
0.076  The best slow-cooked pasta sauce

Notice that "Repairing a punctured wheel" scores 0.999 despite sharing no words with the query, while the two cooking and finance documents fall to 0.076 and 0.129 — the ranking follows meaning, not word overlap. Swap the hand-placed vectors for the output of a real embedding model over hundreds of dimensions, add an approximate nearest-neighbour index so the final line does not scan every document, and this is a working semantic search engine.

Frequently Asked Questions

Keyword (lexical) search matches the literal strings in your query against the strings in documents. Semantic search matches meaning: it turns text into vectors and returns documents whose vectors are closest, so a query can match a document that shares no words with it.
Usually not. Semantic search misses exact identifiers — a part number, an error code, a case citation — that lexical search nails, so most production systems run both and merge the results. That combination is called hybrid search.
It depends on the embedding model, but common sizes run from about 384 to 1536 dimensions. Larger vectors can capture more nuance but cost more memory and compute; the number is a convention of the model you choose, not a fixed rule.
Nearest-neighbour search always returns the closest vectors, even when nothing in the collection is truly relevant. There is no exact-match signal to say 'not found', so a plausible-but-wrong document can come back with a confident-looking similarity score.
Vector search is the index mechanism that finds nearest vectors quickly; semantic search is the idea of retrieving by meaning, which is normally implemented with vector search. Retrieval-augmented generation then feeds those retrieved documents to a language model as context.

Related Terms

Vector Search

Vector search returns the stored vectors nearest to a query vector by a distance metric — the retrieval primitive under semantic search, RAG and recommenders.

Embedding

What an embedding vector actually is, why cosine similarity beats Euclidean distance, what dimensions buy you, and where the geometry misleads you.

Information Retrieval (IR)

Information retrieval ranks documents against a query. How inverted indexes, TF-IDF and BM25 score, and how precision, recall, MAP and nDCG measure it.

Retrieval-Augmented Generation (RAG)

A technique that retrieves relevant documents at query time and feeds them into a language model's prompt, so it answers from current, external facts.

Semantic Understanding

Whether an AI grasps meaning well enough to act correctly on novel inputs, or is pattern-matching that breaks on rephrasing — and why that line is contested.

Continue Learning

Explore our use-case guides and prompts to deepen your AI knowledge.

Browse AI Tools← Back to Glossary