Definition
A vector database is a system that stores embeddings — the fixed-length numeric vectors a model produces for text, images, or other items — alongside their metadata, builds an index over them, and answers "which stored vectors are nearest to this query vector?" quickly, at scale, and with filters. It is the piece of infrastructure that sits behind semantic search boxes, retrieval-augmented generation, and recommendation candidate-generation.
The distinction worth getting right on arrival is the one between this page and vector search. Vector search is the algorithm: given a query vector and a collection, return the k nearest neighbours under a distance metric. A vector database is the system that makes running that algorithm practical on real data — it stores the vectors durably, builds and maintains the approximate-nearest-neighbour index, attaches and filters on metadata ("nearest, but only in-stock items from this seller"), absorbs inserts and deletes, shards across machines when one is not enough, and serves many concurrent queries. Vector search is one function call inside it. A useful analogy: nearest-neighbour search is to a vector database what a sort algorithm is to a relational database. You could sort a list yourself; the database is what you reach for when the list is a hundred million rows, changes constantly, and has to be queried by ten services at once. Everything specific to a vector database — the memory it needs, the recall it trades away, the way filters interact with the index — follows from packaging an approximate index into a durable, queryable, updatable store.
How It Works
A query into a vector database follows a short path, and each step is a design decision the system has made on your behalf.
The item you search over — a document, a product, a face — is first turned into an embedding by a model. That embedding, plus a payload of metadata (an ID, a category, a timestamp, a price), is what the database stores. Storage is two things at once: the raw vectors, kept resident in memory for a fast index or on disk for a large one, and the metadata, kept in a structure you can filter on. Keeping these two consistent as data changes is a large part of what the database does that a bare algorithm does not.
Over the stored vectors the database builds an index, and the choice of index is the single most consequential one it makes. An exhaustive, brute-force scan is exact but linear: one distance computation per stored vector. A collection of ten million 768-dimensional vectors costs 10,000,000 × 768 ≈ 7.68 billion multiply-add operations for one query — far too slow for an interactive search. So the database builds an approximate-nearest-neighbour (ANN) index that touches only a small fraction of the vectors per query, giving up a guarantee of the true nearest neighbours in exchange for orders-of-magnitude less work. The dominant such index, HNSW (Hierarchical Navigable Small World graphs, from Malkov and Yashunin's 2016 paper), links each vector to a few near neighbours in a layered proximity graph; its abstract's headline claim, that the layered structure "allows a logarithmic complexity scaling," is why a query walks the graph in on the order of log₂(10,000,000) ≈ 23 hops of local work rather than scanning ten million vectors. In round terms, an ANN index turns a query from billions of multiply-adds into a few thousand distance computations — roughly a thousandfold less arithmetic — and that ratio is the entire reason vector databases exist as a category.
Memory is the cost that usually binds first, and it is worth doing the arithmetic because it decides what hardware you need. Those ten million 768-dimensional vectors, stored as 32-bit floats, occupy 10,000,000 × 768 × 4 bytes ≈ 30.7 GB — and that is before the index. For a float HNSW index the graph adds its links on top: with the paper's connection parameter M set to 16, the base layer holds about 2M = 32 neighbour IDs per vector at 4 bytes each, roughly 128 bytes per vector, so the graph contributes only about 10,000,000 × 128 bytes ≈ 1.3 GB. The lesson from that comparison is that for a float index the vectors dominate the footprint, not the graph — around 32 GB total, which does not fit on a single 24 GB GPU but sits comfortably in a large machine's RAM. This is exactly the number that pushes people toward compression, covered under Types below.
At query time the database walks the index to a candidate set, applies any metadata filter, ranks the survivors by the true distance, and returns the top k with their payloads. When one machine's RAM or throughput is not enough, it shards the collection across nodes — each node holds a slice, every query fans out to all of them, and their top-k lists are merged — and replicates shards for availability and read throughput. Inserts, deletes, and re-indexing are handled continuously rather than as a rebuild-from-scratch, which is where much of the engineering effort of a production system actually goes.
Types
Vector databases are distinguished less by their branding than by the ANN index family they build, and there is a real, named typology here — practitioners choose among these by name, and each represents a different point on the recall–latency–memory triangle. Most systems offer more than one and let you pick per collection.
- Graph indexes (HNSW). Each vector is a node linked to its near neighbours in a multi-layer graph; a query enters at the sparse top layer and greedily descends toward the query point. HNSW is the default for high-recall, in-memory search and the index most vector databases reach for out of the box. Its cost is memory: the vectors and the graph links stay resident in RAM. Two knobs govern it — the build-time connection count M (the paper notes a "reasonable range of M is from 5 to 48"), which trades index size and build time for reachability, and the query-time effort
ef, which trades latency for recall. - Inverted-file / partition indexes (IVF). The corpus is clustered into cells around centroids; at query time the database scores only the
nprobecells nearest the query rather than the whole collection. IVF uses far less memory than a graph and builds faster, at somewhat lower recall for the same speed — its natural home is very large collections where a graph's per-vector link overhead is unaffordable. - Compression via product quantization (PQ). Rather than a different way to navigate the vectors, PQ is a different way to store them. It splits each D-dimensional vector into m sub-vectors and replaces each sub-vector with the index of its nearest centroid in a small per-subspace codebook — typically 256 centroids, so 8 bits, one byte, per sub-vector. A 768-dimensional float32 vector of
768 × 4 = 3072bytes becomes anm = 96-byte code: a 32-fold reduction, taking those ten million vectors from ~30.7 GB down to under 1 GB. Distances are then estimated from the codes. PQ (from Jégou, Douze, and Schmid's 2011 work) is what lets billion-scale collections fit on a single machine, and its cost is recall — the codes are lossy, so systems usually re-rank a PQ shortlist with exact distances to recover accuracy. - Composite indexes (IVF-PQ, HNSW-PQ). The families combine: partition the space with IVF to narrow the search, and store the vectors as PQ codes to fit them in memory. IVF-PQ is the standard recipe for the largest deployments, precisely because it economises on both the vectors scanned and the bytes each one costs.
The choice among these is not a preference; it is dictated by where your collection sits on the triangle. Ten million vectors that must answer at 99% recall in a millisecond want HNSW and enough RAM. A billion vectors on a budget want IVF-PQ and will accept a re-ranking step.
Real-World Applications
The application that pulled vector databases into mainstream infrastructure is the retrieval stage of retrieval-augmented generation. Before a language model answers, the system embeds the question, asks the vector database for the passages whose embeddings are nearest, and hands those passages to the model as grounding context. The upstream step of splitting source documents into passages before embedding them — chunking — determines what the database can ever retrieve, and a reranking model is often placed after it to re-order the database's candidates with a slower, more accurate scorer; the vector database is the fast, high-recall first stage that these two bracket. This is why a vector store became standard equipment for documentation chatbots, internal search, and support assistants, and increasingly for agent-memory — where an agent writes its past observations into a vector store and retrieves the relevant ones later as a long-term memory it can search by meaning.
Recommendation is the older and larger deployment. "Find items similar to this one" and "find users near this user" are nearest-neighbour queries over learned embeddings, and large recommenders at streaming and e-commerce companies run ANN indexes over hundreds of millions of item vectors to produce candidate lists in milliseconds, which a heavier ranking model then orders. The same primitive powers reverse-image and visual-similarity search, audio and song matching, and near-duplicate detection, where "the same" means two vectors sitting within a small distance of each other.
As concrete systems, the category spans a library, a database extension, and managed services. FAISS, Meta's open-source similarity-search library, is the toolkit many of the others build on — it implements the index families above but leaves storage, filtering, and serving to you. pgvector adds vector columns and ANN indexing to PostgreSQL, keeping vectors beside relational data in a database teams already run. Dedicated systems — among them Milvus, Qdrant, Weaviate, Pinecone, and Chroma — package storage, indexing, metadata filtering, sharding, and a query API into a standalone service. Which one performs best on a given workload is a moving target measured in perishable benchmarks and not worth memorising; what is durable is the index families they all implement and the tradeoffs those families impose, which is what this page is about.
Key Concepts
The concept that separates a vector database from a nearest-neighbour library is metadata filtering, and it is subtler than it looks. Real queries are rarely "nearest neighbours" alone; they are "nearest neighbours that are in stock, priced under fifty, and owned by this tenant." The naive implementation, post-filtering, retrieves the top-N by similarity and then discards the ones that fail the filter — and it breaks on selective filters. Work the numbers: on a 10-million-vector corpus, if the filter matches 1% of items, retrieving the top-1000 nearest neighbours leaves you 1000 × 0.01 = 10 expected survivors, right at the edge of a k=10 request. If the filter matches 0.1%, the same top-1000 yields one expected survivor, and you return an almost-empty result set. The alternative, pre-filtering, restricts the search to matching IDs first, but that fights the index: an HNSW graph's edges may all point to filtered-out vectors, so the greedy walk stalls or recall collapses, and the fallback is a brute-force scan over the filtered subset. Production databases implement filtered search that pushes the predicate into the traversal — visiting the graph but only counting matching nodes toward the result — precisely because neither naive approach survives a selective filter. When a filtered similarity query returns fewer results than expected, this interaction, not the data, is usually the cause.
The second load-bearing concept is recall@k: the fraction of the true top-k neighbours the approximate index actually returns. A recall@10 of 0.95 means the index found, on average, 9.5 of the 10 items an exhaustive scan would have. It is the number that makes "approximate" concrete, and it is a quantity to monitor in production against an exact scan on a sample of queries — not a spec quoted once at launch — because it moves when the data distribution shifts or the index is rebuilt.
Challenges
Every real decision in a vector database is a point chosen on a recall–latency–memory triangle, and you cannot have all three corners. Push recall toward 1.0 by raising the search-effort knob ef and latency rises with it, because the query visits more of the graph. Cut memory with product quantization and recall falls, because the stored codes are lossy. Cut latency by scanning fewer partitions and recall falls again. The exact-scan baseline anchors the trade: it costs the full N × D — 7.68 billion multiply-adds per query on the ten-million-vector example — and delivers perfect recall at latency nothing interactive can afford. Everything a vector database offers is a way of buying back most of that recall for a small fraction of the work, and the only way to know how much recall you actually bought is to measure it, per collection, at your chosen settings.
Freshness and deletes are the operational trap. Graph indexes accept incremental inserts cheaply, so a growing collection is easy. Removing a vector cleanly is not: the graph edges that pointed at it have to be repaired, so most systems tombstone deleted vectors and skip them at query time, rebuilding the index periodically to reclaim the space. A collection with a high churn rate silently accumulates dead vectors between rebuilds — which inflate memory and slow traversal — so the rebuild cadence is a real capacity-planning parameter, not an afterthought.
Choosing the wrong tool is the most common and most expensive mistake, and it points in both directions. Reaching for a distributed vector database to serve two hundred thousand vectors adds an operational system that a pgvector column or an in-process FAISS index would have handled with less to run and no data-synchronisation problem. Conversely, bolting millions of vectors onto a relational database that filters them by post-filtering will produce the empty-result-set failure above and no obvious error. The honest question is not "which vector database is fastest" but "does my scale and query pattern justify a dedicated one at all" — and for many applications, sitting on the database you already operate is the right answer until the collection or the query rate forces the move.
Code Example
The example builds a float HNSW-style footprint against a product-quantized one over the same ten million vectors, printing the memory each needs. It is the recall–memory corner of the triangle made concrete: PQ is what turns a 30 GB index into a 1 GB one, and the code shows exactly where the reduction comes from.
N = 10_000_000 # vectors
D = 768 # embedding dimension
FLOAT_BYTES = 4 # float32
# Raw float vectors: the dominant cost of an in-memory float index.
raw_vectors = N * D * FLOAT_BYTES
# HNSW graph links: base layer holds ~2M neighbour IDs at 4 bytes each (M=16).
M = 16
graph_links = N * (2 * M) * 4
float_index = raw_vectors + graph_links
# Product quantization: split each vector into m sub-vectors, store each as a
# 1-byte code (256-centroid codebook). Here D=768 split into m=96 sub-vectors.
m = 96
pq_codes = N * m # one byte per sub-vector
compression = (D * FLOAT_BYTES) / m
for label, nbytes in [("raw float vectors", raw_vectors),
("+ HNSW graph links", float_index),
("PQ codes (m=96)", pq_codes)]:
print(f"{label:22s} {nbytes / 1e9:6.2f} GB")
print(f"per-vector: 3072 B float -> {m} B code = {compression:.0f}x smaller")
Running it prints:
raw float vectors 30.72 GB
+ HNSW graph links 32.00 GB
PQ codes (m=96) 0.96 GB
per-vector: 3072 B float -> 96 B code = 32x smaller
Two things fall out of the output. First, the graph links add only 1.28 GB on top of the 30.72 GB of raw vectors — a small fraction of the total, confirming that for a float HNSW index you are, to first order, paying for the vectors themselves and not the index structure over them. Second, product quantization collapses those vectors from tens of gigabytes to under one, which is the whole reason billion-scale collections fit on a single machine; the code shows the mechanism is nothing more exotic than replacing a 3072-byte vector with a 96-byte code. What the arithmetic cannot show is the price: those codes are lossy, so a PQ index recovers accuracy by re-ranking its shortlist with the exact vectors, and its recall must be measured, not assumed.