NeuralPath — AI Engineer Mastery Terminal

At scale, vector DB choice and tuning is the difference between $1K/mo and $50K/mo. The algorithms that matter:

HNSW — Hierarchical Navigable Small World

Multi-layer graph; search descends from coarse top layer to fine bottom layer.
Tunable: M (connections per node), ef_construction (build time), ef_search (query time).
High recall at moderate latency. Default for most modern vector DBs.
Memory-heavy: graph + vectors all in RAM.
Scale ceiling: ~100M vectors per node before memory becomes problematic.

Cluster vectors (k-means) into nlist buckets; search only nprobe nearest buckets.
Lower recall than HNSW for same speed; lower memory.
Pairs with PQ (Product Quantization) to compress vectors 8-32×.
IVF-PQ is the workhorse for billion-scale on commodity hardware.

Microsoft, designed for SSD: graph index that fits on disk, navigated efficiently.
Pinecone serverless uses this internally.
Scales to billions on a single node.

Strong recall/speed trade-off; used in Vertex AI Matching Engine.
Anisotropic quantization — quantize differently along different directions of the embedding distribution.

Scale	Recall need	Memory budget	Pick
< 10M	High	Generous	HNSW
10-100M	High	Tight	HNSW (tuned) or IVF-HNSW hybrid
100M-1B	Medium-high	Tight	IVF-PQ or DiskANN
> 1B	Medium	Disk-bound	DiskANN, ScaNN

Two knobs that matter:

The recall/latency curve: plot ef_search from 32 to 512; pick the point that meets your SLO.

Pre-filter vs post-filter is the killer question.

Post-filter: retrieve top-K, then drop those that fail the filter. If filter is selective (e.g., 1 tenant out of 1000), you'll need K very large to get any hits — slow.
Pre-filter: filter first, then ANN within the subset. Most modern DBs (Qdrant, Weaviate, pgvector with newer extensions) do this efficiently.

Always test filter performance at expected selectivity.