Single-vector retrieval averages the sequence before search.
The index is compact and dot products are efficient, but one document vector must preserve every potentially useful token relationship.
wirelessbatterylifehoursmean
A token-level retrieval experiment
One vector can hide the exact match.
ColBERT stores contextual vectors for document tokens, then lets every query token find its own best match at search time. The late interaction is a small operation with a very different information bottleneck.
query tokens
wirelessbatterylife
document tokens
Bluetooth40chargehours
See every query token choose its best document token.
The vectors are transparent teaching vectors, not benchmark measurements. Their cosine similarities isolate mean pooling and ColBERT's MaxSim scoring without hiding the computation behind a model API.
weak similarity related row maximum
selected document score--
mean-pooled score--
query token maxima--
winner margin--
Pooling compresses early. Late interaction waits.
Neither representation is universally superior. The question is where the system can afford to keep token-level evidence and when that detail changes ranking quality.
MaxSim keeps separate evidence.
Each query token contributes its largest similarity against any document token. The document score is the sum of those maxima.
Fine-grained matching costs storage.
Documents retain multiple vectors instead of one. ColBERT systems use compression and retrieval engineering to make that tradeoff practical.
Score decomposition remains inspectable.
wireless0.86
battery0.92
life0.78
Interpret rankings carefully.
Late interaction can preserve a decisive rare-token match that mean pooling dilutes.
Token-level similarity is still learned representation, not proof that the document answers the query correctly.
A practical system may retrieve coarsely first, then apply late interaction to a smaller candidate set.
The late-interaction retrieval path.
ColBERT moves most language-model work to offline encoding, then performs a lightweight token similarity operation at search time.
Encode query and document tokens contextually.
A transformer produces one contextual embedding per token. Documents are encoded ahead of time; the incoming query is encoded at search time.
Find a maximum for every query token.
For each query embedding, compute similarities to document-token embeddings and retain the largest value. This is the MaxSim operation shown in the matrix.
Sum maxima and rank candidates.
The document score is the sum of per-query-token maxima. Retrieval systems then rank documents, often with compressed indexes and multi-stage search.
Use the representation that fits the retrieval stage.
Primary retrieval research.
Read the original ColBERT work, its compressed successor, and the engine that accelerates late-interaction search.
ColBERT: Efficient and Effective Passage SearchKhattab and Zaharia, SIGIR 2020ColBERTv2: Effective and Efficient Retrieval via Lightweight Late InteractionSanthanam et al., NAACL 2022PLAID: An Efficient Engine for Late Interaction RetrievalSanthanam et al., CIKM 2022BEIR: A Heterogeneous Benchmark for Zero-shot EvaluationThakur et al., NeurIPS Datasets and Benchmarks 2021