Negative log probability assigned to each true positive.
Contrastive learning turns pairs into geometry. Pull two views of the same item together, push alternatives away, and see how temperature, hard negatives, and batch size reshape the InfoNCE objective.
Open embedding spaceA positive pair says two observations should share a representation. Negatives define what the representation must distinguish. The encoder learns useful invariances only when augmentations preserve meaning and the comparison set is challenging without containing false negatives.
Negative log probability assigned to each true positive.
Mean similarity between paired augmented views.
Most confusable nonmatching item in the batch.
Anchors whose nearest neighbor is their positive.
L2 normalization puts embeddings on a unit hypersphere, making the dot product equal cosine similarity. Magnitude cannot cheaply satisfy the objective.
sim(zᵢ,zⱼ) = (zᵢ · zⱼ) / (‖zᵢ‖ ‖zⱼ‖)Temperature divides every logit. Low temperature sharpens the softmax and concentrates gradient on near misses; high temperature spreads credit across more negatives.
Lᵢ = −log exp(sim(i,i⁺)/τ) / Σₖ exp(sim(i,k)/τ)Hard negatives teach fine distinctions but may be false negatives that share the anchor's meaning. Strong augmentations can also destroy class identity, turning a supposed positive into contradictory supervision.
useful pair = invariant signal preserved + nuisance changedVery low temperature makes a few hard negatives dominate. Gradients become sharp and mislabeled negatives become dangerous.
Watch for unstable loss spikes.Random negatives that are already far apart contribute little learning signal, even if the numeric loss looks comfortable.
Increase batch diversity or mine negatives.If every point maps to the same representation, positives are close but indistinguishable from negatives. Contrastive denominators penalize this.
Track uniformity as well as alignment.