Partial sequences retained after pruning.
Greedy decoding commits early, beam search keeps several hypotheses alive, and nucleus sampling draws from a dynamic probability mass. Change the controls and watch the same token tree produce different text paths.
Open decoding treeThe most likely token now does not guarantee the most likely complete sequence. Search strategies trade computation, diversity, and global sequence score. Sampling adds controlled randomness; beam search systematically prunes; greedy decoding spends one path.
Partial sequences retained after pruning.
Accumulated token log-probability with length normalization.
Candidate next tokens scored across active branches.
Fraction of retained hypotheses with distinct prefixes.
Select the highest-probability token at every step. It is fast and deterministic but cannot recover when an early local choice blocks a stronger full sequence.
yₜ = argmaxᵥ p(v | y<t, x)Expand every retained hypothesis, add token log-probabilities, and keep the best B candidates. Length penalties reduce the bias toward prematurely short sequences.
score(y) = log p(y|x) / length(y)^αSort tokens by probability and keep the smallest set whose cumulative mass reaches p. High-entropy contexts admit more tokens; confident contexts admit fewer.
V(p) = min set where Σᵥ∈V p(v) ≥ pSearch more sequences but multiply memory and compute. Larger beams can favor generic, high-probability continuations.
More search is not always better text.Flattened logits make lower-probability tokens more competitive, increasing diversity and error risk.
Temperature changes probabilities before truncation.A small nucleus excludes the long tail and behaves more deterministically when probability is concentrated.
The candidate count changes at every step.