Temperature

Example

Same logits, three temperatures. Then 1000 samples at each temperature (fixed seed) to see how often each token is picked:

import math, random
from collections import Counter

tokens = ["on", "delayed", "delivered", "cancelled", "banana"]
logits = [3.1, 2.5, 2.4, 2.2, -1.0]

def softmax_t(scores, t):
    m = max(scores)
    exps = [math.exp((s - m) / t) for s in scores]
    z = sum(exps)
    return [e / z for e in exps]

for t in (0.2, 1.0, 2.0):
    probs = softmax_t(logits, t)
    shown = "  ".join(f"{tok}={pr:.2f}" for tok, pr in zip(tokens, probs))
    rng = random.Random(42)
    counts = Counter(rng.choices(tokens, weights=probs, k=1000))
    print(f"T={t}: {shown}")
    print(f"       1000 samples: {dict((tok, counts[tok]) for tok in tokens)}")

Output

T=0.2: on=0.92  delayed=0.05  delivered=0.03  cancelled=0.01  banana=0.00
       1000 samples: {'on': 911, 'delayed': 43, 'delivered': 31, 'cancelled': 15, 'banana': 0}
T=1.0: on=0.41  delayed=0.22  delivered=0.20  cancelled=0.16  banana=0.01
       1000 samples: {'on': 380, 'delayed': 228, 'delivered': 206, 'cancelled': 174, 'banana': 12}
T=2.0: on=0.31  delayed=0.23  delivered=0.22  cancelled=0.20  banana=0.04
       1000 samples: {'on': 308, 'delayed': 212, 'delivered': 236, 'cancelled': 195, 'banana': 49}