Temperature Sampling

Definition

Temperature sampling is a technique used in AI language models to control the randomness and diversity of generated output. It is one of the most fundamental parameters governing how a model selects the next token (word or sub-word unit) at each step of generation. **How it works:** At each generation step, an LLM produces a probability distribution over its entire vocabulary — essentially a ranked list of all possible next tokens with associated probabilities. Temperature is a scalar value that reshapes this distribution before the model samples from it: - **Temperature = 1.0** (default): The model samples from the raw probability distribution. Common tokens are favored, but lower-probability options still have a meaningful chance of being selected. - **Temperature < 1.0** (e.g., 0.2)**: The distribution is sharpened — high-probability tokens become even more dominant. Output becomes more deterministic, focused, and predictable. Useful for factual tasks like transcription accuracy checks or structured data extraction. - **Temperature > 1.0** (e.g., 1.5)**: The distribution is flattened — lower-probability tokens get a larger relative chance. Output becomes more diverse, creative, and unpredictable. Useful for brainstorming or creative writing, but increases hallucination risk. - **Temperature = 0**: The model always picks the single highest-probability token (greedy decoding). Output is fully deterministic given the same input. Temperature is often used in combination with **top-p sampling** (nucleus sampling) to further refine the token selection pool before sampling occurs. **Relevance to video AI tools:** In video repurposing platforms like OpenClip, temperature settings influence how the underlying language models analyze transcripts, score clip candidates, and generate any descriptive output. Lower temperatures are generally preferred for tasks requiring factual accuracy (e.g., identifying exact quote boundaries, extracting speaker segments), while moderate temperatures may be appropriate for scoring 'viral potential' where some creative judgment is beneficial. Understanding temperature helps creators evaluate why AI tools may produce varying results on identical inputs when temperature is set above zero.