{
  "id": 6634356,
  "title": "LLM Sampling, Demystified: Temperature, Top-k, Top-p, Min-p and Repetition Penalty",
  "url": "https://urgent.news/2026/09/10/llm-sampling-demystified-temperature-top-k-top-p-min-p-and-repetition",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-10T20:02:06.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/shrsv/llm-sampling-demystified-temperature-top-k-top-p-min-p-and-repetition-penalty-4pkh"
  },
  "original_language": "en",
  "account": "LLM Sampling, Demystified: Temperature, Top-k, Top-p, Min-p and Repetition Penalty\n\nA language model does not generate the next token directly. Instead, it creates a probability distribution over the vocabulary, indicating how plausible each word is to follow the given context. The model's task is to interpret this distribution and choose the most likely next token.\n\nDecoding strategies determine how the model converts the probability distribution into an actual sequence of tokens. Greedy decoding selects the token with the highest probability at each step, which can result in repetitive or bland text. Unrestricted sampling, where tokens are chosen directly from the distribution, can lead to nonsensical outputs. Carefully truncated sampling tends to produce text more similar to human writing.\n\nThe key to controlling the model's output lies in how you sample from the probability distribution. This is where parameters like temperature, top-k, top-p, min-p, and repetition_penalty come into play. They modify the distribution in different ways, allowing you to shape the model's behavior according to your needs.\n\nTemperature adjusts the sharpness of the probability distribution. A lower temperature (e.g., 0.5) makes the model more confident in its predictions, while a higher temperature (e.g., 2) makes it more adventurous and likely to generate diverse outputs. However, temperature alone does not address the long tail problem in language-model distributions, which are characterized by a few highly probable tokens followed by many less probable ones.\n\nTop-k sampling restricts the model to consider only the top k most probable tokens. This can be useful for generating longer, coherent sequences, particularly in applications like story generation. However, top-k sampling can also introduce issues when the chosen tokens include nonsensical or irrelevant options.\n\nTop-p sampling, also known as nucleus sampling, is similar to top-k but instead of keeping the top k tokens, it keeps all tokens whose cumulative probability exceeds a given threshold p. This approach can provide more flexibility and diversity in the generated text, as it allows the model to consider a wider range of options while still favoring the most likely ones.\n\nMin-p is a modification of top-p that ensures the top-p threshold is never exceeded by more than a small margin. This can help prevent the model from generating excessively long sequences that dominate the output.\n\nRepetition penalty discourages the model from generating repetitive patterns in the generated text. By applying a penalty factor to the probability of repeated tokens, the model is encouraged to introduce more variety and avoid monotony in its output.",
  "summary": "Hello, I'm Shrijith Venkatramana, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product. A language model does not actually \"write\" the next token. It produces a probability distribution over the vocabulary and says, roughly: Here are…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}