You will hear the word parameters used in two very different ways. Knowing which one someone means saves a lot of confusion.
Meaning 1: the model's parameters
A model's parameters (or weights) are the billions of numbers learned during training. When you read "a 7B model", the 7B means about seven billion parameters. More parameters generally means a more capable — but slower and more expensive — model. You cannot change these when you call a model; they are fixed.
Meaning 2: request settings
When you call a model, you can also pass settings — often also called parameters — that control how it picks its output. The most important ones:
| Setting | What it does | Typical use |
|---|---|---|
| Temperature | How adventurous the choice of next token is. Low = picks the most likely tokens; high = more variety. | Low for facts, extraction, code. Higher for brainstorming and creative writing. |
| Top-p | Only consider the most likely tokens that together add up to probability p. Another way to limit randomness. | Usually adjust temperature or top-p, not both. |
| Max output tokens | The longest the answer is allowed to be. | Control cost and length. Too low and answers get cut off. |
| Stop sequences | Text that makes the model stop as soon as it writes it. | End output at a known marker. |
Available settings and their ranges vary between providers and models, so check the documentation for the model you use.
Temperature, in code
Here is a tiny simulation. The model has scored three possible next words; temperature changes how those scores turn into probabilities.
# academy-check
import math
def probabilities(scores, temperature):
scaled = [s / temperature for s in scores]
top = max(scaled)
weights = [math.exp(s - top) for s in scaled]
total = sum(weights)
return [round(w / total, 2) for w in weights]
words = ["Paris", "Lyon", "Banana"]
scores = [5.0, 3.0, 0.5]
print(dict(zip(words, probabilities(scores, 0.5)))) # {'Paris': 0.98, 'Lyon': 0.02, 'Banana': 0.0}
print(dict(zip(words, probabilities(scores, 2.0)))) # {'Paris': 0.68, 'Lyon': 0.25, 'Banana': 0.07}
assert probabilities(scores, 0.5)[0] > probabilities(scores, 2.0)[0]
At low temperature, "Paris" is almost certain. At high temperature, less likely words get a real chance — which is great for creative variety and bad for factual answers.
Two common misunderstandings
- "Temperature 0 makes the answer correct." It makes the model pick its most likely tokens. If the most likely answer is wrong, you get the same wrong answer more consistently.
- "Temperature 0 always gives identical output." Usually very similar, but not guaranteed identical, especially across model versions.
Watch for cut-off answers
If an answer hits the max-output-token limit, it stops mid-thought. APIs report why generation finished — normally because the answer was complete, or because it hit the length limit. Check this in your code before trusting the answer.
Try it: ask the same creative question three times at a low temperature and three times at a high one (most AI playgrounds let you change it). Compare the variety.
Quick check: you are extracting invoice numbers from emails. High or low temperature? (Low — you want the most likely, consistent answer, not variety.)