Temperature
Parameter that controls the balance between predictability and creativity in a language model's responses, defining the level of randomness in word choice.
What it is
In the context of Artificial Intelligence and Large Language Models (LLMs), Temperature is a configuration hyperparameter that dictates how 'risky' or conservative the model's text output will be. Although we often speak of AI as if it were 'thinking', in reality, it is calculating probabilities. For each subsequent word (or token) in a text, the model generates a list of candidates with different probabilities of occurrence. Temperature is the dial that adjusts these probabilities before the final choice is made.
Imagine temperature as a creative freedom selector. If it is low, the model behaves like an employee extremely focused on rules and facts, always choosing the most obvious and safe option. If it is high, the model transforms into a collaborator in a brainstorming session, willing to choose less likely words to create more varied and unexpected sentences.
How it works
Temperature works through mathematical manipulation of the probability distribution (usually via the softmax function). The value typically ranges between 0 and 1, although some systems allow values up to 2.
When the temperature is 0 (or close to zero), we say the model becomes 'deterministic'. It will almost always choose the token with the highest probability. If you ask 'What is the capital of Portugal?' ten times, you will get the same short and direct answer ten times. The model does not take risks.
As we increase the temperature to values like 0.7 or 1.0, the difference between the most likely word and the others decreases. The model 'flattens' the probability curve, giving a chance to words that would otherwise be discarded. This introduces linguistic variety but also increases the risk of the model drifting off-topic or inventing facts, a phenomenon known as hallucination.
When to use it
The choice of the ideal temperature depends entirely on the task your SME intends to automate. There is no absolute 'correct' value, only the appropriate value for the context.
Low Temperature (0.0 to 0.3): Precision Tasks You should use these values when accuracy is a priority and you want no surprises in format or content. Typical examples include:
- Extracting data from invoices or contracts.
- Classifying support tickets (e.g., deciding if an email is a 'Complaint' or a 'Quote Request').
- Writing programming code or SQL scripts.
- FAQ responses based strictly on a technical document.
Medium Temperature (0.4 to 0.7): Balance and Fluidity Ideal for tasks that require natural writing but still need to maintain a coherent logical structure. Examples:
- Drafting personalized sales emails.
- Summarizing meetings or long articles.
- Creating product descriptions for e-commerce.
High Temperature (0.8 to 1.2+): Creativity and Exploration Use high values when the goal is originality and diversity of ideas. Examples:
- Generating slogans for a marketing campaign.
- Ideating new names for a brand or project.
- Creative writing, such as scripts or poetry.
Common mistakes
The most frequent mistake in SMEs starting to implement AI is ignoring this parameter, leaving it at the default value (usually 0.7 or 1.0) for all tasks. This leads to two main problems:
-
Inconsistency in Rigid Processes: If you are using AI to classify leads in a CRM and the temperature is high, the model may invent new classification categories that do not exist in your system, breaking the subsequent automation.
-
Factual Hallucinations: When asking for a summary of a financial report with high temperature, the model may feel tempted to 'fill in the blanks' with numbers that seem plausible but are false, simply because the temperature math forced it to choose less likely but aesthetically pleasing tokens.
Another mistake is believing that a temperature of 0 guarantees 100% truth. Temperature 0 only guarantees that the model chooses its best bet; if the model has learned incorrect information or does not have access to the correct data, it will still give the wrong answer, but it will do so consistently.
Practical example for an SME
Imagine a real estate consultancy in Lisbon that uses an AI assistant for two distinct functions:
Scenario A: Contract Analysis The company uses a script that reads lease agreements and extracts the expiration date and the rent amount. Here, the developer sets the Temperature to 0.0. This ensures that if the contract says '1200€', the AI extracts exactly '1200€' and does not decide to write 'One thousand two hundred euros' or round the value, facilitating automatic insertion into the management software.
Scenario B: Social Media The same company needs caption ideas for Instagram about a new apartment in Chiado. The marketing manager sets the Temperature to 0.9. The result is three or four options with different tones: one more emotional, one focused on investment, and another shorter and more modern. This variety allows the team to choose the one that best suits the audience, something a low temperature (which would produce very generic and repetitive sentences) could not achieve.
Frequently Asked Questions
Q: Can I change the temperature in the middle of a chat conversation? A: In most common chat interfaces (like the basic versions of ChatGPT), the user does not have direct control over this parameter. However, in professional tools, APIs, or customized interfaces for companies, it is possible (and recommended) to set the temperature for each type of request.
Q: Does temperature affect the cost of using AI? A: No. The cost (processed tokens) is the same regardless of the set temperature. It only changes the word selection logic, not the computational effort required to generate them.
Q: If I want the AI never to lie, should I use temperature 0? A: Not necessarily. Temperature 0 reduces the probability of the model 'inventing' through randomness, but preventing actual untruths (hallucinations) is better solved with techniques like RAG (Retrieval-Augmented Generation), providing the correct data to the model's context.
Q: What happens if I set the temperature to 2.0? A: In most modern models, the text becomes completely unintelligible. Probabilities become so dispersed that the model starts choosing random characters, meaningless punctuation, or words that do not exist, making the response useless.
Practical examples
- 01Setting the temperature to 0.0 when extracting NIFs and total values from digitized invoice images via OCR.
- 02Using temperature 0.8 to generate ten variations of creative names for a new wine from a regional winery.
- 03Keeping the temperature at 0.3 to answer technical customer questions based on the product instruction manual.
- 04Adjusting the temperature to 1.0 to create a draft of a blog post with an enthusiastic and dynamic tone of voice.
Read on the blog
Want to use Temperature in your company?
30 minutes, free, no commitment. We map where it fits.
Free AI diagnosis