In words
What it is, why it matters, and what it is like.
Why am I learning this?
When you use a chatbot or an email assistant powered by a large language model, you may notice that asking the same question twice sometimes gives you different answers. This unpredictability matters for three concrete reasons in your daily work:
1. Testing is harder: You cannot write a test that expects one exact answer if the model can return several slightly different ones. You need strategies that accept a range of correct responses.
2. Downstream systems must be flexible: If your application sends the model's output to a database or another function, that next step must handle small variations in wording without breaking.
3. User expectations must be managed: Users will be confused if they get different answers for the same question. You need to design the product so this variation feels natural or is hidden from them.
Understanding this helps you write robust tests and choose settings that control how much variation occurs.
The idea, in plain terms
A large language model does not have a single fixed answer waiting inside it. Instead, it works like this: it looks at your input and calculates a likelihood for every possible next word. For example, if you type "The capital of France is", the model might decide there is a 95% chance the next word is "Paris", a 2% chance it is "a", and a 1% chance it is "the". It also assigns tiny chances to many other words.
When it actually produces the answer, it does not just pick the most likely word every time. Instead, it performs a random selection based on likelihoods. It draws one word from that list of chances. Because "Paris" has the highest chance, it appears most often, but occasionally the model picks "a" or "the" instead.
This randomness is built into how the model operates. When you build larger sentences, these small differences accumulate. If the model picks a slightly different word at position 5, that changes the context for position 6. This can cause the rest of the answer to drift further away from what you might expect.
Even if you try to force the model to be completely predictable by setting its variation setting (often called "temperature") to zero, it is still not perfectly consistent. The computer calculations that happen behind the scenes involve tiny rounding errors because computers approximate real numbers using a limited number of bits. These micro-differences can shift which word is chosen last minute. This is known as floating-point arithmetic imprecision. Because of these computational limits, you cannot rely on identical inputs always producing identical outputs in practice.
An analogy
Imagine asking a committee of experts the same question: "What is the best way to learn French?" The experts are not robots; they are people who weigh information differently and may feel more like one approach than another on different days. If you ask them today, one might say "immersion," another "daily practice." If you ask again tomorrow, they might give the same answers, or they might swap their priorities based on what they read that morning.
The large language model works similarly but for a mechanical reason: it randomly selects from a list of weighted possibilities each time, rather than because its mind has changed. Unlike the experts, who have fixed knowledge and could theoretically agree to give the same answer if forced, the model’s randomness is fundamental to how it generates text.
Definition
LLM nondeterminism is the tendency of a language model to produce different outputs for the same input, caused by its method of randomly selecting words based on likelihoods and by tiny rounding errors in computer calculations.
It is best described as: A large language model produces variable outputs for identical inputs because it selects words from a probability distribution rather than choosing a single fixed answer, and because underlying computational approximations introduce minor instabilities.
Where this sits
This concept links directly to Temperature and Sampling, which are the controls you use to adjust how much randomness (the random selection based on likelihoods) influences the output. It also connects to Tokenization and Context Window, because the model makes these random choices one piece of word at a time within a limited memory space. Later, when you study Retrieval-Augmented Generation and AI Agents, you will see that these systems are designed specifically to handle this variability, often by asking the model multiple times and comparing results to find a reliable answer.
