Why the same prompt gives different answers.

Send the same prompt to Claude Sonnet twice and you'll usually get two different (but similar) replies. This surprises users who expect deterministic behavior. It's not a bug.

The mechanism

Language models generate one token at a time. At each step, the model outputs a probability distribution over the whole vocabulary — "the next token is 12% likely to be 'the', 8% 'a', 5% 'she'..." A sampler then picks one token from that distribution.

If the sampler is set to "always pick the highest-probability token" (temperature = 0), the output is deterministic. Same input → same output.

If the sampler is set to sample from the distribution (temperature > 0), the output varies. High-probability tokens dominate but occasionally a lower-probability token wins. Compounded across a full reply, small variations early cascade into meaningfully different outputs.

Most consumer products (LADLE, Claude.ai, ChatGPT) use a modest temperature (0.5-1.0 range) because temperature-0 outputs tend to be repetitive and feel robotic. The variation is a feature — it's what makes replies feel fresh across similar prompts.

Where variation matters

- **Creative tasks** — you want the model to produce different draft variants, not the same one every time. - **Brainstorming** — variance is the point. - **Multi-attempt problem-solving** — if the first reply is off, re-asking gets a genuinely different attempt.

Where variation frustrates

- **Reproducibility.** Documenting "the AI said X" for a specific prompt — the next tester runs the same prompt and gets a different reply. - **Test suites for AI-generated code.** A test that passes today might not tomorrow. - **Comparison studies.** Comparing model A to model B requires running the same prompt many times to average out variance.

What you can do about it

**Via the API**, you can set temperature = 0 (deterministic) or specify a specific "seed" that makes the sampling reproducible. Neither is exposed in LADLE's consumer chat surface.

**Via prompting**, you can tighten the reply space — a very specific prompt with strong constraints produces less varied output than a loose prompt. If you're getting frustrated by variance, add constraints.

**Via workflow**, you can generate multiple replies and pick the best. Some tasks are best served by "give me 5 variants, I'll pick." Others by "give me 1, iterate."

The philosophical wrinkle

Because of the variance, a language model isn't a lookup table — it's more like a competent collaborator whose specific phrasing varies. Two conversations with the same person about the same topic go differently too; that's a feature, not a bug.

For most everyday use, this is fine. For specific use cases needing reproducibility (compliance, testing, documentation), plan around it: use temperature-0 via the API, or specify your prompt tightly, or accept that the model will phrase things slightly differently each time.