Why creativity settings are product decisions

Temperature gets described as a creativity dial. Accurate enough, and it hides the thing that matters in a product: temperature decides how much variation your users experience.

That makes it a product decision rather than a tuning preference, and it should be set per task rather than once for the whole system.

One global setting is the bug

Most codebases set temperature once, in a config, and every call inherits it. Then the same value governs a marketing-copy generator, a customer-facing answer, and a classifier whose output feeds a database write.

Those three want completely different things. The copy generator benefits from variety; identical suggestions every time are useless. The classifier needs the same input to produce the same label, every time, or your data is quietly non-deterministic. The customer answer sits in between and probably leans consistent, because two users asking the same question and getting materially different answers is a support problem.

A single number cannot serve all three, and the one that gets chosen is usually whatever made the demo feel impressive.

Where randomness bites silently

Creative output that varies is fine. The dangerous case is randomness leaking into a step something downstream depends on.

A field that is usually "approved" and occasionally "Approved." A list that is normally five items and sometimes four. A JSON response that once in fifty runs arrives with a sentence in front of it. Each is a parsing failure, a data inconsistency, or an audit problem, and each will be intermittent, which makes it expensive to diagnose.

Low temperature does not guarantee determinism, incidentally. It narrows the distribution; it does not eliminate variation. Where output shape genuinely must hold, constrain the shape as well as the sampling.

Test the spread, not the sample

Run the same input ten times at your chosen setting and look at how far the outputs move apart.

That spread is what users actually experience. One good sample tells you the best case; the spread tells you the product. And it is a fast enough check that there is no excuse for shipping a setting nobody measured.

Closing thought

Creativity settings are product decisions because variance is part of the user experience. Choose per task, write down which tasks tolerate variety and which demand repeatability, and give the defaults an owner, before someone tunes them by feel at 5pm.

Which of your features needs consistency more than creativity, even if the demo looks less impressive for it?