Determinism Was the Illusion We Could Maintain

A 911 dispatcher routes an emergency. A database query returns a result. A medical algorithm recommends a treatment, and a doctor decides whether to follow it. We expect the same thing every time: same input, same output, no surprises. We built the world on that promise. Call it determinism. Call it reliability. Call it the only way we knew to build systems we could understand and trust.

It was never entirely true; we just got disciplined enough to fake it where it mattered. And where it was true, it proved less than we thought. This piece is about both failures, and about why the second one is the one that should keep you up at night.

The illusion and how we maintained it

Determinism is a simple idea: a system is deterministic if you can feed it the same input twice and get the same output both times. No randomness. No surprises. Pure function. The same query returns the same rows. The same algorithm produces the same recommendation. Run the test again and it passes or fails in exactly the same way.

This was the deal we struck with ourselves: if we keep systems deterministic, we can understand them. We can audit them. We can prove they’re safe.

But we were always lying.

Floating point results are not reproducible across environments. The order in which threads execute is not deterministic. The timing of network packets is not deterministic. The state of a cache is not deterministic. And yet we built hospitals on top of databases on top of operating systems on top of all of this, and we told ourselves it was fine because we had locking mechanisms, transaction isolation, careful ordering. We had techniques.

These techniques worked because our systems were small enough. A single database server. A single thread. A single request at a time. We could be disciplined. We could enforce an order. We could maintain the illusion that the randomness wasn’t there.

So we built bigger systems the same way. We added another server and used replication. We added concurrency and used mutexes. We added networks and used consensus algorithms. At each layer, the non-determinism got worse, but we had a new trick to contain it. Not eliminate it. Contain it, and bill the complexity to the layer above.

The illusion held because the gaps between the theory and the reality stayed small enough that we could pretend they didn’t exist.

When it started to crack

The cracks came in layers.

Floating point broke the promise first, though not in the way it usually gets told. IEEE 754 arithmetic is inexact, but the inexactness is deterministic: 0.1 + 0.2 misses the true answer by the same amount every time, on every compliant machine. What broke was reproducibility across environments. Compilers reordered operations. One chip fused a multiply and an add into a single differently-rounded step while another did not. Parallel sums came out differently depending on which core finished first. Same source code, same inputs, different results. Engineers shrugged and learned to live with it. The illusion survived because the error was small and we could reason about the bounds.

Concurrency broke it worse. Once you had multiple threads or processes running at the same time, the order became unpredictable. A race condition is a non-deterministic bug: it happens sometimes, not always, depending on timing. The debugger makes it disappear because the act of adding logging changes the timing. We had a name for this: the Heisenbug. We learned to work around it with locks and semaphores, turning the unpredictable interleaving back into a disciplined sequence. The illusion survived because we could still enforce determinism if we were careful.

Networks shattered it. A message might arrive out of order. A server might be down. A connection might hang. We invented consensus algorithms and retry logic and timeouts. We could build systems that were reliable despite being non-deterministic. Byzantine Fault Tolerance, Paxos, Raft: these are technologies for building trust on top of chaos. But the cost went up. The complexity exploded. And the illusion started to crack, because consensus does not fake determinism. It manufactures the real thing, at enormous cost. Determinism stopped being the default and became something you purchase. And anything you have to purchase, you can fail to afford.

Why we needed the illusion

The illusion mattered because it stood in for something more important: the ability to understand the system and prove it was safe.

When you could assume determinism, you could reason about a system: trace through an execution path, predict the outcome, audit what happened. You could debug by running the test again. You could write a specification and prove the code matched it. You could look at a database transaction and know exactly what would happen.

This was powerful. It let you design safety around the system. It let regulators trust the system. It let you sleep at night knowing what you’d shipped.

But be careful about what was actually doing the work. Determinism never produced understanding on its own. A deterministic system can be perfectly opaque: the risk-scoring algorithms that courts adopted for sentencing and parole were deterministic models, same defendant in, same score out, and nobody outside the vendor could tell you why the score was what it was. What produced understanding was that our systems were small enough to read. Determinism was how we checked: predict the output, run it, see if you were right. Reproducibility was the test that verified comprehension.

Which means the test could outlive the thing it tested. As systems grew past anyone’s ability to read them, the outputs still reproduced and the checks still passed, and we kept the feeling of understanding long after the fact of it was gone. That is what made determinism the illusion we could maintain. Not a false fact about the machine: the outputs really did reproduce. What we maintained was what the reproduction was supposed to mean.

When the randomness crept in, we didn’t give up the illusion. We just compartmentalized it. Yes, the operating system is non-deterministic, but your application layer is deterministic. Yes, the network is chaos, but your application protocol is reliable. Yes, the hardware might glitch, but your error-correction code catches it. You could still maintain islands of determinism even though the ocean around them was random.

And as long as you stayed on the island, you could pretend the ocean didn’t exist.

The problem now

The island is shrinking.

We are now shipping systems that break the proxy outright. Start with an honest correction, because this gets stated sloppily all the time, including by me: large language models are not intrinsically non-deterministic. Pin the model version, fix the seed, decode greedily, and the same prompt returns the same tokens. In practice almost nothing runs that way. Production serving reintroduces drift through batching and parallel hardware, and most applications sample on purpose, because variation is where a lot of the usefulness lives. So the output you get is, in any practical sense, not reproducible.

But suppose you did it. Suppose you locked every seed and every version and got bit-identical outputs forever. You would have recovered determinism and gained almost nothing. You still could not trace the decision. You still could not say why the model answered the way it did. Same input, same output, and you understand it no better than before. This is the tell that determinism was always the proxy and never the prize.

We are putting these systems into places that used to require absolute certainty: screening for disease, flagging fraud, scoring parole risk, answering medical questions, drafting contracts. A human still signs off at the end of most of those chains, for now. Humans have always signed off on inputs they could not personally interrogate; no doctor re-derives the lab assay. But there used to be an expert somewhere who could. The chain of possible interrogation ended at a person. Now it ends at a model, and that is the difference that matters. We are no longer building islands of determinism in a chaotic sea. We are building on the chaos itself and hoping the chaos is well-shaped enough to trust.

The illusion is dead. And we don’t have a replacement.

Be precise about what died. LLMs did not take our understanding away. That was gone before they arrived, in the codebases no one could read and the scores no one could explain. What they took was the ritual that let us avoid noticing. That is why a forty-year-old problem suddenly feels like an emergency.

We don’t have a new language for “I understand this system.” We can’t trace through an LLM and predict what it will do. Running the same input twice and showing the same output was never proof of understanding, but it was the ritual that let us claim it, and with these systems even the ritual is gone. “Show me what it would do with the same input” stops working when the same input produces a different answer every time.

We used to say: a system is reliable if it is deterministic. Now we’re saying: a system is reliable if it produces results that are good enough on average, even though you can’t predict what it will do on any single case.

That is a fundamentally different kind of claim. And outside a few corners of the discipline, software has never had to stand behind one.

The engineer’s bind

An engineer trained in the old world learned: determinism is the goal. Design for it. Test for it. When you can’t achieve it, hide it behind a protocol. Reproducibility is a feature. If a test passes sometimes and fails sometimes, something is wrong. A system you can’t understand is a system you can’t trust. I spent twenty years hiring to that playbook and reviewing to it.

Now they are asked to ship systems that fail the determinism test completely. The system will give different answers to the same question. You often cannot reproduce the failure, and when you can, the reproduction explains nothing. You cannot trace through the execution and see what went wrong. You cannot prove the system is safe the way you used to prove it.

But you are still being asked to take responsibility for it. When the system makes a bad call, discriminates, or fabricates, someone will answer for it. The law has not settled who. Unsettled, in practice, means it rolls downhill to whoever shipped it.

The playbook they were given doesn’t work. The tools they learned don’t work. The promise they were trained to make (determinism equals safety) has been revoked. And nobody has given them a replacement promise.

They are being asked to build with confidence on top of something they cannot fully understand. That is a new kind of burden. And the profession has not yet figured out how to carry it.

What comes next

Here is the part that is uncomfortable and oddly hopeful at the same time: every other high-stakes field solved this long ago, because no other field ever had determinism to lose.

Medicine cannot run the same patient twice. It verifies treatments with clinical trials: good outcomes on average, failure modes characterized, confidence intervals attached. Aviation certifies aircraft against failure probabilities, one in a billion per flight hour for catastrophic events, not proofs that nothing can go wrong. Insurance has priced unpredictable individual outcomes for three centuries. Manufacturing has run statistical process control since the 1920s. Even inside software, the SRE movement already reasons in error budgets, which are probabilistic reliability claims made with a straight face.

Software engineering is the odd one out. We are the only discipline that got to grow up believing in exact reproduction, so we are the only one that never built the statistical muscle. The pieces of the answer mostly exist next door. Importing them looks something like this:

Empirical verification instead of logical proof. You show that the system produces good outputs on a large, representative test set. Not perfect outputs on all inputs, but good outputs on average. This is what we do with LLMs now: benchmark them and hope the benchmark is representative. A clinical trial with worse statistics, for now.

Monitoring and intervention. You don’t assume the system is safe. You watch it. You measure its outputs in production. You have a human in the loop ready to catch mistakes. This moves the burden of safety from “the system is deterministically correct” to “we catch the errors before they cause damage.”

Regulatory frameworks that assume non-determinism. Software regulators are still trained to think deterministically: “prove it always works,” “show me the edge case.” Their counterparts in aviation and medicine learned to think in distributions decades ago. The transfer has not happened yet.

A new definition of reliability. Not “always works,” but “works well enough for its purpose, with clear understanding of failure modes and fallback plans.”

Notice what every item on that list has in common. None of them gets you back to the deterministic column. All of them climb toward the legible row. The fixes are not about making the machine repeat itself. They are about making its behavior something a person can reason about.

None of this is settled in software. But we are not feeling our way in the dark, not really. The lights are on in the building next door.

The weight of it

What we lost when the illusion collapsed is still real: the test that verified our understanding, the proof style a regulator could accept, the clean line of responsibility from input to output to the person who shipped it.

What we gained is something powerful: systems that can explore, reason, adapt, discover things we didn’t program in. Systems that can work in domains too complex for us to specify deterministically.

But we haven’t figured out how to live in that trade-off yet. We are trying to run new kinds of systems through old kinds of reasoning. We are asking for deterministic guarantees from non-deterministic tools. We are treating non-determinism as a defect to apologize for, when most of it is a price we chose to pay for capability.

The illusion kept us safe because it gave us a story we believed: the system is under control, I understand what it does, I can prove it’s safe. That story was always a lie. But it was a lie we could afford to tell.

We cannot afford it anymore. And we have not yet found an honest story to replace it.

---

Further reading

On why determinism was never real: Leslie Lamport’s “Time, Clocks, and the Ordering of Events in a Distributed System” (1978). Foundational paper on why causality in distributed systems is not what we think it is.

On the cost of maintaining the illusion: Jim Gray’s “Why Do Computers Stop and What Can Be Done About It?” (1985). Shows how the gap between theory and practice grows as systems get bigger.

On what comes next: there is no canonical paper yet, which is itself the point. Watch how the ML evaluation field talks: benchmarks, red-teaming, distributional claims. Verification is quietly being reinvented as statistics, and it looks nothing like proof.

The philosophical version: David Hume on causation. He argued (in 1748) that we never actually observe causation, we just observe patterns and assume causation. Distributed systems do not prove him right. They do make him uncomfortably hard to dismiss.

On regulation: “Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability” (AI Now Institute, 2018). An early, serious attempt at the question underneath this whole piece: how do you regulate something you cannot fully predict?