Essay: Why writing correct software is hard
proomt.com

Essay: Why writing correct software is hard

Archived — this story has rotated out of today’s deck. It is kept here in full.

The gist

An essay aggregates computability and complexity theory to argue programs and full understanding are at odds.
The piece pushes back on claims that the right tools make correctness easy.

Background

The essay, published on pron.github.io, draws on computability theory and complexity theory to argue that producing correct software is essentially difficult, and that correctness carries a significant cost rooted in the "natural laws" of computation. It grew out of a talk at Curry On about computational complexity results in software verification, a subject the author took up while formally specifying and verifying an upcoming revamped version of Galaxy. It responds to online statements claiming software correctness is easy to achieve with the right tools.

How it unfolded

  1. 1930Mathematician David Hilbert, in a Königsberg lecture, rejected the "ignoramus et ignorabimus" view, declaring "We must know, we shall know!"
  2. 1936Alonzo Church and Alan Turing proved the halting theorem, breaking the third pillar of Hilbert's program, the decision problem.
  3. Oct 15, 2025Gary Marcus published a piece decoding and debunking Hard Fork's Kevin Roose, arguing writing code and debugging it are different things.
  4. Dec 3, 2025UNESCO published an article titled "Coding is dead"? Teaching computer programming in the age of AI.
  5. Nov 8, 2026Software engineer and writer Mike Parsons posted "So, AI? Confessions of a software engineer and writer" on Facebook.
  6. Nov 10, 2026The Nasscom community published a post, "Engineering Tomorrow: The Future of Software Development."

Key terms

computability theory
Branch of mathematical logic and computer science studying which problems can be solved by computation.
complexity theory
Field classifying computational problems by the resources they require and relating those classes.
halting theorem
A result proving no general algorithm can decide, for every program and input, whether the program will halt or run forever.

Source: Wikipedia

Who’s saying what

Author
Programs and the complete understanding required for correctness are fundamentally at odds, and correctness comes at a significant cost that cannot be avoided.
Caution
Gary Marcus argues any coder knows writing code is one thing, debugging it another, and maintaining it a year or a decade later harder still, with the last 20% extremely difficult.
Official
Nasscom states AI-generated software is not automatically reliable or production-ready and that human oversight remains essential for validating quality, architecture, security and compliance.
Expert
Mike Parsons says AI models now produce solutions that once took him days or weeks, often better than his own, though he rejects predictions that AI will make human intelligence irrelevant.

Still unverified

The essay notes that some say Gödel's incompleteness theorems also demolished the third pillar of Hilbert's program, "only he didn't realize it."

Sources

See today’s stories in the app Gistline — free on the App Store