Formal verification is the key concept behind the saying that testing alone cannot prove the absence of bugs. It explains both what a mathematical proof can guarantee about software and where those guarantees stop. Below, we examine how formal verification works, how it differs from testing and fuzzing, and what changes when an LLM writes the proofs.
Key Takeaway
Formal verification states the rules a program must follow (the specification) mathematically and proves that the code obeys those rules across all inputs within the stated scope. While tests and fuzzing check only selected or randomized inputs, a proof checker verifies a proof mechanically. Still, if the specification or machine model diverges from reality, “formally verified” does not mean “bug-free.”
What is formal verification?
One-line answer: It is a method that translates code into a mathematical model and mechanically checks a proof—using a proof checker—that the code always follows a defined specification.
Formal verification translates a program into a mathematical model and attempts to prove that it always obeys a defined set of rules. The workflow centers on three parts:
- Specification (spec): The rules defining what the program must do. Correctness is defined by the specification.
- Proof: A logical derivation showing that the mathematical model of the code satisfies the specification. If a proof attempt fails, there could be a bug, and the failure hints at what it is so developers can fix the issue and repeat.
- Proof checker: A mechanical tool (such as Rocq) that verifies the proof against the specification.
For many real-world programs, obtaining a complete, unambiguous specification is hard, which remains a major obstacle to formal verification.
How does it differ from testing and fuzzing?
One-line answer: Tests check chosen inputs and fuzzing runs many mutated random inputs, whereas formal verification proves a property across all inputs within the specification’s scope.
Standard tests evaluate only the specific inputs chosen by a developer. Bugs on untested paths remain hidden.
In fuzzing, a fuzzer feeds huge numbers of semi-random, mutated inputs into an entry point to find crashes. The longer a fuzzer runs, the more bugs it may uncover, but it is still brute force with no guarantee of exercising every path. Because there are infinitely many possible inputs, the search can never be complete.
Formal verification instead makes a claim about all inputs within the specification’s stated range. For instance, an existing fuzz test can be turned into a specification stating that the test never crashes for any input in a stated range, and a proof then establishes that claim across the entire range.

Runtime checkers such as AddressSanitizer (ASan) catch memory errors only on paths that are actually executed; see AddressSanitizer: Finding Memory Errors at Runtime (Korean) for how runtime instrumentation works.
What does “formally verified” guarantee, and what does it not?
One-line answer: It guarantees only the proven theorem under the assumed specification and machine models, and cannot prevent gaps in the specification or mismatches with the real execution environment.
Even when a proof checker accepts a proof, several components still have to be trusted:
- Formal models of the machine and OS: The mathematical descriptions of the CPU instruction set and operating system process behavior.
- Exact theorem statements: What was actually claimed in the specification.
- The proof checker itself: The tool that mechanically checks the proof.
Because of these assumptions, “formally verified” does not mean “bug-free.” A proof checker cannot tell whether the specification matches reality; bugs in verified code appear when the specification or the assumed machine semantics differ from reality. The real machine may do something the model does not describe—for example, when proofs target a different CPU or linking setup than most deployments. Moreover, the proven claim may not cover everything you care about: the sanitizer used to define a “crash” does not detect every memory-safety bug, not all entry points may be covered, and callers may still misuse a library.

In the accompanying Hacker News discussion, commenters debated how to know whether the specification itself is right—often framed as verification (“are we building the product right”) versus validation (“are we building the right product”)—as well as whether such proofs cover concurrency.
What changes when an LLM writes the proofs?
One-line answer: It automates the costly, specialized work of writing large proofs while leaving the final verdict to the mechanical proof checker rather than human line-by-line review.
Historically, formal proofs were costly to produce: they can be much larger than the code itself and require specialized skills, confining formal verification mostly to mission-critical domains.
A research post published by an AI research organization (LLMs are making formal software verification feasible at a much larger scale) argues that turning existing fuzz tests into specifications and having an LLM write the proofs shifts the proof-writing workload to the tool. Bugs were also found during proof attempts.
Crucially, because the proof checker (Rocq) checks the proof against the specification mechanically, these very large proofs do not need to be reviewed by people line by line. One caution is reward hacking: when the LLM is also allowed to patch code, it could make a theorem trivially provable by rewriting the program into one that never crashes, so the authors discouraged this behavior and audited patches.
What are the frequently asked questions?
One-line answer: Here is how formal verification relates to C++26 Contracts, runtime tools like ASan and Valgrind, and which proof tools developers can explore.
How does formal verification relate to C++ Contracts?
C++26 Contracts state conditions that are checked at run time per the C++26 design, whereas a formal proof shows that a condition holds for all inputs without running the program. For background on contracts, see What are C++26 Contracts? Checking Function Conditions with pre, post, and contract_assert.
Does formal verification replace AddressSanitizer (ASan)?
No; they are different tools: runtime checkers find errors on executed paths, while proofs depend on the specification and trusted models—and the research post itself defined “crash” using ASan, which does not catch every memory bug. For a comparison of runtime memory checkers, see What is Valgrind? How It Differs from ASan When Finding C/C++ Memory Leaks.
Can general developers use formal verification tools?
Yes; depending on your language and goal, well-known tools include Rocq, Lean, Verus (for Rust), and Alloy (for modeling).
What related posts are worth reading?
One-line answer: You can explore these related posts on runtime memory checking, automatic resource management, and C++26 function contracts.
- AddressSanitizer: Finding Memory Errors at Runtime (Korean)
- What is Valgrind? How It Differs from ASan When Finding C/C++ Memory Leaks
- What is C++ RAII? Automatically Releasing Resources with Destructors
- What are C++26 Contracts? Checking Function Conditions with pre, post, and contract_assert
What are the sources?
One-line answer: Below are the primary research post and community discussion referenced in this article.
- Anthropic Research, “LLMs are making formal software verification feasible at a much larger scale” (published 2026-10-22, checked 2026-10-25)
- Hacker News, “LLMs are making formal software verification feasible at a much larger scale” discussion (submitted 2026-10-23, checked 2026-10-25)