A missing checkpoint in the code that landed Apollo 11

I have been in love with flight since I was a child. Planes, rockets, anything that climbed. I read about the Apollo missions, and later the Space Shuttle, the way other kids read about their football heroes, and that never wore off.

So when I learned that the actual software that landed Apollo 11 on the Moon is public now, every line of it, I spent an evening reading it. I told myself that was all I was doing. Within an hour I was looking for a mistake.

There was a second pull, too. I started my career close to the metal, writing C and assembly, the kind of programming where you are aware of every byte. I have not done much of that in years. Opening the Apollo listings meant reading raw assembly again, and I had forgotten how much I loved it. It felt less like work and more like meeting an old friend.

I found one. And what happened next is the part I have not stopped thinking about.

Looking for a crack in the most checked code ever written

Luminary 099 is the program that flew in the Apollo 11 lunar module. About 65,000 lines of assembly, written by hand at the MIT Instrumentation Laboratory, for a computer with a few kilobytes of memory. It has been studied for decades. If any software on Earth has had every single line inspected, it is this.

So finding a genuine defect in it should be almost impossible. That was exactly why I wanted to try. How close to impossible?

The lunar module ran a loop called SERVICER about twice a second all the way down to the surface. One of its jobs was small and oddly human: keep a running tally of how much the spacecraft’s speed had changed, and show that number to the astronauts.

The bug

The loop kept that tally by adding each new slice of speed onto the total. Add, not replace. Fine, until you remember the one thing the Apollo computer is legendary for: restarting itself.

When the machine got overwhelmed, it rebooted and picked each task back up from a saved checkpoint. This is not a flaw, it is the reason the landing succeeded. It is the mechanism behind the 1201 and 1202 alarms that fired while Armstrong was flying down, the alarms that nearly waved off the landing and did not, because the computer kept calmly restarting and carrying on.

But a restart means re-running whatever came after the last checkpoint. And the instruction that added to the speed total sat in a gap with no checkpoint in front of it. So if the computer restarted in that gap, it came back, re-read the same slice of speed, and added it a second time. The number on the astronauts’ display would read a little too high.

The clue that this was an oversight and not a decision was sitting right next to it. The very same loop also updated the spacecraft’s mass, and that update was written the safe way: it reloaded its starting value before adding, so re-running it always landed on the same answer. The speed total had no such guard. One neighbor was protected. The other was not. That asymmetry is what caught my eye.

Here is what it actually did. The velocity total the astronauts saw would come out wrong, overcounted by one slice of the descent, at the exact moment the computer was fighting for its life. It did not touch the guidance or the engine, and it reset at the next ignition. But make no mistake about what this is: a genuine defect in the software that flew humans to the surface of another world, sitting one gap in the code away from the most watched sixty seconds in the history of flight.

For about a day, I thought it might be new.

Someone had been here before

I posted what I had to the Virtual AGC project, the group that transcribed this code from the original printouts, and asked directly: is this already known, and what should I be citing?

Mike Stewart answered within hours. He has reconstructed several of these programs from scratch and knows their internals as well as anyone alive.

“What you’re describing was indeed a bug, assigned anomaly number L-1D-06.”

It had been caught in 1970. Two copies of the anomaly report sit in the Luminary archives. The fix went into a later revision of the program, documented in its own engineering memo. This stretch of the software, Mike noted, is among the best-understood code in the entire Apollo program.

So no, I was not the first. A team at Mission Control beat me to it by 55 years. I sat with that for a moment, and then I pulled their original 1970 documents to see exactly what they had written.

The same answer, 55 years apart

The MIT anomaly report, L-1D-06, was filed on 8 July 1970. It describes the bug as the “Delta Velocity measured from SERVICER” being “incremented twice in DVTOTAL” after a “Restart occurred after one SERVICER cycle.” It names the cause as the “DAS instruction to DVTOTAL” being “performed twice” with “no Phase change immediately following” it. And in the box on the form marked mission effect, one word: “None.”

Read that again. It is my analysis, almost to the word. Same mechanism. Same cause. Same conclusion: the error lived in the crew’s velocity readout, and the guidance never depended on it.

A team at Mission Control wrote that on a paper form while the world watched them aim at the Moon. I arrived at the same place on a laptop, on a weeknight, in 2026. We never met across those 55 years, and we agreed on every point that counts.

That is the thing I cannot get over. Not that I found a bug, but that the code was clear enough, and the physics honest enough, that two people separated by half a century read the same lines and reached the same verdict. Good engineering is a message in a bottle. Write it plainly enough and someone opens it decades later and understands you exactly.

And the fix, from Luminary memo #175, dated 13 October 1970, is described exactly the way the corrected code reads. The engineer titled the anomaly, plainly, “Restart protect DVTOTAL,” and wrote: “the old DVTOTAL plus the increment is stored in DVTEMP; a phase change is done; then DVTEMP is picked up and double exchanged with DVTOTAL.” Compute into a scratch cell, save your checkpoint, then commit. The exact pattern the buggy code was missing.

How you reported a bug in 1970

Here is the detail that turned my whole view of this around. When I found the bug, I reported it by typing a message into a web forum. It took me four minutes. Anyone on Earth can read it, argue with it, or ignore it.

The document I was reading, the one from 1970, is not a forum post. It is a form. Specifically it is a “Program Change Routing Slip,” and printed across the top of it is the machinery of how a change to Apollo’s flight software actually happened.

That bug did not get fixed because one engineer noticed it and quietly edited a file. It got a formal anomaly number, L-1D-06. It was written up by an originator at NASA’s Manned Spacecraft Center. It carried fields for approval by MIT’s Instrumentation Laboratory, by NASA, by a Mission Design Review Board, by technical committee meetings, by a project manager, by a supervisor, with signature and date lines for each. The fix itself then rode into the program inside a numbered engineering memo alongside a batch of other changes, each traceable to its own paperwork.

A defect in a readout, and it moved through a review chain that half the companies I have worked at would envy for their production systems.

I do not romanticize all of it. That process was slow and heavy, and it had to be, because you could not ship a patch to a spacecraft that was already flying. The cost of a mistake was not a rollback, it was lives. So the bureaucracy was not red tape, it was a load-bearing wall. Every signature was a person putting their name against the claim that this change would not kill anyone.

Today I found a 55-year-old bug and reported it before my coffee got cold. That speed is a gift, and I would not trade it back. But it is worth remembering what we quietly gave up to get it. Somewhere between their world and ours, “who signed off on this” stopped being a question we could always answer.

What this was really about

In the middle of the most intense engineering effort in human history, with lives and a nation’s pride riding on the schedule, someone took a defect that only reached a cockpit readout, gave it a formal tracking number, wrote it up twice, filed an engineering memo, and fixed it in the next version.

That is the standard. Not brilliance in a single stroke, though there was plenty of that. Care. Documented, disciplined, almost stubborn care about things that did not even matter, because the habit of caring about the things that do not matter is exactly what keeps you alive when you reach the things that do.

I went looking for a crack in Apollo. What I found is that there are almost none, and the reason is not luck. It is that the people who built it would not let a single flaw go unwritten. I have worked in software my whole career. I have rarely seen anything held to that bar. It is worth remembering that we once did, with less computer than you are reading this on.

So yes, the engineers got there first. I would have been amazed if they had not. What I did not expect was to sit at my desk, half a century later, and shake hands with them through the code.

Read it yourself

• The Apollo 11 code: Luminary 099, SERVICER.agc, the loop and the DAS DVTOTAL line
• The 1970 anomaly report, L-1D-06: ibiblio.org/apollo/Documents/contents_of_luminary_1d.pdf, page 57
• The fix, Luminary memo #175: ibiblio.org/apollo/Documents/LUM175-DD_text.pdf
• The discussion where the experts set me straight: github.com/virtualagc/virtualagc/discussions/1340
• The Virtual AGC project, which made all of this possible: ibiblio.org/apollo

With thanks to Mike Stewart and Ron Burkey.