The Second System Effect in AI: The Rewrite Is the Failure Mode, Not the Fix

What makes the second system effect worse in an AI workflow?

The most dangerous moment in an AI workflow is not the pilot. It is the quarter after the pilot works, when someone with budget authority proposes the rewrite that will fix everything.

Fred Brooks named the dynamic in 1975, in chapter 5 of The Mythical Man-Month: the second system a team builds is the most dangerous one, because it carries every idea the first system was too constrained to include. The first system shipped against a deadline and one impatient user. The second one ships against a budget and a diagram.

The second system effect in AI is more expensive than in ordinary software, because the rewrite discards the accumulated tuning — the prompt adjustments, the edge cases, the operators' willingness to trust the output — and none of that was ever in version control.

Three mechanisms make the AI version worse than the software version.

The failure stops being loud. A broken build does not deploy. A broken AI workflow deploys, returns plausible output, and degrades by a few points nobody measures, because the evaluation set lived in someone's browser history and three screenshots.

Authority moves into the model. In the working version, routing is an if statement you can read. In the rewrite, the model chooses which tool to call. Every decision handed to the model is a decision no longer visible in a diff, and the change is almost always described as making the system more general.

The bill stops being a number. The working path makes one call per document, so cost is volume times a known rate. The rewrite runs a loop, and a loop re-sends its transcript at every step and pays for it again, so input tokens grow roughly with the square of the step count rather than with the number of tasks. A twenty-step run carrying an average 100,000-token transcript pays for about two million input tokens to do what a single call did with 100,000. That is arithmetic on the token accounting your provider already shows you, not a benchmark, and you can check it against the invoice in the first month the loop lands.

What does the rewrite proposal translate into?

Most rewrite proposals are not a system design. They are one deferred idea, dressed up. I fill in this table during the conversation, using the requester's words in the left column.

The right column matters. If you cannot name what you are giving up, you are not making a decision, you are making an announcement. In five of those six rows the honest answer is "little this quarter," which is a fact about timing rather than a verdict on the eventual platform.

How do you know you are about to build a second system?

The tell is not the size of the proposal. It is the word properly. Nobody rewrites a working system because it is broken; they rewrite it because it is not proper. Four phrasings I have learned to hear as a warning.

"We built it for the first use case, now we understand the domain." Understanding the domain is exactly what makes the second system dangerous, because every new understanding becomes a requirement and none of them get cut.

"We should not be the only team relying on this." True, and it does not follow that the answer is a platform. It follows that the second team needs an interface, which is a much smaller thing.

"The current version will not scale." Ask to what number. A working path handling 4,000 documents a month at 30 seconds each is not near a limit, and scaling is usually a claim about a load you have not measured.

"We will rebuild it in a weekend." This one is often true, and it is the most expensive sentence in the room, because it is true about the code and false about everything the code was tuned to do.

What does a small reversible change look like in practice?

Five techniques, and they compose. None of them require a rewrite to adopt.

Shadow first. Run the new path alongside the old one on identical inputs, act on nothing, log every disagreement. Two weeks of that log is an evaluation set built from your real distribution, which is the artifact a rewrite destroys and then spends a month trying to recreate.

Keep the old path reachable, not merely present. There is a large difference between "the old code still exists" and "the person at the desk can put this one case back on it." A rollback that requires a deployment is not a rollback; it is an incident with a plan. The switch belongs in a config row with the operator's name on it.

Widen on a number, not a feeling. Send the new path a small slice of traffic and widen it only when the operator override rate on that slice is at or below the old path's rate. Without a threshold like that, "it seems better" decides, and "it seems better" is also how the first pilot got approved — which is precisely the judgment you already know is unreliable. The operational gates a pilot has to clear before it counts as production are the right frame here, because most of them are about reversibility rather than accuracy.

Cap the loop. A hard ceiling on model calls per task converts an unbounded cost distribution into a known worst case. When the cap fires you get a log line instead of an invoice, and the case lands in a queue a human can read.

One change per release. If a prompt edit and an architecture change ship together, a regression tells you nothing, and you will spend two weeks deciding which of the two to revert when the answer was written down in the release notes you skipped.

When is the rewrite actually the right call?

Not never. I have argued for rewrites, and here is the test I use.

If the first system was a prototype by agreement — a demo that nobody intended to carry volume, built to answer a question rather than to run a process — the rewrite is not a second system, it is the first one, and the discipline is to date the throwaway explicitly at the start rather than discovering it later.

If the dependency is gone, the rewrite is forced. A deprecated model, a vendor whose API changed shape, a licence that no longer fits. That is a migration, and migrations should be as boring as possible: same behaviour, new plumbing, shadowed before it is trusted.

If the process changed shape underneath the tool — the intake form changed, the volume moved, the policy was rewritten — you are not fixing the old system, you are building for a different one, and that is legitimate work.

The fourth case is the one that actually justifies a platform, and it is diagnosable rather than felt. When the exception queue has grown past what one person can clear in a day, and the exceptions are structurally different from each other rather than variations on one theme, you have a platform problem. When the queue is stable and the exceptions rhyme, you have a product that is working, and the person proposing the rewrite is responding to ambition rather than evidence.

What I cannot tell you is the accept rate of a version that does not exist yet. Nobody can, so the decision cannot be made on expected quality. It has to be made on the asymmetry: extending a working path badly costs a week and a revert, while replacing it badly costs the tuning and the operators' trust, both of which took months to earn and neither of which comes back with the new repository.

What do you ask for before approving the rewrite?

Ask for two artifacts: a two-week version of the first slice, and the single action that routes traffic back to the old path. A team that can produce both is describing real work with a gate on it, and you can fund that with confidence. A team that can only produce a diagram, a framework choice and a quarter-long estimate is describing enthusiasm, and the cheap answer is no. The code was never the expensive part of the system you are replacing — the tuning, the edge cases and the override history were, and those are the things a second system quietly throws away. Extend the working path until its exception queue tells you, in writing, that it cannot take the next case.