The deploy that quietly undoes last month's fix

Ask a team how production broke and the honest answer is rarely a new bug. More often it is an old fix that stopped being there. Somebody corrected a file directly on the server, or on a branch that never merged, and weeks later a perfectly ordinary deploy from somewhere else copied the old version back over it. The build passed. The tests passed. Nothing was wrong except the one thing that mattered.

Tests do not catch this, and it is worth being precise about why. A test checks what the code does. It has no opinion about what the code used to do that you cared about, and it certainly has no idea that a fix exists on the server but not in the repository. A deploy script that copies files is worse still: it has no memory at all. Every deploy is its first.

So the first thing I want from a deploy is a memory. Record, on the server, which commit produced each file that is live. Before shipping, check that the commit behind the live code is contained in what you are about to ship. If it is not, stop — you are about to overwrite work your branch does not have. That single check turns the most common silent regression into a loud refusal, at the one moment somebody is paying attention.

The second is a short list of invariants: plain rules about the built output, checked before anything is copied. This file must contain that call. That string must never appear in that module. They are crude, and the crudeness is the point — they run in a second, they need no test environment, and they encode the fixes that were subtle enough to be undone without anyone noticing. A good rule of thumb: when a fix would be silently reverted by a stale build, it earns a line on the list.

The third is honesty about what 'zero downtime' means. Process managers will happily report a worker as online the instant it has been started rather than when it is actually accepting requests, and reload the next one straight after. Reload one worker at a time, wait until it answers a health check, and only then touch the next. The difference is measured in the requests your customers see fail, and it is invisible until somebody watches the site during a deploy.

The fourth is to finish with real requests. A handful of smoke tests against the live site after the switch — the home page, a search, the pages that make money — and if any of them fails, restore the previous version automatically. One caution learned the hard way by many teams: a smoke test outlives the endpoint it checks. Remove an endpoint without removing its test and every future deploy fails its own check and rolls itself back.

And watch what the deploy does not do. If your pipeline ships compiled code but never installs dependencies, then the first change that adds a package produces a build that is green on a laptop and an application that cannot start on the server. Either install on the server as part of the deploy, or make adding a runtime dependency something the pipeline refuses until somebody has done it by hand.

None of this needs a platform or a vendor. It is a script, a manifest and a list of rules, and it pays for itself the first time it stops you. A deploy is the one step that touches every line you have ever written. It deserves to know which of them matter.