1062 words
5 minutes
The Bug That Copied Itself Twenty Times
阅读中文版

Our release pipeline builds every package twice.

Here’s the log from a real release, two steps back to back:

[Python Semantic Release step]
21:48:53 Successfully built dist/my_package-2.4.1.tar.gz
21:48:53 Successfully built dist/my_package-2.4.1-py3-none-any.whl
Build completed successfully!
[Build distribution packages step]
21:48:57 Successfully built dist/my_package-2.4.1.tar.gz
21:48:57 Successfully built dist/my_package-2.4.1-py3-none-any.whl

Same files, byte-for-byte, four seconds apart. The second build overwrites the first with identical output and contributes exactly nothing. It’s been doing that on every release, in this repo and — as I’d find out — in about twenty others, for the better part of a year. Nobody noticed, because nothing ever failed.

Last time I wrote about finding a dead-code bug while reading our PR workflow line by line. This is what happened when I kept reading into the release workflow. It’s a better story, because this bug has three acts: the find, the spread, and the part where fixing it broke something.

Act 1: One bug#

The release workflow uses python-semantic-release to automate versioning (how that whole machine works is its own post). The relevant detail is one line in pyproject.toml:

[tool.semantic_release]
build_command = "pip install uv && uv build"

Per the tool’s docs, when build_command is set, the release step builds the package itself, internally, as part of the version bump. And right after that step, our workflow had:

- name: Build distribution packages
run: uv build

An explicit second build of the thing that was just built. I didn’t trust the docs alone — CI configs are exactly where documented behavior and actual behavior drift apart — so I pulled the logs of a real release. That’s the excerpt above. Both builds, timestamped, four seconds apart. Confirmed.

Harmless, right? A few wasted seconds per release. Hold that thought for Act 3.

Act 2: Twenty bugs#

Before fixing it, I wanted to know how it got there. git blame on both lines — the build_command config and the redundant step — and the trail led somewhere more interesting than this repo: both lines were introduced in the same commit, by the same author, in the template repository that our Python services are scaffolded from. Months before this service even existed.

Think of a template repo as a photocopier. A smudge on the master copy isn’t one dirty page — it’s a smudge on every page the machine will ever print. Scaffolding a new service is hitting the copy button, and every copy since that commit came out with the smudge included: config and redundant step, together, born duplicated.

An org-wide code search turned up roughly two dozen repos sharing this release workflow. Then came a lesson I nearly skipped: I started drafting an announcement listing the affected repos straight from the search results — and when I actually opened each repo’s files to verify, two of them turned out to be false positives. They had the explicit build step but no build_command, meaning that step was their only build. Not redundant at all. Search results told me where to look; only the files themselves could tell me what was true. Nineteen confirmed, two acquitted.

Act 3: The bug bites back#

The fix looked like the easiest PR of my life. Delete three lines:

- name: Build distribution packages
run: uv build

The internal build already produces dist/, the later publish steps pick it up from disk, done. PR opened, approved, merged.

Then an AI review bot left a comment on the downstream PR that I had to read three times:

Removing the unconditional build step means dist/ only exists when a release actually happens. The artifact-upload step runs on every push with if-no-files-found: error — runs with no release will now fail.

It was right. Here’s the path neither I nor eight months of green checkmarks had ever considered: the release tool doesn’t always release. Push a commit like docs: fix typo to main and it scans the history, decides no version bump is warranted, and returns early — before running build_command. No release, no build, no dist/.

Under the old workflow that never mattered, because the “redundant” explicit build ran unconditionally. Every run had a dist/, so the unguarded upload step downstream always found its files.

The wall I was knocking down was load-bearing.

It looked purely decorative — an obvious dead weight, confirmed redundant by docs and logs. But for eight months it had been silently propping up a different bug: an upload step that should have been gated on “did we actually release?” and never was. Two mistakes, leaning on each other, adding up to green. Sound familiar? The dead export in the last post survived the same way — cancelled out by an unrelated config quirk. This is apparently a pattern: CI doesn’t stay green because it’s correct; it stays green because its bugs come in mutually-cancelling pairs.

The real fix ships the demolition and the support beam together:

- name: Build distribution packages
run: uv build
- name: Publish package to GitHub Releases
if: steps.release.outputs.released == 'true'
uses: python-semantic-release/publish-action@...
- name: Upload | Distribution Artifacts
if: steps.release.outputs.released == 'true'
uses: actions/upload-artifact@...

Delete the redundant build, and gate every step that assumes dist/ exists on the release actually having happened — the same guard the PyPI-publish step already had, which someone once thought to add there and nowhere else.

Three rules from one bug#

Templates multiply mistakes at birth. A bug in a normal repo is one bug. A bug in a template is a bug in every repo that will ever be scaffolded from it — ours got photocopied nineteen times before anyone looked. Review templates with scrutiny proportional to their copy count, and when you find a bug in any scaffolded repo, always check the master copy first. Fix the photocopier, not the pages.

Redundancy can be load-bearing. Before deleting anything “obviously useless,” ask what might be leaning on it. The safest fixes ship the deletion and the replacement guard in the same PR. (And a bot may see what you can’t: I verified the happy path with docs and logs and still missed the no-release branch. The bot enumerated paths instead of following the story. Humans narrate; machines enumerate.)

A green checkmark is evidence of nothing but itself. Same lesson as last time, bigger blast radius: twenty repos, most of a year, zero failures — and two real bugs the whole time. Green means the broken paths haven’t been walked yet. The only way to find them early is to go read the files nobody reads, in the repo everyone copies.