E2E Validation: Why Shipped Isn't Done

"I think it works."

That sentence has cost more engineering hours than any bug ever written. It is the verbal equivalent of merging without running tests — confident in the absence of evidence.

The failure pattern is universal: build → merge → notify "Done" — without actually running the real code path. The code is committed. The PR is merged. The Slack message is sent. And somewhere in production, a broken image is loading, a webhook is silent, a cron is firing into a dead endpoint.

Shipped means the code exists. Done means the user experience is verified. They are not the same thing.

The Three-Layer Validation Model

Most engineers think in binary: does it work or not? The real answer almost always requires three simultaneous truths.

Layer 1 — File exists on disk. The code was written. The asset was generated. The config was updated. This is the layer developers naturally check because it is the most visible.

Layer 2 — Configuration references correct paths. Frontmatter, environment variables, import statements, and config files point to the right locations with the right values. This layer is where most silent bugs live — because it passes human review but fails at runtime.

Layer 3 — Component renders correctly in browser / executes correctly in production. The actual user-facing experience works. Not locally. Not in dev. In the environment where it matters.

Working on two of three layers is not working.

Every developer who has said "it worked on my machine" was passing Layer 1 while failing Layer 3. Every broken image that made it to production passed Layers 1 and 2 while failing Layer 3. The pyramid does not collapse gracefully. All three must hold.

The Blog-Autopilot Image Incident

This is not a hypothetical. This is a post-mortem.

The blog-autopilot pipeline generates hero images for published articles. The pipeline executed correctly:

• Image file created and saved to disk ✅
• Frontmatter updated with /images/blog-autopilot/slug.png ✅
• PR merged, deploy triggered ✅
• Browser rendered a broken image ❌

Three layers. Two passed. One failed. And the failure was invisible until someone actually opened the article in a browser.

The root cause: the base path in the frontmatter was correct for local dev but wrong for the production Cloudflare Pages deployment. The file existed. The reference existed. The path was wrong in a way that only manifested in production.

The lesson codified into a standing rule: "File committed + frontmatter set" ≠ "image renders on site." Verify all three layers. That rule now lives in CLAUDE.md.

The 90% Coverage Mandate

The 90% floor is not arbitrary. It was chosen because of what lives below it.

Below 90%, your test suite covers happy paths and the obvious failure cases. The edge cases — the ones that actually fail in production — are unguarded. You are essentially testing the code you thought about and leaving unguarded the code you did not.

Above 90%, you are starting to cover systemic behaviors: what happens when a dependency is unavailable, when an input is malformed, when a cron fires twice in rapid succession, when a file path is wrong in exactly the way the blog-autopilot incident demonstrated.

At 90%+, you are shipping with confidence backed by evidence. Below 90%, you are guessing with varying degrees of optimism.

The polymarket-bot has 1063 tests at 93% coverage. That number did not happen by accident. It happened because the system trades real money, and "I think it works" costs real dollars when it is wrong. The testing mandate came from the stakes, not from bureaucratic process.

E2E Is Part of the Implementation Plan, Not an Afterthought

The failure mode that produces broken images, silent webhooks, and wrong paths is not a testing failure. It is a planning failure.

When E2E validation is treated as the last step — something you do after the code is written — it gets abbreviated under time pressure. "I'll write the full tests later." Later never comes. The coverage drops. The broken behavior ships.

The fix is structural: E2E validation belongs in the implementation plan before the first line of code is written.

The questions to answer upfront:

• What does success look like in the real environment?
• What is the command or interaction that proves it works end to end?
• What are the three layers this feature needs to pass?

When you answer those questions before writing code, you build toward a verifiable target. When you answer them after writing code, you rationalize the existing behavior as good enough.

The CI Mindset

Every PR is a bet. You are betting that the code in this branch is correct, behaves as expected in all edge cases, and does not break anything that was working before.

Tests are your due diligence. CI pipelines that enforce them are your solvency check before the bet is placed.

The repositories in this system have .github/workflows/test.yml enforcing coverage gates. A PR that drops below 90% does not merge — not because the rule is punitive, but because below 90% is below confidence. The gate enforces the standard when human judgment fails under pressure.

Lesson 14 Drill

For your next feature — before writing a single line of code — write your E2E validation plan.

Three questions to answer:

1. What does the working end state look like from the user's perspective?
2. What is the exact verification step that confirms all three layers pass?
3. What is the specific command, browser action, or test assertion that proves it?

Write it down. Build toward that target. Do not declare done until you have executed that plan.

Bottom Line

The difference between a junior engineer and a senior engineer is not line count. It is how they define done.

Juniors define done as "code is committed." Seniors define done as "behavior is verified in the real environment." The gap between those definitions is exactly where bugs hide, pipelines break, and images fail to load.

Write the E2E plan first. Run it last. Nothing in between changes what done means.