All learnings

The deploy hook that silently renamed itself

Fri Aug 07
Deploy service D shipped a refactor that silently renamed its incoming webhook key. CI Provider C kept sending POSTs to the legacy path, which 301-redirected to the new key, but the deploy script couldn't follow the redirect during a fresh clone, so every subsequent deploy silently failed.
Standardized webhook path naming and added a startup check that fails the deploy if the configured hook path returns non-200. Forced re-auth of the deploy hook from CI Provider C's UI and treated the silent rename as a deployment-breaking change going forward.
Monitor the deploy endpoint itself with a 30-second ping. If the deploy endpoint starts returning 3xx-only, you've got a regression waiting to happen — a deploy that runs against a redirected hook is a deploy that lies about succeeding.

Add your first endpoint and get alerts on the patterns these postmortems are built from.

Start monitoring free →