The problem
A transaction success rate sitting at 70% doesn’t look like a crisis on a dashboard. It looks like a metric that’s always been “about right.” But 30% of failed transactions in a high-volume consumer payments flow isn’t noise, it’s real revenue quietly not landing, retried inconsistently, and disputed by customers who don’t understand why a payment they made didn’t register.
The approach
The instinct in most organizations is to treat this as a single technical fix — patch the gateway, add a retry logic layer, move on. That’s necessary but not sufficient. The deeper issue was reconciliation: failures weren’t being classified by cause, so the same categories of failure kept recurring without anyone building a permanent fix for the actual root pattern. The real work was building a systematic reconciliation and monitoring layer — one that classified failure types, surfaced them by frequency, and made the recurring patterns visible instead of buried in a support ticket queue.
The outcome
Transaction success rates moved from 70% to 90+% in production. More durable than the number itself: the monitoring framework built to get there kept catching issues afterward — it became infrastructure the business relied on, not a one-time project that ended when the dashboard looked better.
The lesson
A failing metric is rarely a single bug. It’s usually a missing feedback loop, a system that’s failing the same way repeatedly because nothing is watching for the pattern, only the individual incidents.