This incident has been resolved. We will not provide further information in a public incident report.
Summary
On Mar 16, 2024 from 20:04 to 21:21 UTC, shoppers navigating in a subset of our stores experienced intermittent errors throughout the navigation and purchase flow.
Our global sales flow was affected for a total of 18 minutes during this incident. This impact was spread across 5 shorter periods of 3-5 minutes.
We apologize for any inconvenience this may have caused.
Timeline
Our monitoring systems indicate that orders were below forecasted levels during the following periods:
20:04 - 20:08 UTC (4 min)
20:12 - 20:15 UTC (3 min)
20:21 - 20:24 UTC (3 min)
20:41 - 20:46 UTC (5 min)
21:18 - 21:21 UTC (3 min)
At 20:07 UTC, our incident response team was notified of the issue and started investigating.
At 20:10 UTC, our team identified a spike of authorization errors in the Logistics service, leading to communication errors with multiple other services. As a cascading effect, this triggered intermittent errors in stock availability requests.
At 21:19 UTC, our team identified the contributing factors of the issue and started implementing a fix.
At 21:34 UTC, the fix was deployed and the incident was fully mitigated.