This incident has been resolved. We will not provide further information in a public incident report.
Summary
On Mar 16, 2024 from 20:04 to 21:21 UTC, shoppers navigating in a subset of our stores experienced intermittent errors throughout the navigation and purchase flow.
Our global sales flow was affected for a total of 18 minutes during this incident. This impact was spread across 5 shorter periods of 3-5 minutes.
We apologize for any inconvenience this may have caused.
Timeline
Our monitoring systems indicate that orders were below forecasted levels during the following periods:
20:04 - 20:08 UTC (4 min)
20:12 - 20:15 UTC (3 min)
20:21 - 20:24 UTC (3 min)
20:41 - 20:46 UTC (5 min)
21:18 - 21:21 UTC (3 min)
At 20:07 UTC, our incident response team was notified of the issue and started investigating.
At 20:10 UTC, our team identified a spike of authorization errors in the Logistics service, leading to communication errors with multiple other services. As a cascading effect, this triggered intermittent errors in stock availability requests.
At 21:19 UTC, our team identified the contributing factors of the issue and started implementing a fix.
At 21:34 UTC, the fix was deployed and the incident was fully mitigated.
Resolved
This incident has been resolved. We will not provide further information in a public incident report.
Summary
On Mar 16, 2024 from 20:04 to 21:21 UTC, shoppers navigating in a subset of our stores experienced intermittent errors throughout the navigation and purchase flow.
Our global sales flow was affected for a total of 18 minutes during this incident. This impact was spread across 5 shorter periods of 3-5 minutes.
We apologize for any inconvenience this may have caused.
Timeline
Our monitoring systems indicate that orders were below forecasted levels during the following periods:
20:04 - 20:08 UTC (4 min)
20:12 - 20:15 UTC (3 min)
20:21 - 20:24 UTC (3 min)
20:41 - 20:46 UTC (5 min)
21:18 - 21:21 UTC (3 min)
At 20:07 UTC, our incident response team was notified of the issue and started investigating.
At 20:10 UTC, our team identified a spike of authorization errors in the Logistics service, leading to communication errors with multiple other services. As a cascading effect, this triggered intermittent errors in stock availability requests.
At 21:19 UTC, our team identified the contributing factors of the issue and started implementing a fix.
At 21:34 UTC, the fix was deployed and the incident was fully mitigated.
Monitoring
The fix for the issue in stock availability requests from our Logistics service has been implemented.
Shoppers should no longer be experiencing intermittent errors throughout the navigation and purchase flow.
Our incident response team is monitoring to guarantee that normal platform behavior is fully reestablished.
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.
Identified
We have identified the contributing factors of the issue in stock availability requests from our Logistics service.
Our incident response team estimates that the fix will be implemented in the next 30 minutes.
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.
Investigating
We are currently investigating an issue in stock availability requests from our Logistics service.
Shoppers may be experiencing intermittent errors throughout the navigation and purchase flow, whenever stock availability is required. Our monitoring systems indicate theses errors occur in brief periods of less than 5 minutes.
Our incident response team is working to identify the root cause and implement a solution.
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.