This incident has been resolved. We will provide further information in a public incident report.
Summary
On November 13, from 4:36 PM UTC to 5:17 PM UTC, shoppers experienced issues when completing a purchase.
Our sales flow was partially affected for 43 minutes during this incident.
We apologize for any inconvenience this may have caused.
Timeline
At 4:41 PM UTC, our incident response team was notified of the issue.
At 4:50 PM UTC, we identified the requests responsible for causing the latency increase in the promotions module.
At 4:57 PM UTC, we initiated the manual rate-limiting process for this identified behavior after identifying the accounts responsible for these requests.
At 5:17 PM UTC, we observed that shoppers were able to place orders again normally.
At 5:27 PM UTC, our team started monitoring, with no observable impact on the sales flow.
No components marked as affected
Resolved
This incident has been resolved. We will provide further information in a public incident report.
Summary
On November 13, from 4:36 PM UTC to 5:17 PM UTC, shoppers experienced issues when completing a purchase.
Our sales flow was partially affected for 43 minutes during this incident.
We apologize for any inconvenience this may have caused.
Timeline
At 4:41 PM UTC, our incident response team was notified of the issue.
At 4:50 PM UTC, we identified the requests responsible for causing the latency increase in the promotions module.
At 4:57 PM UTC, we initiated the manual rate-limiting process for this identified behavior after identifying the accounts responsible for these requests.
At 5:17 PM UTC, we observed that shoppers were able to place orders again normally.
At 5:27 PM UTC, our team started monitoring, with no observable impact on the sales flow.
Monitoring
We are still monitoring and investigating the issue. Once we are confident this is over, we are going to resolve this incident.
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.
Monitoring
The fix for the issue in Promotions has been applied.
Shoppers should no longer experience issues in the checkout process.
Our incident response team is monitoring to guarantee that normal platform behavior is fully reestablished.
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.
Identified
We have identified the root cause of the issue in the Promotions module
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.