This incident has been resolved. We will provide further information in a public incident report.
Summary
On Dec, 15, 2023, from 01:58 UTC to 02:40 UTC, all accounts experienced partial outage on sales flow.
Our global sales flow was affected for 42 minutes during this incident.
We apologize for any inconvenience this may have caused.
Timeline
At 01:58 UTC, our system detected an issue with the RNB system, and the team was alerted.
The team responded promptly and identified at 02:03 UTC that some machines in the system were unhealthy, leading to an increase in error 500.
The team began to take action by immediately deploying more machines as a mitigation strategy. Although the action was taken, it was not successful.
At this point, we initiated a disaster recovery maneuver, successfully partially restoring the sales flow at 02:37 UTC.
At 02:40 UTC, the incident was fully mitigated.
Resolved
This incident has been resolved. We will provide further information in a public incident report.
Summary
On Dec, 15, 2023, from 01:58 UTC to 02:40 UTC, all accounts experienced partial outage on sales flow.
Our global sales flow was affected for 42 minutes during this incident.
We apologize for any inconvenience this may have caused.
Timeline
At 01:58 UTC, our system detected an issue with the RNB system, and the team was alerted.
The team responded promptly and identified at 02:03 UTC that some machines in the system were unhealthy, leading to an increase in error 500.
The team began to take action by immediately deploying more machines as a mitigation strategy. Although the action was taken, it was not successful.
At this point, we initiated a disaster recovery maneuver, successfully partially restoring the sales flow at 02:37 UTC.
At 02:40 UTC, the incident was fully mitigated.
Monitoring
Our incident response team is still monitoring the platform, but we are experiencing normal platform behavior, since last update.
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.
Monitoring
Users should not longer be experiencing any problems on sales flow.
The fix for the issue in Rates and Benefits has been implemented.
Our incident response team is monitoring to guarantee that normal platform behavior is fully reestablished.
We will send an additional update in the next 30 minutes, or as soon as we have more information to share.
Investigating
We are currently investigating an issue with the Rates and Benefits module.
Users may be experiencing issues with the purchase flow.
Our incident response team is working to identify the cause and implement a solution.
We will send an additional update in the next 30 minutes or as soon as we have more information to share.
