Elevated 5xx Error Rate in Navigation flow
Affected components

No components marked as affected

Updates

Write-up published

Read it here

Resolved

Wed, Aug 30, 2023, 12:17 PM
5d earlier...

Resolved

We have been actively monitoring our services in the last hour and we are operating normally for this period. We will work during the day to apply changes that will prevent the issue from happening again.

Thu, Aug 24, 2023, 01:35 PM
36m earlier...

Monitoring

We continue to monitor our platform. We aren't seeing issues related to this incident.

Thu, Aug 24, 2023, 12:59 PM
35m earlier...

Monitoring

We can confirm the full recovery of our navigation flow, and the platform has been operating normally for 30 minutes now. We are monitoring the results of our actions and reviewing internal metrics to see if we have any isolated impacts.

Thu, Aug 24, 2023, 12:24 PM
35m earlier...

Identified

We continue to work on the issue. We already recovered most of our storefront platform. We are still moving the remaining traffic to recover fully. We expected to complete this process within 15 minutes.

Thu, Aug 24, 2023, 11:48 AM
1h earlier...

Identified

We have an impact of ~10-13% in our navigation flow. This is a partial impact so far. We are avoiding making risky maneuvers to restore the 10% degraded and not have a bigger issue.

We continue to apply changes to mitigate the issue.

Thu, Aug 24, 2023, 10:18 AM
50m earlier...

Identified

We are fixing the issue in two replicas of our storefront. We are having a partial impact on our navigation flow, affecting the store performance of IO Storefronts. Shoppers are perceiving slowness or errors in the navigation flow.

Thu, Aug 24, 2023, 09:28 AM
59m earlier...

Identified

We are continuing to fix the issue.

Thu, Aug 24, 2023, 08:29 AM
18m earlier...

Identified

We applied patches in our platform, and customers will see fewer errors in the navigation flow. We are still making additional actions to recover fully.

Thu, Aug 24, 2023, 08:11 AM
12m earlier...

Investigating

We are investigating an increase of 7% in 5xx error rates on our storefront platform.

Thu, Aug 24, 2023, 07:58 AM