Affected components

No components marked as affected

Updates

Write-up published

Read it here

Resolved

From 15h30 UTC to 16h10 UTC, FastStore WebOps faced an outage caused by its Router service. FastStore WebOps is the underlying platform where clients using FastStore deploy their stores.

At 15h40 UTC, we identified that our router service had been deleted, and we promptly started the restore process. Meanwhile, we opened a separate investigation to map what happened. At 15h48 UTC, we finished the main cluster mitigation and restored other support infrastructure. 

At 15h56 UTC, FastStore traffic was restored, but a secondary service used to serve optimized images was still unreachable. At 16h10 UTC, our image optimizer service was restored. 

For us to continue to be the trusted partner to the success of our clients, our team is working on follow-up actions to ensure that this incident does not happen again and that we identify and recover from future incidents faster. We are committed to improving our systems to guarantee a reliable and trusted platform.

Read the complete report here.

Tue, Feb 7, 2023, 01:08 PM
19h earlier...

Resolved

At 3:30 PM (GMT), our team identified an issue in our FastStore WebOps, the underlying infrastructure of stores built on FastStore. This instability affected all FastStore applications. Our team mitigated the problem at 3:56 PM (GMT), and we had stores backing to normal right after the mitigation action.

Our sincere apologies for the disruption in our services today. We appreciate you bearing with us in the meantime. Once we finish a detailed public report, we will update this status with the link.

Mon, Feb 6, 2023, 05:27 PM
1h earlier...

Monitoring

The issue with FastStore WebOps is solved. All stores are fully operational. We are monitoring our systems to ensure that we fully resolved the issue.

Mon, Feb 6, 2023, 04:25 PM
15m earlier...

Identified

We identified the issue and promptly fixed it. Stores running on FastStore WebOps are back to normal. But, we are still experiencing partial disruption in loading assets from the FastStore application.

Mon, Feb 6, 2023, 04:09 PM
13m earlier...

Investigating

We’re currently investigating an outage in FastStore WebOps, and we’ll keep you posted as soon as we have more information. All other Storefronts(Portal CMS, VTEX IO Render) are fully operational.

Thanks for your patience, and we apologize for the disruption.

Mon, Feb 6, 2023, 03:56 PM