This incident has been resolved. We will provide further information in a public incident report.
Summary
On Mar 12, 2025, from 09:13 to 12:03 UTC, most merchants were unable to login to their Admin and some shoppers experienced errors navigating or placing orders. This was triggered by a downstream dependency failure in the VTEX IO platform.
Our global sales flow was unaffected during this incident. Some accounts may have experienced sales below forecasted levels due to customized Checkout flows and payment connectors depending on VTEX IO infrastructure.
We apologize for any inconvenience this may have caused.
Timeline
At 09:13 UTC, our platform health monitoring indicated increased errors in VTEX IO.
At 09:17 UTC, our incident response team was notified of the issue.
At 11:43 UTC, we identified an issue in a downstream dependency used by a VTEX IO authZ service.
At 11:54 UTC, we performed a mitigation action and the platform started to gradually recover.
At 12:03 UTC, the incident was fully mitigated.
