This incident has been resolved. We will provide further information in a public incident report.

Summary

On March 25, 2024, from 19:56 to 20:25 UTC, accounts experienced 404 errors, with products incorrectly being shown as unavailable.

Our global sales flow was partially affected for 29 minutes during this incident.

We apologize for any inconvenience this may have caused.

Timeline

At 19:13 UTC, a configuration repository deployment began.

At 19:56 UTC, we identified some 5xx errors and started investigating the issue. The deployment affected VTEX systems, such as catalog search, reporting that some products were unavailable in the sales flow.

At 20:15 UTC, the problematic version was rolled back, and platform behavior was re-established.

At 20:25 UTC, the incident was fully mitigated.