Degraded Admin Environment

Incident Report for VTEX

Postmortem

Between 11h00 UTC and 12h00 UTC, we started receiving error reports related to VTEX Admin access caused by a degradation in the service routing layer of VTEX IO. A mismanagement of routing configuration caused part of the infrastructure to fail. Once the configuration was fixed, the platform returned to normal behavior.

To be the trusted partner to your success, our team is working on follow-up actions to ensure that this incident does not happen again and that we identify and recover from future incidents faster. We are committed to improving our systems to guarantee a reliable and trusted experience. You can read the full report here.

Posted May 02, 2023 - 16:32 GMT-03:00

Resolved

We have applied mitigation actions and all operations are back to normal. We rolled back a malformed configuration that prevented our traffic from scaling up. More information will be available in an incident report.

Posted Apr 14, 2023 - 09:37 GMT-03:00

Identified

We are aware of the issue with Administrative Environment that is impacting the access to our Admin panels. This issue was caused by a change on our platform, and we are working to fix this issue. We'll get back to you with more information.

Posted Apr 14, 2023 - 09:02 GMT-03:00

Investigating

We’re currently investigating an issue in Administrative Modules, and we’ll keep you posted as soon as we have more information. Thanks for your patience, and we apologize for the disruption.

Posted Apr 14, 2023 - 08:45 GMT-03:00

This incident affected: Administrative Environment.