High error rate on VTEX IO
Resolved

This incident has been resolved. You can read the incident report here: https://io.vtex.com.br/incident-report/2022-02-03-High_error_rates_on_VTEX_IO.pdf

Fri, Feb 4, 2022, 04:07 AM
(3 years ago)
·
Affected components

No components marked as affected

Updates

Resolved

This incident has been resolved. You can read the incident report here: https://io.vtex.com.br/incident-report/2022-02-03-High_error_rates_on_VTEX_IO.pdf

Fri, Feb 4, 2022, 04:07 AM

Monitoring

We can confirm the errors stopped at 21:59 UTC ir our platform. We are monitoring the result of our actions and operating when necessary.

Fri, Feb 4, 2022, 01:40 AM(2 hours earlier)

Identified

An operation to fully restore VTEX IO to get out of the mitigation state was successfully applied to a minority part of VTEX IO.

Thu, Feb 3, 2022, 11:47 PM(1 hour earlier)

Identified

We've applied some operations in our infrastructure in order to mitigate the issue and errors are no more a concern.

Thu, Feb 3, 2022, 10:45 PM(1 hour earlier)

Identified

We have identified the root cause of the increased error rates. We are working to resolve this issue.

Thu, Feb 3, 2022, 10:35 PM

Investigating

We are investigating increased error rates in VTEX IO.

Thu, Feb 3, 2022, 09:43 PM(52 minutes earlier)