No components marked as affected
Write-up published
Resolved
“Between 2:02 PM to 6:50 PM UTC, shoppers experienced degraded search experience on Stores using Intelligent Search. You can read further details here: https://io.vtex.com.br/incident-report/2022-10-12-Intelligent\_Search\_outage.pdf”
Resolved
This incident has been resolved.
Monitoring
We are reviewing and doing a double check on the new backend search. The current error rate is still very low and at nominal values. Also, we are working to normalize the indexing latency, now that should be back in 40 minutes.
Monitoring
We noticed that product indexing is affected and we expect a latency in product indexing of roughly 2 hours (from the time the product is submitted for indexing to it being available on the store). We are working to reduc/normalizing this latency, which should be ok in 2h time as well
Monitoring
While we are still doing data movement operations and using fallback search mechanisms, the current error rate is very low and at nominal values. User experience should be either ok (for stores that we already did data movement) or slightly degraded in search quality/hits (for stores that we changed the search mechanism). Overall latency and error rates should be ok for all. We are still working on our data movement operations which will take some considerably time (hours scale)
Identified
We are still working on mitigating this issue by using a different search backend.
Identified
We continue to work on sorting this issue. The impact was already minimised for a group of stores. As data movement is necessary this is taking a while. Meanwhile we are exploring further mitigating this issue by using a different search backend. This will provide a non ideal search experience while we work in re-establishing the main search mechanism
Identified
The problem has already been identified and the engineers are working on the solution
Identified
Intelligent Search outage on uncached requests since 11:20 AM BRT