Search connection outage
Updates
The underlying incident has been resolved. Some small number of clusters still experiencing 502 errors due to cached or otherwise incorrect DNS resolution, and should either refresh their DNS caches or replace "bonsaisearch.net" in their cluster URL with the fallback of "bonsai.io". For example, "oak-123.us-east-1.bonsaisearch.net" becomes "oak-123.us-east-1.bonsai.io". Anyone experiencing lingering errors is encourage to reach out to [email protected] for individual assistance.
Fixes were rolled out, resolving the incident for most customers. Some customers may still be impacted due to a DNS issue. For those impacted, update your connection URL to use bonsai.io instead of bonsaisearch.net while we work to resolve.
The service has been largely recovered, but with isolated cases of clusters experiencing persistent HTTP 502 errors. These seem to be an artifact of DNS resolving to outdated load balancer IPs. The workaround we recommend is to replace the top level "bonsaisearch.net" in your cluster's URL with "bonsai.io" — for example, "oak-123.us-east-1.bonsaisearch.net" becomes "oak-123.us-east-1.bonsai.io".
As of 07:07 UTC, fixes were rolled out. Lingering issues on a small number of clusters are still present, which are being addressed.
A solution was identified and fixes are rolling out
Bonsai is experiencing a system-wide incident for all search cluster connectivity