Today starting at around 08:00 UTC we received several user reports of issues resolving cluster urls, caused by ongoing issues with the root nameservers of the .io top-level domain. This was a global internet infrastructure issue affecting all .io domains, resulting in intermittent "hostname not known" DNS resolution errors for a small percentage of customers.
To improve global connectivity to cluster URLs, we have prepared a new domain of bonsaisearch.net for Bonsai clusters in all regions. For example, the hostname production-1234.us-east-1.bonsai.io can be changed to production-1234.us-east-1.bonsaisearch.net.
Customers are encouraged to update their application configurations to use the bonsaisearch.net hostname at their earliest convenience. Moving forward, we will continue to support existing hostnames under the bonsai.io domain, however bonsaisearch.net is now the new default and primary hostname for all clusters.
Customers affected by cached NXDOMAIN responses for the "bonsai.io" domain should update their cluster URL to use the "bonsaisearch.net" domain.
The impact has been narrowed down to the .io top level domain with domains no longer delegating out to authoritative name servers. We are working with our DNS provider AWS's Route53 to investigate.