Intermittent DNS resolution errors impacting .io TLD
Incident Report for Bonsai

Today starting at around 08:00 UTC we received several user reports of issues resolving cluster urls, caused by ongoing issues with the root nameservers of the .io top-level domain. This was a global internet infrastructure issue affecting all .io domains, resulting in intermittent "hostname not known" DNS resolution errors for a small percentage of customers.

To improve global connectivity to cluster URLs, we have prepared a new domain of for Bonsai clusters in all regions. For example, the hostname can be changed to

Customers are encouraged to update their application configurations to use the hostname at their earliest convenience. Moving forward, we will continue to support existing hostnames under the domain, however is now the new default and primary hostname for all clusters.

Posted Oct 28, 2016 - 10:45 UTC

We have been updated that .io TLD service has now been fully restored.
Posted Oct 28, 2016 - 10:44 UTC
A fix has been implemented and we are monitoring the results.

Customers affected by cached NXDOMAIN responses for the "" domain should update their cluster URL to use the "" domain.
Posted Oct 28, 2016 - 09:47 UTC
We have prepared a fallback domain of "" for use with cluster URLs. All impacted customers are advised to update their cluster URL to use this hostname. For example, `` can be changed to ``.
Posted Oct 28, 2016 - 09:35 UTC
We have confirmed the root cause is originating at the .io top level domain. We are preparing a hotfix to provide an alternative domain for affected customers to switch to.
Posted Oct 28, 2016 - 09:06 UTC
We're aware of issues with DNS and are investigating. Some customers may be seeing errors when trying to use bonsai.

The impact has been narrowed down to the .io top level domain with domains no longer delegating out to authoritative name servers. We are working with our DNS provider AWS's Route53 to investigate.
Posted Oct 28, 2016 - 09:01 UTC