Log4j Zero-Day RCE
Incident
Report for Bonsai
Resolved
We have received independent confirmation via Elastic, Inc. that Elasticsearch is not vulnerable to RCE due to its use of the Java Security Manager. Our team will finish rolling out mitigations, but otherwise are standing down on updates here, pending any new developments.
Update
All relevant versions for new cluster deployments have been updated, and we have re-enabled Sandbox cluster creation. We appreciate the patience from everyone who was stuck at the last step of new account creation this afternoon!
Update
We are continuing to make steady progress in rolling out updates, with all of ES 5.x clusters updated, approximately 80% of ES 6.x, and over 50% of ES 7.x clusters updated.
Update
Our team is continuing to roll out updates and making steady progress.
We’ve determined that a configuration based mitigation is not available in some early versions of Elasticsearch 5.x. Some customer clusters running on early versions of Elasticsearch 5.x have been upgraded to Elasticsearch 5.6.16.
Updates to Elasticsearch 6.x, 7.x, and OpenSearch 1.x are still under way.
We’ve determined that a configuration based mitigation is not available in some early versions of Elasticsearch 5.x. Some customer clusters running on early versions of Elasticsearch 5.x have been upgraded to Elasticsearch 5.6.16.
Updates to Elasticsearch 6.x, 7.x, and OpenSearch 1.x are still under way.
Update
At this time we're reasonably confident that Bonsai is not susceptible to the Remote Code Execution in this vulnerability.
However, we believe certain combinations of Java, Elasticsearch, and log4j can plausibly execute a remote ping. Out of an abundance of caution, we’re moving forward with a rollout of configuration mitigations.
For those following along and interested in the details of this incident, there are different combinations of the JDK version alongside the version of Log4j that are relevant to reproducibility. Per the security update from Apache (https://logging.apache.org/log4j/2.x/security.html)
>>>
Java 8u121 (see https://www.oracle.com/java/technologies/javase/8u121-relnotes.html) protects against RCE by defaulting "com.sun.jndi.rmi.object.trustURLCodebase" and "com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".
<<<
Java 8u121 was released in January 2017, and Bonsai is running with newer versions of Java than that across the board. We believe this default has made our systems safe by default from this particular vulnerability.
However, we believe certain combinations of Java, Elasticsearch, and log4j can plausibly execute a remote ping. Out of an abundance of caution, we’re moving forward with a rollout of configuration mitigations.
For those following along and interested in the details of this incident, there are different combinations of the JDK version alongside the version of Log4j that are relevant to reproducibility. Per the security update from Apache (https://logging.apache.org/log4j/2.x/security.html)
>>>
Java 8u121 (see https://www.oracle.com/java/technologies/javase/8u121-relnotes.html) protects against RCE by defaulting "com.sun.jndi.rmi.object.trustURLCodebase" and "com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".
<<<
Java 8u121 was released in January 2017, and Bonsai is running with newer versions of Java than that across the board. We believe this default has made our systems safe by default from this particular vulnerability.
Update
We have temporarily disabled creation of Sandbox clusters pending updates to the underlying services.
Identified
Our engineers have identified the services within our platform which may be affected, however have not been able to reproduce the vulnerability. Out of an abundance of caution we are proceeding to roll out additional safeguards in the underlying service configurations.
Update
We are continuing to investigate this issue.
Investigating
The team is currently investigating the issue.