Connectivity issues affecting some <=1.7.1-clusters

Incident Report for Elastic Cloud (Public)

Postmortem

A detailed root cause/impact-analysis is available at https://www.elastic.co/blog/elastic-cloud-outage-april-2016

Posted Apr 22, 2016 - 02:25 UTC

Resolved

Any support issues should be directed at https://discuss.elastic.co/c/cloud.

Posted Apr 11, 2016 - 02:41 UTC

Monitoring

We've re-enabled cluster-modification/-creation, and are progressing through upgrading clusters from 1.7.1 to 1.7.5.

Posted Apr 10, 2016 - 22:10 UTC

Update

Most affected clusters are routing while we still work on this issue.

Posted Apr 10, 2016 - 21:46 UTC

Identified

A bug present in clusters running on 1.7.1 is preventing them from properly re-joining our coordination system (ZooKeeper) upon certain connectivity issues.

We're working on steps necessary to bring back up the infrastructure necessary to upgrade all these clusters, as we're suspecting that to be the root cause of problems in the first place.

Posted Apr 10, 2016 - 20:54 UTC

Investigating

We are currently investigating this issue.

Posted Apr 10, 2016 - 20:18 UTC