Failing upgrades to Elasticsearch 6.3.0 from clusters in 5.x series

Incident Report for Elastic Cloud (Public)

Resolved

This incident has been resolved.

Posted Jun 25, 2018 - 17:20 UTC

Update

Upgrades of Elasticsearch from 5.x to 6.3.0 have be seen to fail. A fix is to be released soon, but in the meantime clusters can be recovered with these two steps:

1. Force flush the offending index
`POST //_flush?force=true`
2. Retry the cluster allocation
`POST /_cluster/reroute?retry_failed`

Posted Jun 25, 2018 - 16:36 UTC

Monitoring

We have identified the issue as an Elasticsearch issue: https://github.com/elastic/elasticsearch/issues/31482, which affects the way Cloud upgrades Elasticsearch clusters.

We're working with the Elasticsearch team to find a resolution. As a precaution, we have temporarily disabled the creation and upgrade to 6.3.0 clusters in Elastic Cloud.

If you have an affected cluster or need to provision a 6.3.0 cluster, you can get in touch with support@elastic.co so we can assist you.

Posted Jun 25, 2018 - 12:37 UTC

Update

We are investigating cases of failed upgrades from clusters running certain Elasticsearch versions in the 5.x series to version 6.3.0.

If you have a cluster that fails to start or recover some shards after this upgrade, please contact us at support@elastic.co.

Posted Jun 25, 2018 - 10:29 UTC

Investigating

We are currently investigating this issue.

Posted Jun 25, 2018 - 10:24 UTC