Issue impacting deployment availability

Incident Report for ESS (Public)

Resolved

This incident has been resolved.

Posted Jul 23, 2023 - 02:29 UTC

Monitoring

The fix has been applied and the impacted deployments are now operational. We will continue monitoring for a period of time before resolving the incident.

Posted Jul 22, 2023 - 22:59 UTC

Update

A fix has been identified and is being rolled out to the affected allocators. We will provide an update within one hour.

Posted Jul 22, 2023 - 22:12 UTC

Identified

We have identified the root cause and are working on a solution. We will provide an update within one hour.

Posted Jul 22, 2023 - 21:38 UTC

Update

We are continuing to investigate the issue. We will provide an update within an hour.

Posted Jul 22, 2023 - 20:57 UTC

Update

We are continuing to investigate the issue. We believe the impact is limited to a subset of deployments in the identified regions in AWS. We will provide an update within an hour.

Posted Jul 22, 2023 - 20:00 UTC

Investigating

We are investigating an issue impacting the availability of deployments. Currently, we believe the impact is limited to AWS regions. Impacted deployments are displaying this message in response to Elasticsearch API requests:

{
"error": {
"root_cause": [
{
"type": "master_not_discovered_exception",
"reason": null
}
],
"type": "master_not_discovered_exception",
"reason": null
},
"status": 503
}

We will provide an update within one hour.

Posted Jul 22, 2023 - 19:19 UTC

This incident affected: AWS Mumbai (ap-south-1) (Elasticsearch connectivity: AWS ap-south-1), AWS Tokyo (ap-northeast-1) (Elasticsearch connectivity: AWS ap-northeast-1), AWS Ohio (us-east-2) (Elasticsearch connectivity: AWS us-east-2), AWS N. California (us-west-1) (Elasticsearch connectivity: AWS us-west-1), AWS Oregon (us-west-2) (Elasticsearch connectivity: AWS us-west-2), AWS Frankfurt (eu-central-1) (Elasticsearch connectivity: AWS eu-central-1), AWS Ireland (eu-west-1) (Elasticsearch connectivity: AWS eu-west-1), AWS N. Virginia (us-east-1) (Elasticsearch connectivity: AWS us-east-1), AWS Singapore (ap-southeast-1) (Elasticsearch connectivity: AWS ap-southeast-1), and AWS Sydney (ap-southeast-2) (Elasticsearch connectivity: AWS ap-southeast-2).