Instances unexpectedly replaced during cluster restarts and deployment updates in specific AWS regions
Incident Report for ESS (Public)
Resolved
There have been no further issues associated with this incident. Please reach out to support if you believe your deployment has been effected.
Posted Feb 21, 2022 - 04:03 UTC
Monitoring
The fix has been implemented across the affected regions for the last remaining instance types & we're monitoring for any other issues related to this issue. We'll update again in 4 hours.
Posted Feb 21, 2022 - 00:07 UTC
Update
We have implemented fixes for this issue for all instance configuration types except r4 and classic instances. We are continuing to work on fixes for the remaining affected instance types.
Posted Feb 19, 2022 - 12:18 UTC
Update
We have implemented a fix for this issue for aws.data.highio.i3 and aws.data.highstorage.d2/d3/i3en Elasticsearch instances (in hot/warm/cold data tiers). We are continuing to work on fixes for the remaining affected instance types.
Posted Feb 19, 2022 - 04:42 UTC
Identified
We are aware of an issue causing instances to be unexpectedly replaced during cluster restarts and deployment updates for deployments created before October 11, 2021 in the following AWS regions: Ireland, N. Virginia, N. California, Oregon, Sao Paulo, Singapore, Sydney, and Tokyo. We are currently working a fix for this issue
Posted Feb 19, 2022 - 00:28 UTC
This incident affected: AWS Tokyo (ap-northeast-1) (Deployment orchestration (Create/Edit/Restart/Delete): AWS ap-northeast-1), AWS Ireland (eu-west-1) (Deployment orchestration (Create/Edit/Restart/Delete): AWS eu-west-1), AWS Sydney (ap-southeast-2) (Deployment orchestration (Create/Edit/Restart/Delete): AWS ap-southeast-2), AWS Oregon (us-west-2) (Deployment orchestration (Create/Edit/Restart/Delete): AWS us-west-2), AWS N. California (us-west-1) (Deployment orchestration (Create/Edit/Restart/Delete): AWS us-west-1), AWS São Paulo (sa-east-1) (Deployment orchestration (Create/Edit/Restart/Delete): AWS sa-east-1), AWS Singapore (ap-southeast-1) (Deployment orchestration (Create/Edit/Restart/Delete): AWS ap-southeast-1), and AWS N. Virginia (us-east-1) (Deployment orchestration (Create/Edit/Restart/Delete): AWS us-east-1).