The underlying issue has been mitigated, and we can see DNS resolution times have been restored to expected levels.
Posted Jul 28, 2020 - 12:21 UTC
Monitoring
We are seeing a significant improvement of internal DNS resolution latency, following the mitigation efforts of our cloud provider. Monitoring of the situation will continue, with an update being provided within an hour.
Posted Jul 28, 2020 - 11:41 UTC
Investigating
We have been alerted to an ongoing issue with our cloud provider internal DNS infrastructure. https://status.aws.amazon.com/. This may impact certain functionality on our platform that relies on this backend service.
This is currently restricted to deploymnets in our AWS us-east-1 region. Connectivity between deployments components may experience problems, resulting in higher latency for connections.
We are continuing to monitor the situation, and determine the scope of impact.
Updates will be provided within the hour.
Posted Jul 28, 2020 - 10:50 UTC
This incident affected: AWS N. Virginia (us-east-1) (Deployment logging: AWS us-east-1, Deployment metrics: AWS us-east-1, Elasticsearch connectivity: AWS us-east-1, Kibana connectivity: AWS us-east-1, APM connectivity: AWS us-east-1).