The June 3 incident report is so terse and opaque (featuring an incomplete sentence, even). The outage didn’t seem minor and has probably affected internal DNS / 6pn, and logging. At least one user here reported a lost db instance (though, it is uncertain if the incident played a part).
June 3, 2022
Identified - The issue has been identified and a fix is being implemented.
Jun 2, 16:39 CDTInvestigating - We are investigating logging failures. These are preventing logs from reaching log shipper apps, as well as
Jun 2, 15:30 CDT
versus
Jun 2, 2022
Update - We are continuing to monitor for any further issues.
Jun 2, 13:12 CDTMonitoring - We restored the Consul cluster and are monitoring individual services. Most service is restored, but some lingering issues may persist.
Jun 2, 12:30 CDTIdentified - Our Consul servers are in out-of-memory loops. We are working to recover them. Consul issues can cause problems with internal DNS resolution, deploys, and API access. We are working to recover the servers.
Jun 2, 12:19 CDT