GitHub experienced two significant incidents in November 2020 that degraded availability for issues, pull requests, and GitHub Actions services.
Incident 1: November 2 (12:00 UTC, lasting 32 minutes)
Root Cause: “The SSL certificate for *.githubassets.com expired, impacting web requests for GitHub.com UI and services.”
Contributing Factor: “There was an auto-generated issue indicating the certificate was within 30 days of expiration, but it was not addressed in time.”
Resolution: “The on-call engineer remediated it promptly” once impact was reported.
Follow-up: GitHub committed to evaluating “current processes, as well as our tooling and automation, within this area” to prevent recurrence.
Incident 2: November 27 (16:04 UTC, lasting one hour and one minute)
Root Cause: “Our service monitors detected abnormal levels of replication lag within one of our MySQL clusters affecting the GitHub Actions service.”
Status: “Due to the recency of this incident, we are still investigating the contributing factors and will provide a more detailed update in next month’s report.”
Tags
Only on this entry: ssl-certificate