In June, GitHub experienced two incidents that resulted in degraded performance across services.
Incident 1: June 5, 2024 (17:05–19:27 UTC, lasting 142 minutes)
Affected Service: GitHub Issues.
Impact: Events related to projects were not displayed on issue timelines, specifically failing to show when issues were added to/removed from projects or when statuses changed.
Root Cause: A scheduled secret rotation resulted in one of the configured services using old expired secrets. A bug during service configuration cleanup prevented proper credential updates.
Resolution: The team fixed the service configuration; the simplified configuration will help avoid similar incidents in the future.
Incident 2: June 27, 2024 (20:39–21:37 UTC, lasting 58 minutes)
Affected Service: GitHub Migration.
Impact: All in-progress migrations failed. New migrations were paused to prevent additional failures until resolution.
Root Cause: An invalid infrastructure credential required manual intervention.
Resolution: First responders identified and corrected the issue, then resumed queued migrations.
Preventive Measures: GitHub plans to address specific gaps in monitoring and alerting for infrastructure credentials going forward.