CircleCI Status

Resolved: This incident has been resolved.

Monitoring: Error rates viewing orgs are recovering. We will continue to monitor to verify the fix resolves the issues.

Identified: We have identified the cause of errors accessing orgs. We are implementing a fix, and will monitor for recovery as the change rolls out.

Investigating: We are continuing to investigate reports that some customers cannot access their organizations in CircleCI.

Our investigation continues to point to an authorization issue between CircleCI and GitHub. Some affected users have received a 403 from GitHub indicating the resource is SAML protected and the OAuth token may not be authorized.

We previously shared re-authenticating as a potential workaround. We have since received reports that this didn't restore access for all affected customers, so we don't consider it a reliable resolution at this time.

Our engineering team is continuing to investigate this to determine the cause and next steps. We will post another update as soon as we have more information.

We apologize for the continued disruption and appreciate your patience.

Investigating: We are still investigating reports that some customers cannot access their organizations in CircleCI.

Early findings suggest the problem may be related to GitHub authorization, where affected users receive a 403 from GitHub indicating the resource is SAML protected and the OAuth token may not be authorized.

We have not yet confirmed this as the root cause. As a potential workaround, some users may be able to restore access by performing a re-authentication: https://support.circleci.com/hc/en-us/articles/360051228052-How-to-Perform-a-Re-Authentication

We will post another update once we have more certainty.

Investigating: We are continuing to investigate reports that some customers are unable to access their organizations in CircleCI. Those affected may see errors when loading their organizations.

Our engineering team are still working to identify the cause. We will share another update as soon as we have more information.

Investigating: We are investigating reports that some customers are unable to access their organizations in CircleCI. Those affected may see errors when loading their organizations.

Our engineering team is actively investigating.

Resolved: We are concluding this incident based on a confirmed fix with our infrastructure provider and successful tests. For any customers seeing failed runs, please run again for a successful connection.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Due to continued reports of intermittent failures and a lack of sustained improvement while in monitoring status, this incident is elevated back to investigating to escalate with our infrastructure provider.

Monitoring: We are continuing to monitor. We still observe a low percentage of intermittent failures. The failure rate is trending down as we continue to monitor. We advise customers to contact CircleCI Support if they see a change in degraded services and increased failure rate.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Our team is moving this incident to Monitoring status based on significant improvement seen in our tests and low error rate. We are seeing improvement in network routing for Mac jobs and are actively engaged with our infrastructure provider. If your Mac jobs experience network-related failures or degraded service please reach out to CircleCI support.

Identified: We are seeing improvement in Mac job network routing.

We will continue to monitor for signs of recovery.

Identified: Mac jobs are still seeing intermittent failures with network routing.

We are working with our infrastructure provider to address the networking issues.

Identified: We have implemented a change to address the routing issues and are monitoring the results.

Investigating: What's happening Our engineers are investigating network routing issues for some Mac jobs.

What's impacted Mac jobs that make outbound requests, including checking out from GitHub.