Coveralls Status

Monitoring: Day 2 | Evening Status: Unresolved — No ETA

We want to be transparent with you: after two full days of working directly with our hosting provider’s account team, we still do not have an ETA for service restoration. Despite having internal sponsorship within our provider’s organization, we have been unable to get the traction needed to resolve what appears to have been an automated action — one that no one on their side has taken ownership of causing or fixing.

We understand the severity of this disruption. Your CI workflows depend on Coveralls, and we can see the real impact this outage is having on your teams. That is not lost on us for a single moment.

This experience has made something painfully clear: no matter how trusted or established a cloud provider may be, a single vendor dependency without a failover plan is an unacceptable risk — for us and for you. We are already evaluating backup infrastructure options to ensure that nothing like this can take our service offline this way again. We’ll share more in our full postmortem.

In the meantime, to those of you who have stayed patient and stuck with us through this — thank you. We see you, we appreciate you, and we are committed to making this right. If there is anything we can do to help in the interim, please reach out.

We will continue to post updates as we have them.

Monitoring: As of 6am PST Thu, Feb 26, we are still waiting on restoration. We thought this would be resolved by this morning. We are in touch with our account team and are doing what we can to restore service. We apologize for the ongoing inconvenience.

Monitoring: After a full day of working with our hosting provider's account team (who've been really helpful), we've discovered that this outage stems from an account restriction that, by all indications, shouldn't have happened (we believe the technical term is "snafu"). They are actively working to get it reversed.

This is not a technical failure — our infrastructure and all customer data are fully intact. Services will be restored as soon as account access is.

We're hopeful for resolution this evening or tomorrow morning. We'll have more to say once we're back online.

Monitoring: We are continuing to communicate with our hosting provider around any updates or ETA.

Monitoring: We are experiencing an outage that appears to have originated at our infrastructure provider. We are unable to access our account and are working with our account manager to resolve the issue. Unfortunately, due to the nature of the incident, we have no way to establish an ETA.

We are doing everything we can to restore service, and will update our status page as soon as we have a fix or an ETA.

Please follow along here: https://status.coveralls.io/

You can subscribe for email updates by clicking SUBSCRIBE TO UPDATES button in the top right corner of this page.

Identified: Outage will be extended for at least several more hours.

To avoid further disruption to your CI workflows, we recommend employing the fail-on-error: false input option available with all official coveralls integrations.

See: https://docs.coveralls.io/integrations#official-integrations.

Official Integration: fail-on-error option

• GitHub Action: fail-on-error: false
• CircleCI Orb: fail_on_error: false
• Coverage Reporter (CLI): --no-fail

Identified: The issue has been identified and a fix is being implemented.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently investigating this issue.

Monitoring: Day 2 | Evening Status: Unresolved — No ETA

We want to be transparent with you: after two full days of working directly with our hosting provider’s account team, we still do not have an ETA for service restoration. Despite having internal sponsorship within our provider’s organization, we have been unable to get the traction needed to resolve what appears to have been an automated action — one that no one on their side has taken ownership of causing or fixing.

We understand the severity of this disruption. Your CI workflows depend on Coveralls, and we can see the real impact this outage is having on your teams. That is not lost on us for a single moment.

This experience has made something painfully clear: no matter how trusted or established a cloud provider may be, a single vendor dependency without a failover plan is an unacceptable risk — for us and for you. We are already evaluating backup infrastructure options to ensure that nothing like this can take our service offline this way again. We’ll share more in our full postmortem.

In the meantime, to those of you who have stayed patient and stuck with us through this — thank you. We see you, we appreciate you, and we are committed to making this right. If there is anything we can do to help in the interim, please reach out.

We will continue to post updates as we have them.

Monitoring: As of 6am PST Thu, Feb 26, we are still waiting on restoration. We thought this would be resolved by this morning. We are in touch with our account team and are doing what we can to restore service. We apologize for the ongoing inconvenience.

Monitoring: After a full day of working with our hosting provider's account team (who've been really helpful), we've discovered that this outage stems from an account restriction that, by all indications, shouldn't have happened (we believe the technical term is "snafu"). They are actively working to get it reversed.

This is not a technical failure — our infrastructure and all customer data are fully intact. Services will be restored as soon as account access is.

We're hopeful for resolution this evening or tomorrow morning. We'll have more to say once we're back online.

Monitoring: We are continuing to communicate with our hosting provider around any updates or ETA.

Monitoring: We are experiencing an outage that appears to have originated at our infrastructure provider. We are unable to access our account and are working with our account manager to resolve the issue. Unfortunately, due to the nature of the incident, we have no way to establish an ETA.

We are doing everything we can to restore service, and will update our status page as soon as we have a fix or an ETA.

Please follow along here: https://status.coveralls.io/

You can subscribe for email updates by clicking SUBSCRIBE TO UPDATES button in the top right corner of this page.

Identified: Outage will be extended for at least several more hours.

To avoid further disruption to your CI workflows, we recommend employing the fail-on-error: false input option available with all official coveralls integrations.

See: https://docs.coveralls.io/integrations#official-integrations.

Official Integration: fail-on-error option

• GitHub Action: fail-on-error: false
• CircleCI Orb: fail_on_error: false
• Coverage Reporter (CLI): --no-fail

Identified: The issue has been identified and a fix is being implemented.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently investigating this issue.