Coveralls Status

Resolved: Build times are normal for all new builds. Background queues have been cleared of 99% of backlogged jobs; the only ones that remain are for large repos (>5K source files), which should clear in the next 30-45 minutes depending on size.

No further effects on latency expected.

Closing this incident.

Monitoring: Update:

• Normal build times have been recovered for all new jobs.
• Backlogged jobs are 80% cleared (20% remaining).

We will continue clearing backlogged jobs until cleared and monitoring for any further issues.

Monitoring: We had an incident overnight that entailed several server outages affecting APAC and EU users. The impact has been latency in recent build times for those users. We have resolved the issue and are scaling to clear backlogged background jobs. US Central and West users should not be affected.

Resolved: We had to pause some queues to recover performance for new jobs, but will clear those as soon as we've recovered normal build times for new jobs (ETA: 15-min).

If you are a user in APAC or EU, you may have had your jobs paused. One repo in particular has represented 90% overnight workload. We will reach out to that user.

Monitoring: We are continuing to monitor as we scale to clear backlogged jobs.

Identified: We are continuing to experience elevated latency in APAC and EU.

Resolved: Build times are normal for all new builds. Background queues have been cleared of 99% of backlogged jobs; the only ones that remain are for large repos (>5K source files), which should clear in the next 30-45 minutes depending on size.

No further effects on latency expected.

Closing this incident.

Monitoring: Update:

• Normal build times have been recovered for all new jobs.
• Backlogged jobs are 80% cleared (20% remaining).

We will continue clearing backlogged jobs until cleared and monitoring for any further issues.

Monitoring: We had an incident overnight that entailed several server outages affecting APAC and EU users. The impact has been latency in recent build times for those users. We have resolved the issue and are scaling to clear backlogged background jobs. US Central and West users should not be affected.

Resolved: We had to pause some queues to recover performance for new jobs, but will clear those as soon as we've recovered normal build times for new jobs (ETA: 15-min).

If you are a user in APAC or EU, you may have had your jobs paused. One repo in particular has represented 90% overnight workload. We will reach out to that user.

Monitoring: We are continuing to monitor as we scale to clear backlogged jobs.

Identified: We are continuing to experience elevated latency in APAC and EU.