GitGuardian Services Status

Resolved: We monitored this over the weekend and did various checks today, and we believe we have resolved the incident. Customer Support has already contacted all customers who were impacted.

Monitoring: Technical fixes are deployed, and all services are stable. We're maintaining monitoring through the weekend before final incident closure on Monday.

Detailed customer communication and a full postmortem will follow early next week. Any weekend developments will be posted immediately.

Monitoring: Latest Progress:

We have successfully deployed further performance optimizations and fixes to production.

Current Status:

• Database performance remains stabilized with the new optimizations
• Integration services are being closely monitored
• Real-time monitoring is active to ensure continued stability

Next Steps:

Our engineering team will continue monitoring system performance over the next few hours to confirm the effectiveness of our fixes and carry out various tests. We prepared additional safeguards if needed.

We will provide additional updates as soon as our monitoring confirms that it is stable across all services.

Monitoring: Current Status:

We have successfully deployed an initial fix that has stabilized our systems. Database performance has returned to normal levels, and we are no longer experiencing the issues that caused integration disruptions.

Next Steps:

Our engineering team is deploying additional performance optimizations within the next 1-2 hours. These enhancements will provide long-term stability and allow us to safely restore all affected functionality.

Monitoring: System performance metrics have returned to baseline levels. Our team maintains active monitoring to ensure continued stability throughout the optimization deployment.

We expect to provide another update within 2-3 hours following the completion of our performance enhancements. Thank you for your continued patience as we work toward full resolution.

Monitoring: Database Performance Issue Mitigated

We successfully mitigated a database performance issue that was causing service slowdowns.

Impact:

• Check runs experienced temporary slowdowns
• A small number of check runs timed out during the incident. If any check run was impacted, restart it and it should work normally
• There was no widespread service disruption

The underlying database performance issue has been mitigated

All services are operating normally

The issue is now resolved, and we continue monitoring system performance.

Identified: The issue has been identified and a fix is being implemented.

Investigating: We are currently experiencing performance issues affecting multiple GitGuardian services. Our engineering team is actively investigating and working to resolve these issues as quickly as possible.

Current Impact:

• Dashboard may load slowly or fail to load intermittently
• Public API response times are degraded, which may cause slower performance in ggshield and other integrations

What we're doing: Our engineering team has been alerted and is actively investigating the root cause. We will provide updates as significant developments occur.

Resolved: All GitGuardian services have been fully restored and are operating normally. Real-time scanning has returned to normal service levels, and our monitoring confirms stable performance across all components.

We implemented mitigation measures to address the immediate symptoms while continuing to investigate the underlying root cause.

What Happened: A database performance issue caused automatic removal of GitGuardian cloud integrations, specifically affecting Jira Cloud and Confluence Cloud installations.

Impact:

• Jira Cloud integrations: Both scanning sources and notification integrations were uninstalled
• Confluence Cloud integrations: Source scanning integrations were uninstalled
• Service availability: Real-time monitoring and GitGuardian features are currently unavailable for affected installations

Resolution option: Customers can reinstall integrations to immediately restore functionality

Current Status:

• We are compiling a comprehensive list of all impacted customers
• Our customer support team will proactively reach out to affected customers with detailed guidance

Important: While reinstallation is possible now, there remains a small risk of recurrence until we fully resolve the root cause.

Additional proactive monitoring has been implemented to prevent future occurrences.

Next Steps:

• Complete root cause analysis and full postmortem documentation
• Notify all customers once the underlying issue is definitively resolved
• Customers may choose to wait for our all-clear notification before reinstalling

We sincerely apologize for this disruption and will keep you updated as our investigation progresses.

Monitoring: We are continuing to monitor our services as they return to normal operations.

Notice: Some check runs may have failed due to timeout errors during the incident. These can be safely restarted manually and should now complete successfully.

We will provide a final update once monitoring confirms no further delays on real-time scanning services.

Monitoring: We have successfully implemented a resolution that has restored normal operations for the majority of affected services.

Service Status: Dashboard: Performance has been fully restored across all regions Public API: Response times have returned to baseline levels Check-runs: Processing has resumed normal operational parameters Real-time scanning: Experiencing residual delays; remediation in progress

Next Steps: Our engineering team is actively addressing the remaining performance impact on real-time scanning services.

Identified: The issue has been identified and we are taking remedial actions.

Investigating: We are continuing to investigate this issue.

Investigating: Some GitHub check-runs may be failing, leading to blocked PRs

Investigating: We are currently experiencing performance issues affecting multiple GitGuardian services. Our engineering team is actively investigating and working to resolve these issues as quickly as possible.

Current Impact:

• Dashboard may load slowly or fail to load intermittently
• Public API response times are degraded, which may cause slower performance in ggshield and other integrations
• Real-time event processing is experiencing delays, resulting in delayed incident notifications

What we're doing: Our engineering team has been alerted and is actively investigating the root cause. We will provide updates as significant developments occur.

We apologize for any inconvenience this may cause and appreciate your patience as we work to restore full service performance.

Resolved: We monitored this over the weekend and did various checks today, and we believe we have resolved the incident. Customer Support has already contacted all customers who were impacted.

Monitoring: Technical fixes are deployed, and all services are stable. We're maintaining monitoring through the weekend before final incident closure on Monday.

Detailed customer communication and a full postmortem will follow early next week. Any weekend developments will be posted immediately.

Monitoring: Latest Progress:

We have successfully deployed further performance optimizations and fixes to production.

Current Status:

• Database performance remains stabilized with the new optimizations
• Integration services are being closely monitored
• Real-time monitoring is active to ensure continued stability

Next Steps:

Our engineering team will continue monitoring system performance over the next few hours to confirm the effectiveness of our fixes and carry out various tests. We prepared additional safeguards if needed.

We will provide additional updates as soon as our monitoring confirms that it is stable across all services.

Monitoring: Current Status:

We have successfully deployed an initial fix that has stabilized our systems. Database performance has returned to normal levels, and we are no longer experiencing the issues that caused integration disruptions.

Next Steps:

Our engineering team is deploying additional performance optimizations within the next 1-2 hours. These enhancements will provide long-term stability and allow us to safely restore all affected functionality.

Monitoring: System performance metrics have returned to baseline levels. Our team maintains active monitoring to ensure continued stability throughout the optimization deployment.

We expect to provide another update within 2-3 hours following the completion of our performance enhancements. Thank you for your continued patience as we work toward full resolution.

Monitoring: Database Performance Issue Mitigated

We successfully mitigated a database performance issue that was causing service slowdowns.

Impact:

• Check runs experienced temporary slowdowns
• A small number of check runs timed out during the incident. If any check run was impacted, restart it and it should work normally
• There was no widespread service disruption

The underlying database performance issue has been mitigated

All services are operating normally

The issue is now resolved, and we continue monitoring system performance.

Identified: The issue has been identified and a fix is being implemented.

Investigating: We are currently experiencing performance issues affecting multiple GitGuardian services. Our engineering team is actively investigating and working to resolve these issues as quickly as possible.

Current Impact:

• Dashboard may load slowly or fail to load intermittently
• Public API response times are degraded, which may cause slower performance in ggshield and other integrations

What we're doing: Our engineering team has been alerted and is actively investigating the root cause. We will provide updates as significant developments occur.

Resolved: All GitGuardian services have been fully restored and are operating normally. Real-time scanning has returned to normal service levels, and our monitoring confirms stable performance across all components.

We implemented mitigation measures to address the immediate symptoms while continuing to investigate the underlying root cause.

What Happened: A database performance issue caused automatic removal of GitGuardian cloud integrations, specifically affecting Jira Cloud and Confluence Cloud installations.

Impact:

• Jira Cloud integrations: Both scanning sources and notification integrations were uninstalled
• Confluence Cloud integrations: Source scanning integrations were uninstalled
• Service availability: Real-time monitoring and GitGuardian features are currently unavailable for affected installations

Resolution option: Customers can reinstall integrations to immediately restore functionality

Current Status:

• We are compiling a comprehensive list of all impacted customers
• Our customer support team will proactively reach out to affected customers with detailed guidance

Important: While reinstallation is possible now, there remains a small risk of recurrence until we fully resolve the root cause.

Additional proactive monitoring has been implemented to prevent future occurrences.

Next Steps:

• Complete root cause analysis and full postmortem documentation
• Notify all customers once the underlying issue is definitively resolved
• Customers may choose to wait for our all-clear notification before reinstalling

We sincerely apologize for this disruption and will keep you updated as our investigation progresses.

Monitoring: We are continuing to monitor our services as they return to normal operations.

Notice: Some check runs may have failed due to timeout errors during the incident. These can be safely restarted manually and should now complete successfully.

We will provide a final update once monitoring confirms no further delays on real-time scanning services.

Monitoring: We have successfully implemented a resolution that has restored normal operations for the majority of affected services.

Service Status: Dashboard: Performance has been fully restored across all regions Public API: Response times have returned to baseline levels Check-runs: Processing has resumed normal operational parameters Real-time scanning: Experiencing residual delays; remediation in progress

Next Steps: Our engineering team is actively addressing the remaining performance impact on real-time scanning services.

Identified: The issue has been identified and we are taking remedial actions.

Investigating: We are continuing to investigate this issue.

Investigating: Some GitHub check-runs may be failing, leading to blocked PRs

Investigating: We are currently experiencing performance issues affecting multiple GitGuardian services. Our engineering team is actively investigating and working to resolve these issues as quickly as possible.

Current Impact:

• Dashboard may load slowly or fail to load intermittently
• Public API response times are degraded, which may cause slower performance in ggshield and other integrations
• Real-time event processing is experiencing delays, resulting in delayed incident notifications

What we're doing: Our engineering team has been alerted and is actively investigating the root cause. We will provide updates as significant developments occur.

We apologize for any inconvenience this may cause and appreciate your patience as we work to restore full service performance.