LaunchDarkly Status

Resolved: We have validated a fix to resolve all remaining experiments with inaccurate sample ratio mismatches (SRMs) or empty experiment results. The vast majority of experiments and experimentation customers were not impacted.

Monitoring: We've released a fix to prevent this issue in new experiments and iterations. We're working to resolve this issue for experiment data recently processed.

Identified: We've identified the issue and are implementing a fix for this behavior.

Investigating: We are aware of an issue affecting some customers that may result in sample ratio mismatches (SRMs) or empty experiment results if flag variations are edited between experiment iterations.

Resolved: Experimentation results processing and data warehouse export was caught up as of 12:28 pm PT.

Monitoring: We have implemented a fix and are catching up on backlogged experimentation results and data export. We estimate that we will be fully caught up in about 8 hours.

Identified: We've identified the root cause of the experimentation results processing and data export delays and are working on implementing a fix.

Investigating: We are investigating an issue causing delayed experimentation results processing and delayed experimentation data warehouse export.

Resolved: We have validated a fix to resolve all remaining experiments with inaccurate sample ratio mismatches (SRMs) or empty experiment results. The vast majority of experiments and experimentation customers were not impacted.

Monitoring: We've released a fix to prevent this issue in new experiments and iterations. We're working to resolve this issue for experiment data recently processed.

Identified: We've identified the issue and are implementing a fix for this behavior.

Investigating: We are aware of an issue affecting some customers that may result in sample ratio mismatches (SRMs) or empty experiment results if flag variations are edited between experiment iterations.

Resolved: Experimentation results processing and data warehouse export was caught up as of 12:28 pm PT.

Monitoring: We have implemented a fix and are catching up on backlogged experimentation results and data export. We estimate that we will be fully caught up in about 8 hours.

Identified: We've identified the root cause of the experimentation results processing and data export delays and are working on implementing a fix.

Investigating: We are investigating an issue causing delayed experimentation results processing and delayed experimentation data warehouse export.

Resolved: We've fully caught up on the metrics data backlog.

Monitoring: We're catching up on backlogged metric data in EU and are monitoring progress. No metric data has been lost.

Identified: We've identified that metrics ingest was delayed between 11:11 - 11:27am ET. No metric data has been lost. We're working on catching up on the delayed data.

Investigating: We are investigating an issue causing under-reporting of customer metrics in the EU region.

Resolved: We've caught up on the streaming data export backlog.

Monitoring: We've identified latency in Pub/Sub export that were causing streaming data export delays that has since resolved. We're catching up on the export backlog now.

Investigating: We are investigating a delay with streaming data export delivery. No data is lost, but customers using streaming data export may notice a multi-minute lag in receiving data.

Resolved: Observability alerts are functioning and delayed alerts have been redelivered.

Monitoring: A fix has been deployed for Observability alerting and we are working to redeliver delayed notifications.

Identified: We've identified the root cause delaying Observability alert notifications and are deploying a fix. All other notification types are operational.

Investigating: We are currently investigating an issue prevent Observability alert notifications from being delivered.