Splunk Observability Cloud US1 Status

Resolved: The system has recovered and we ingesting archived data with no delay.

Monitoring: To accelerate the recovery process, we are currently adding additional capacity to our clusters. Our teams remain actively engaged and are continuing to monitor system performance as it stabilizes.

Monitoring: We are continuing to monitor the system to ensure ongoing stability. Our teams remain actively engaged in observing performance metrics to confirm that all operations are functioning as expected.

Monitoring: We continue monitoring to ensure full service functionality.

Monitoring: We have successfully restored datapoint ingestion and are seeing steady progress as the data backlog is processed. We are actively monitoring this process to ensure full service functionality returns as quickly as possible.

Monitoring: The system is showing steady progress in its recovery. Our teams remain actively engaged, closely observing system metrics to ensure continued stability

Monitoring: The system is continuing to recover, and we are actively monitoring it to ensure continued stability.

Identified: As of 7:48 AM PDT, we are no longer observing any data being dropped. The system is continuing to recover, and we are actively monitoring it to ensure continued stability.

Identified: Beginning at 5:50 AM PDT, datapoint ingestion for the archival tier was impacted, resulting in a small number of datapoints not being accepted. The impact was limited to the archival tier. Detectors and charts were not affected.

We have implemented remediation and are actively monitoring the system to ensure continued stability. We will provide updates every 15 minutes.

Resolved: The issue has been resolved, and datapoint ingestion is operating normally. We have confirmed that the system remains stable following remediation.

This incident is now resolved.

Monitoring: Between 8:52 PDT and 9:47 PDT, a small number of data points for new time series were not accepted. As a result, new time series may have experienced a delay before becoming available in charting and detectors.

Remediation is in place, and we are continuing to monitor the system to ensure stability.

Investigating: Datapoint ingest is affected and we are dropping datapoints some datapoints from 8:52 PDT. We are investigating and will provide an update every 15 mins.