Temporal Status

Resolved: API latencies have recovered and remained stable since the observed periods of elevated latency.

Investigating: We are investigating short periods of elevated API latency that occurred between 11:58–11:59 UTC and 12:17–12:19 UTC.

We will provide an update as soon as it becomes available.

Investigating: We are currently investigating elevated API latencies in GCP us-east4

Resolved: We have continued to see normal activity in this region with latencies holding at pre-incident levels. This incident is now resolved.

Monitoring: Our engineering has implemented a fix and we are seeing normal service return to this region. We are continuing to monitor.

Identified: As of 17:53 UTC our engineering team has identified the issue and is actively working to restore normal service. We will provide another update as more information becomes available.

Investigating: We are continuing to investigate this issue.

Investigating: Some customers in AWS eu-west-1 may experience elevated latencies starting 13:47 UTC time.

Resolved: This incident has been resolved.

Monitoring: The affected infrastructure has recovered. Latencies and error rates have returned to normal, and workloads have resumed.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: A fix has been implemented and we are continuing to monitor.

Identified: We have mitigated the primary impact, and service has been restored for most affected customers. Recovery efforts continue for the remaining infrastructure, with a small number of customer namespaces still affected. Our engineering team is actively working to complete restoration and will provide another update as progress is made.

Investigating: We are currently investigating this issue

Resolved: We're seeing recovery on our end and errors and latencies on Visibility APIs have dropped. We will continue to monitor this but the incident is mitigated and resolved.

Monitoring: We have identified the issue and implemented mitigation efforts. We are currently monitoring.

Investigating: We are currently investigating this issue.

Resolved: Some customers may have experienced lag in Temporal Cloud metrics (OpenMetrics - v1), starting from 12:45 UTC to 13:25 UTC. Metrics lag have since recovered back to nominal.

Resolved: A spike in workload caused resource contention in the underlying infrastructure, resulting in elevated latency for critical APIs. We scaled up the underlying resources to handle the increased load, and latencies have returned to normal. We will continue to monitor the system, but the mitigation is complete and the incident has been resolved.

Monitoring: Latencies are back to normal levels, we’re monitoring.

Investigating: The failover has been completed.

Investigating: We are failing over MRN namespaces

Investigating: Customers in AWS eu-west-1 may experience elevated API latencies.