CVaaS Status

Identified: We are continuing to stabilize the platform. We will post updates every 3 hours to reduce spam.

Identified: We are continuing to stabilize the platform. We will post updates every 3 hours to reduce spam.

Identified: We are continuing to stabilize the platform. We will post updates every 3 hours to reduce spam.

Identified: We are continuing to stabilize the platform. We will post updates every 3 hours to reduce spam.

Identified: We are continuing to stabilize the platform. We will post updates every 3 hours to reduce spam.

Identified: Our team is continuing to respond to this incident. At this time, some devices may see intermittent streaming latency increases, and potentially erroneous "Device Stopped Streaming" events, but not at broad scale. We expect this to impact about < 1 % of our devices currently for cv-prod-us-central1-b.

We are continuing to stabilize the platform.

Identified: We are continuing to stabilize the platform.

Identified: We are continuing to stabilize the platform.

Identified: We are continuing to stabilize the platform.

Identified: We are continuing to stabilize the platform.

Identified: We are continuing to stablize the platform.

Identified: Currently, lag has resolved and our publish latencies are stable - however we are seeing elevated error increase, which we are investigating.

Monitoring: We have fully recovered from the ingestion lag. We are continuing to monitor.

Identified: Our Publish latency for P90 and P95 have stabilized. We are continuing with recovery.

Identified: We are actively investigating as well as working on recovery.

Investigating: We are experiencing high device streaming latency on this affected cluster.

Some impact may be user visible, such as:

• A delay in processing streamed device data. Some events such as "Device stopped streaming" or "Streaming Latency Breached Threshold" may fire as a result of this. In this case, the device updates are behind schedule and will be processed with some delay during the impact window.
• There may be some stale data, or sporadic timeouts viewing parts of the UI.
• Some provisioning features may be delayed or timeout due to the high latency.

We are currently investigating the issue and working on mitigation. Thank you for your patience.