PostHog Status

Resolved: This incident has been resolved.

Monitoring: We're monitoring the ingestion pipeline, as it processes the delayed messages. We're estimating that the system will fully recover within an hour.

Investigating: We are still investigating intermittent latency spikes in the event ingestion pipeline. Events are still being processed with a delay, which should decrease over time.

Investigating: We are still investigating the root cause of the issue. Events are still delayed but the delay is no longer increasing. We hope to have a resolution shortly

Investigating: Our data processing infrastructure is running behind which is causing inaccuracies in the reporting tools. No data has been lost and the system should be caught up shortly.

Resolved: This incident was resolved over the weekend.

Monitoring: We've shed load and haven't seen errors re-occur yet. We'll continue monitoring this over the weekend.

Investigating: The API query endpoint is throwing intermittent 500 errors due to capacity limits on our end. We are working to fix this on our end and make the errors more clear. If known valid queries are failing with 500s, we recommend retrying queries with exponential backoff.

Resolved: Bug fixed, ingestion workers scaled back up and lag recovering rapidly. No data loss should be observable.

Monitoring: We've identified the root cause of the issue. We are reprocessing exception events and continuing to monitor to make sure the pipeline fully recovers.

Identified: We are currently experiencing downtime in our error tracking data pipeline, while a bug is resolved. No data loss has occurred.