PubNub Status

Postmortem: ### Problem Description, Impact, and Resolution 

On Tuesday, August 25th, 2026, at 12:10 UTC, our internal monitoring alerted us to an issue where a very small subset of messages within one availability zone of one region (our US East point of presence) may not have been immediately delivered to subscribers in that same region. All messages were persisted normally within PubNub Persistence service, thus no message data was lost. Traffic to or from any other region or AZ was not affected, and no other PubNub services were affected. No customers reported impact.

Root Cause

During planned internal load testing, a group of internal routing servers was removed from service. Due to a configuration flaw, their network addresses were not fully retired and remained cached by our publishing layer.

This presented no problem for the most part. However, one address was subsequently reassigned to an unrelated internal component that responded normally on the same protocol and port. Because that response appeared successful, the publishing layer received no error, and messages sent to that address were routed incorrectly.

A rolling restart of the affected servers cleared the outdated addresses at 13:25 UTC.

Mitigation Steps and Recommended Future Preventative Measures

• Once the affected servers were identified, a rolling restart was performed to clear the outdated routing addresses.
• Following an investigation that confirmed the root cause, a fix ensuring outdated internal routing addresses are correctly retired was deployed to all publishing servers worldwide on August 25th, 2026 at 22:00 UTC.

Resolved: With no further issues observed for the past 30 minutes, the incident has been resolved. We will follow up soon with a root cause analysis. If you believe you experienced an impact related to this incident, please report it to PubNub Support at [email protected].

Monitoring: As of 13:55 UTC, the issue affecting Publish and Subscribe services in the US East PoP has been mitigated. Synthetic monitoring has returned to normal, and no missed messages have been detected.

The incident is currently believed to have been related to a monitoring failure. Further investigation is ongoing, and the service continues to be monitored closely.

Investigating: As of 12:15 UTC, Users in our US East PoP are currently experiencing an issue with Publish and Subscribe service that is causing some messages to not appear in calls to that service. No errors are being returned from the service, despite messages not being returned correctly. We are investigating the issue.

Postmortem: ### Problem Description, Impact, and Resolution 

On Tuesday, August 25th, 2026, at 12:10 UTC, our internal monitoring alerted us to an issue where a very small subset of messages within one availability zone of one region (our US East point of presence) may not have been immediately delivered to subscribers in that same region. All messages were persisted normally within PubNub Persistence service, thus no message data was lost. Traffic to or from any other region or AZ was not affected, and no other PubNub services were affected. No customers reported impact.

Root Cause

During planned internal load testing, a group of internal routing servers was removed from service. Due to a configuration flaw, their network addresses were not fully retired and remained cached by our publishing layer.

This presented no problem for the most part. However, one address was subsequently reassigned to an unrelated internal component that responded normally on the same protocol and port. Because that response appeared successful, the publishing layer received no error, and messages sent to that address were routed incorrectly.

A rolling restart of the affected servers cleared the outdated addresses at 13:25 UTC.

Mitigation Steps and Recommended Future Preventative Measures

• Once the affected servers were identified, a rolling restart was performed to clear the outdated routing addresses.
• Following an investigation that confirmed the root cause, a fix ensuring outdated internal routing addresses are correctly retired was deployed to all publishing servers worldwide on August 25th, 2026 at 22:00 UTC.

Resolved: With no further issues observed for the past 30 minutes, the incident has been resolved. We will follow up soon with a root cause analysis. If you believe you experienced an impact related to this incident, please report it to PubNub Support at [email protected].

Monitoring: As of 13:55 UTC, the issue affecting Publish and Subscribe services in the US East PoP has been mitigated. Synthetic monitoring has returned to normal, and no missed messages have been detected.

The incident is currently believed to have been related to a monitoring failure. Further investigation is ongoing, and the service continues to be monitored closely.

Investigating: As of 12:15 UTC, Users in our US East PoP are currently experiencing an issue with Publish and Subscribe service that is causing some messages to not appear in calls to that service. No errors are being returned from the service, despite messages not being returned correctly. We are investigating the issue.