JustCall Status

Investigating: We are continuing to investigate this issue.

Investigating: We are currently investigating the issue.

Postmortem: Date: 9th July 2025

Duration: 14:51 UTC - 15:25 UTC

Affected Services:

• Customer-facing Dashboard
• SMS Delivery
• MMS Delivery
• Inbound Calling
• Outbound Calling
• Associated internal services

Impact

A subset of customers experienced degraded performance or complete unavailability in:

• Accessing the dashboard
• Users experienced delays in sending and receiving SMS and MMS messages.
• Intermittent degradation in both inbound and outbound call performance.
• Delayed or failed service actions tied to within a designated cloud service region.

Root Cause

A malfunction in one of our cache systems triggered a reboot, leading to the invalidation of all cache entries in that system:

• Massive cache miss rate
• Significant surge in requests directly hitting primary databases and downstream services
• Snowball/piggyback effect where database latency grew, increasing queue sizes and causing cascading slowdowns across services
• Overwhelmed compute resources and degraded service health in that region.

Resolution

• Engineers manually restored cache to a known good state from cache backups.
• Temporarily scaled up database resources and async queues.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Some customers are experiencing issues with loading the JustCall dashboard, and failures in sending SMS, MMS, and making or receiving calls via the web interface. Investigation is ongoing.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently investigating the issue.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently investigating the issue.

Postmortem: Date: 9th July 2025

Duration: 14:51 UTC - 15:25 UTC

Affected Services:

• Customer-facing Dashboard
• SMS Delivery
• MMS Delivery
• Inbound Calling
• Outbound Calling
• Associated internal services

Impact

A subset of customers experienced degraded performance or complete unavailability in:

• Accessing the dashboard
• Users experienced delays in sending and receiving SMS and MMS messages.
• Intermittent degradation in both inbound and outbound call performance.
• Delayed or failed service actions tied to within a designated cloud service region.

Root Cause

A malfunction in one of our cache systems triggered a reboot, leading to the invalidation of all cache entries in that system:

• Massive cache miss rate
• Significant surge in requests directly hitting primary databases and downstream services
• Snowball/piggyback effect where database latency grew, increasing queue sizes and causing cascading slowdowns across services
• Overwhelmed compute resources and degraded service health in that region.

Resolution

• Engineers manually restored cache to a known good state from cache backups.
• Temporarily scaled up database resources and async queues.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Some customers are experiencing issues with loading the JustCall dashboard, and failures in sending SMS, MMS, and making or receiving calls via the web interface. Investigation is ongoing.

Postmortem: Date: 9th July 2025

Duration: 14:51 UTC - 15:25 UTC

Affected Services:

• Customer-facing Dashboard
• SMS Delivery
• MMS Delivery
• Inbound Calling
• Outbound Calling
• Associated internal services

Impact

A subset of customers experienced degraded performance or complete unavailability in:

• Accessing the dashboard
• Users experienced delays in sending and receiving SMS and MMS messages.
• Intermittent degradation in both inbound and outbound call performance.
• Delayed or failed service actions tied to within a designated cloud service region.

Root Cause

A malfunction in one of our cache systems triggered a reboot, leading to the invalidation of all cache entries in that system:

• Massive cache miss rate
• Significant surge in requests directly hitting primary databases and downstream services
• Snowball/piggyback effect where database latency grew, increasing queue sizes and causing cascading slowdowns across services
• Overwhelmed compute resources and degraded service health in that region.

Resolution

• Engineers manually restored cache to a known good state from cache backups.
• Temporarily scaled up database resources and async queues.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Some customers are experiencing issues with loading the JustCall dashboard, and failures in sending SMS, MMS, and making or receiving calls via the web interface. Investigation is ongoing.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently investigating the issue.