OutSystems Status

Postmortem: On October 20th, our cloud service provider experienced a high error rate and latency in the N. Virginia Region, which temporarily affected some of our internal tools and caused service disruption for O11 and ODC customers.
The issue was first reported at 08:37 UTC+1 for ODC, affecting access to ODC Studio and running applications in ODC Portal.
O11 had the first report at 08:54 UTC+1. Cloud customers weren't able to access their Service Centers, IdP Authentication started to present some failures, CloudWatch started to fail, and several other symptoms were felt, since it affected several networking resources, including 503 error messages.
Customers also experienced delays or interruptions using our services, including Forge, MABs, Licensing Registration, and accessing our Portals and telephony services.
Our Support and Operations teams closely monitored the situation throughout the day, providing continuous updates to affected customers and coordinating internally to restore full functionality as quickly as possible.
By 14:06 UTC+1, it was reported that a full recovery of services had been achieved, and customers began confirming that their services were operating normally. However, later in the afternoon, around 16h32 UTC+1, we started again receiving reports of connectivity issues, affecting O11 customers. During the recovery process, due to latency errors, several customers experienced network connectivity issues. Our teams continued providing updates and ensured all systems remained stable.
AWS confirmed that all its services returned to normal operations by 23:01 UTC+1. This disruption affected 142 AWS services, causing a direct impact on OutSystems services. They determined that the incident was caused by DNS resolution issues affecting DynamoDB. Following this, EC2 instance launches and Network Load Balancer health checks were also impacted, which in turn affected connectivity for Lambda, DynamoDB, and CloudWatch.

We apologize for any inconvenience caused and thank you for your patience.

Resolved: We are pleased to inform you that the issue has been resolved and the reported concerns from our customers earlier today have been successfully addressed.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: AWS Team has already identified a potential root cause and are observing recovery. During this time, requests may continue to fail as the team works towards full resolution. We will continue monitoring this situation to ensure full recovery.

Identified: We are continuing to monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Identified: We want to report an issue affecting multiple SaaS platforms globally. This is related to AWS service disruption in one of its regions (N. Virginia). We are currently monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Postmortem: On October 20th, our cloud service provider experienced a high error rate and latency in the N. Virginia Region, which temporarily affected some of our internal tools and caused service disruption for O11 and ODC customers.
The issue was first reported at 08:37 UTC+1 for ODC, affecting access to ODC Studio and running applications in ODC Portal.
O11 had the first report at 08:54 UTC+1. Cloud customers weren't able to access their Service Centers, IdP Authentication started to present some failures, CloudWatch started to fail, and several other symptoms were felt, since it affected several networking resources, including 503 error messages.
Customers also experienced delays or interruptions using our services, including Forge, MABs, Licensing Registration, and accessing our Portals and telephony services.
Our Support and Operations teams closely monitored the situation throughout the day, providing continuous updates to affected customers and coordinating internally to restore full functionality as quickly as possible.
By 14:06 UTC+1, it was reported that a full recovery of services had been achieved, and customers began confirming that their services were operating normally. However, later in the afternoon, around 16h32 UTC+1, we started again receiving reports of connectivity issues, affecting O11 customers. During the recovery process, due to latency errors, several customers experienced network connectivity issues. Our teams continued providing updates and ensured all systems remained stable.
AWS confirmed that all its services returned to normal operations by 23:01 UTC+1. This disruption affected 142 AWS services, causing a direct impact on OutSystems services. They determined that the incident was caused by DNS resolution issues affecting DynamoDB. Following this, EC2 instance launches and Network Load Balancer health checks were also impacted, which in turn affected connectivity for Lambda, DynamoDB, and CloudWatch.

We apologize for any inconvenience caused and thank you for your patience.

Resolved: We are pleased to inform you that the issue has been resolved and the reported concerns from our customers earlier today have been successfully addressed.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: AWS Team has already identified a potential root cause and are observing recovery. During this time, requests may continue to fail as the team works towards full resolution. We will continue monitoring this situation to ensure full recovery.

Identified: We are continuing to monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Identified: We want to report an issue affecting multiple SaaS platforms globally. This is related to AWS service disruption in one of its regions (N. Virginia). We are currently monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Postmortem: Our Support teams have completed corrective actions to restore full service availability. The issue has been resolved across all regions, and all systems are now operating as expected.
We sincerely apologize for any inconvenience this may have caused and appreciate your patience and understanding while we worked to restore normal operations.

Resolved: The fix has been deployed in all regions. Our team reverted the service version that was causing authentication errors, and the process was successfully completed across all regions. All services have been fully restored, and this incident is now resolved.

Investigating: We are investigating an issue affecting external authentication. Customers in US (Virginia) and Asia Pacific (Tokyo) regions may encounter errors or may not be redirected to the External IdP page to complete the authentication flow as expected.

Our team is actively working on a solution and will provide an update as soon as more information becomes available.

Postmortem: Our Support teams have completed corrective actions to restore full service availability. The issue has been resolved across all regions, and all systems are now operating as expected.
We sincerely apologize for any inconvenience this may have caused and appreciate your patience and understanding while we worked to restore normal operations.

Resolved: The fix has been deployed in all regions. Our team reverted the service version that was causing authentication errors, and the process was successfully completed across all regions. All services have been fully restored, and this incident is now resolved.

Investigating: We are investigating an issue affecting external authentication. Customers in US (Virginia) and Asia Pacific (Tokyo) regions may encounter errors or may not be redirected to the External IdP page to complete the authentication flow as expected.

Our team is actively working on a solution and will provide an update as soon as more information becomes available.

Postmortem: On October 20th, our cloud service provider experienced a high error rate and latency in the N. Virginia Region, which temporarily affected some of our internal tools and caused service disruption for O11 and ODC customers.
The issue was first reported at 08:37 UTC+1 for ODC, affecting access to ODC Studio and running applications in ODC Portal.
O11 had the first report at 08:54 UTC+1. Cloud customers weren't able to access their Service Centers, IdP Authentication started to present some failures, CloudWatch started to fail, and several other symptoms were felt, since it affected several networking resources, including 503 error messages.
Customers also experienced delays or interruptions using our services, including Forge, MABs, Licensing Registration, and accessing our Portals and telephony services.
Our Support and Operations teams closely monitored the situation throughout the day, providing continuous updates to affected customers and coordinating internally to restore full functionality as quickly as possible.
By 14:06 UTC+1, it was reported that a full recovery of services had been achieved, and customers began confirming that their services were operating normally. However, later in the afternoon, around 16h32 UTC+1, we started again receiving reports of connectivity issues, affecting O11 customers. During the recovery process, due to latency errors, several customers experienced network connectivity issues. Our teams continued providing updates and ensured all systems remained stable.
AWS confirmed that all its services returned to normal operations by 23:01 UTC+1. This disruption affected 142 AWS services, causing a direct impact on OutSystems services. They determined that the incident was caused by DNS resolution issues affecting DynamoDB. Following this, EC2 instance launches and Network Load Balancer health checks were also impacted, which in turn affected connectivity for Lambda, DynamoDB, and CloudWatch.

We apologize for any inconvenience caused and thank you for your patience.

Resolved: We are pleased to inform you that the issue has been resolved and the reported concerns from our customers earlier today have been successfully addressed.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: AWS Team has already identified a potential root cause and are observing recovery. During this time, requests may continue to fail as the team works towards full resolution. We will continue monitoring this situation to ensure full recovery.

Identified: We are continuing to monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Identified: We want to report an issue affecting multiple SaaS platforms globally. This is related to AWS service disruption in one of its regions (N. Virginia). We are currently monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Postmortem: On October 20th, our cloud service provider experienced a high error rate and latency in the N. Virginia Region, which temporarily affected some of our internal tools and caused service disruption for O11 and ODC customers.
The issue was first reported at 08:37 UTC+1 for ODC, affecting access to ODC Studio and running applications in ODC Portal.
O11 had the first report at 08:54 UTC+1. Cloud customers weren't able to access their Service Centers, IdP Authentication started to present some failures, CloudWatch started to fail, and several other symptoms were felt, since it affected several networking resources, including 503 error messages.
Customers also experienced delays or interruptions using our services, including Forge, MABs, Licensing Registration, and accessing our Portals and telephony services.
Our Support and Operations teams closely monitored the situation throughout the day, providing continuous updates to affected customers and coordinating internally to restore full functionality as quickly as possible.
By 14:06 UTC+1, it was reported that a full recovery of services had been achieved, and customers began confirming that their services were operating normally. However, later in the afternoon, around 16h32 UTC+1, we started again receiving reports of connectivity issues, affecting O11 customers. During the recovery process, due to latency errors, several customers experienced network connectivity issues. Our teams continued providing updates and ensured all systems remained stable.
AWS confirmed that all its services returned to normal operations by 23:01 UTC+1. This disruption affected 142 AWS services, causing a direct impact on OutSystems services. They determined that the incident was caused by DNS resolution issues affecting DynamoDB. Following this, EC2 instance launches and Network Load Balancer health checks were also impacted, which in turn affected connectivity for Lambda, DynamoDB, and CloudWatch.

We apologize for any inconvenience caused and thank you for your patience.

Resolved: We are pleased to inform you that the issue has been resolved and the reported concerns from our customers earlier today have been successfully addressed.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: AWS Team has already identified a potential root cause and are observing recovery. During this time, requests may continue to fail as the team works towards full resolution. We will continue monitoring this situation to ensure full recovery.

Identified: We are continuing to monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Identified: We want to report an issue affecting multiple SaaS platforms globally. This is related to AWS service disruption in one of its regions (N. Virginia). We are currently monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Postmortem: On October 20th, our cloud service provider experienced a high error rate and latency in the N. Virginia Region, which temporarily affected some of our internal tools and caused service disruption for O11 and ODC customers.
The issue was first reported at 08:37 UTC+1 for ODC, affecting access to ODC Studio and running applications in ODC Portal.
O11 had the first report at 08:54 UTC+1. Cloud customers weren't able to access their Service Centers, IdP Authentication started to present some failures, CloudWatch started to fail, and several other symptoms were felt, since it affected several networking resources, including 503 error messages.
Customers also experienced delays or interruptions using our services, including Forge, MABs, Licensing Registration, and accessing our Portals and telephony services.
Our Support and Operations teams closely monitored the situation throughout the day, providing continuous updates to affected customers and coordinating internally to restore full functionality as quickly as possible.
By 14:06 UTC+1, it was reported that a full recovery of services had been achieved, and customers began confirming that their services were operating normally. However, later in the afternoon, around 16h32 UTC+1, we started again receiving reports of connectivity issues, affecting O11 customers. During the recovery process, due to latency errors, several customers experienced network connectivity issues. Our teams continued providing updates and ensured all systems remained stable.
AWS confirmed that all its services returned to normal operations by 23:01 UTC+1. This disruption affected 142 AWS services, causing a direct impact on OutSystems services. They determined that the incident was caused by DNS resolution issues affecting DynamoDB. Following this, EC2 instance launches and Network Load Balancer health checks were also impacted, which in turn affected connectivity for Lambda, DynamoDB, and CloudWatch.

We apologize for any inconvenience caused and thank you for your patience.

Resolved: We are pleased to inform you that the issue has been resolved and the reported concerns from our customers earlier today have been successfully addressed.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: AWS Team has already identified a potential root cause and are observing recovery. During this time, requests may continue to fail as the team works towards full resolution. We will continue monitoring this situation to ensure full recovery.

Identified: We are continuing to monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Identified: We want to report an issue affecting multiple SaaS platforms globally. This is related to AWS service disruption in one of its regions (N. Virginia). We are currently monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Postmortem: On October 20th, our cloud service provider experienced a high error rate and latency in the N. Virginia Region, which temporarily affected some of our internal tools and caused service disruption for O11 and ODC customers.
The issue was first reported at 08:37 UTC+1 for ODC, affecting access to ODC Studio and running applications in ODC Portal.
O11 had the first report at 08:54 UTC+1. Cloud customers weren't able to access their Service Centers, IdP Authentication started to present some failures, CloudWatch started to fail, and several other symptoms were felt, since it affected several networking resources, including 503 error messages.
Customers also experienced delays or interruptions using our services, including Forge, MABs, Licensing Registration, and accessing our Portals and telephony services.
Our Support and Operations teams closely monitored the situation throughout the day, providing continuous updates to affected customers and coordinating internally to restore full functionality as quickly as possible.
By 14:06 UTC+1, it was reported that a full recovery of services had been achieved, and customers began confirming that their services were operating normally. However, later in the afternoon, around 16h32 UTC+1, we started again receiving reports of connectivity issues, affecting O11 customers. During the recovery process, due to latency errors, several customers experienced network connectivity issues. Our teams continued providing updates and ensured all systems remained stable.
AWS confirmed that all its services returned to normal operations by 23:01 UTC+1. This disruption affected 142 AWS services, causing a direct impact on OutSystems services. They determined that the incident was caused by DNS resolution issues affecting DynamoDB. Following this, EC2 instance launches and Network Load Balancer health checks were also impacted, which in turn affected connectivity for Lambda, DynamoDB, and CloudWatch.

We apologize for any inconvenience caused and thank you for your patience.

Resolved: We are pleased to inform you that the issue has been resolved and the reported concerns from our customers earlier today have been successfully addressed.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: AWS Team has already identified a potential root cause and are observing recovery. During this time, requests may continue to fail as the team works towards full resolution. We will continue monitoring this situation to ensure full recovery.

Identified: We are continuing to monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.

Identified: We want to report an issue affecting multiple SaaS platforms globally. This is related to AWS service disruption in one of its regions (N. Virginia). We are currently monitoring the official AWS status page. At this point, AWS is reporting significant signs of recovery. Most requests should now be succeeding.