Scaleway Status

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Network devices in fr-par-3 are currently unreachable. impacted servers 51.159.121.50 -> 51.159.121.145 (these servers are currently offline)

Resolved: This incident has been resolved.

Monitoring: The situation is back to it's nominal state, our team is closely monitoring the product to make sure everything stays this way.

Investigating: Starting at 1AM CEST, we are seeing an elevated error rate on the fr-par region of the object storage product.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: We have identified that the increased latencies on Container Registry were caused by the following Object Storage incident: https://status.scaleway.com/incidents/mhz6qs2vq305.

Follow-up updates can be found on the linked status.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Serverless Container and Functions are also affected by this incident, which can lead to failed function builds, slower deployments and sporadic deployments errors.

Investigating: Some Registry services are experiencing 50X Bad Gateway errors when pushing images. This may cause delays in image uploads and affect deployment processes.

We are currently investigating this issue

Resolved: This incident has been resolved.

Monitoring: We have identified that the increased latencies on Container Registry were caused by the following Object Storage incident: https://status.scaleway.com/incidents/mhz6qs2vq305.

Follow-up updates can be found on the linked status.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Serverless Container and Functions are also affected by this incident, which can lead to failed function builds, slower deployments and sporadic deployments errors.

Investigating: Some Registry services are experiencing 50X Bad Gateway errors when pushing images. This may cause delays in image uploads and affect deployment processes.

We are currently investigating this issue

Identified: We are continuing to work on a fix for this issue.

Identified: Provisioning of the infrastructure powering managed inference deployments is currently impacted by increased latency. This may impact the time required to create and update deployments.

Resolved: This incident has been resolved.

Monitoring: The situation is back to it's nominal state, our team is closely monitoring the product to make sure everything stays this way.

Investigating: Starting at 1AM CEST, we are seeing an elevated error rate on the fr-par region of the object storage product.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: The situation is back to it's nominal state, our team is closely monitoring the product to make sure everything stays this way.

Investigating: Starting at 1AM CEST, we are seeing an elevated error rate on the fr-par region of the object storage product.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Identified: We are continuing to work on a fix for this issue.

Identified: Provisioning of the infrastructure powering managed inference deployments is currently impacted by increased latency. This may impact the time required to create and update deployments.

Resolved: This incident has been resolved.

Monitoring: The situation is back to it's nominal state, our team is closely monitoring the product to make sure everything stays this way.

Investigating: Starting at 1AM CEST, we are seeing an elevated error rate on the fr-par region of the object storage product.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Resolved: This incident has been resolved.

Monitoring: The Elastic Metal product is now back to full function. We are continuing to monitor for any further issues.

Monitoring: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: Update: The situation is now stabilized and we put up mitigations to keep it in nominal state. The definitive fix is being developed and tested and will be deployed in the beginning of next week. There should be no more impact but we will keep this status open to keep you updated.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: Update: 5xx error rate is back to nominal, but latencies are still high. Sadly the hotfix release did not fix every issue and our team is working on a patch that will be released tomorrow first thing in the morning. There is still some impacts on registry product.

Monitoring: Update: The hotfix release is still in progress on all the region and will take a while. There was a slight raise of error rates (0.11%) arround 15:05 CEST due to a cluster member having some issues and necessitating a reboot and resync but quickly fixed.

Impacts on registry product are still monitored but show improvements.

Monitoring: The situation is improving.

AMS and WAW region's 500s error rate is less than 0.001% and stable.

For PAR region, the impact on instances and RDB products were reduced but must be improved to return to nominal state. For registry product, object-storage backend latencies still implies some 500s due to timeouts.

The object-storage product team have prepared a new hotfix that is currently deployed on all PAR region. That action is estimated to take 7 hours to deploy at minimum.

But object-storage response times should progressively decrease during this timeframe on this region. We are still closely monitoring this issue.

Monitoring: Update: 500s errors on object-storage rate dropped on all regions. PAR region still experiences high latencies. Thoses latencies impacts Registry product by creating 500s due to timeouts. We are still working in improving response time.

Monitoring: Update: Impacts on ams and waw regions (500s errors) will settle very soon.

Monitoring: Update: Latencies worsened this monday morning. Service started show degradation degraded on ams region too. All hands are on deck to fix this issue as soon as possible.

Monitoring: Since 00h00 UTC P95 latencies are rising again. We are working on mitigating the load.

Monitoring: The vast majority of Service latencies (P95) are back to acceptable levels thanks to the actions done by the object-storage team. We are still working to further improve the P99s.

Identified: Identified : Thoses latencies are caused by the load on our AZ being unbalanced. That implies higher latencies for requests being routed to some servers of this AZ. This issue also may cause latencies on others products (Registry and Serverless)

Our team is currently working on rebalancing the load. Situation have slightly improved since the start of this operation (14H UTC). A more aggressive action is planned and will be applied soon to alleviate the load on the concerned servers.

Next update will be done in 2 hour or less.

Investigating: The Object Storage service in the fr-par region is experiencing high latencies. This may result in slower access to stored objects. We are working to resolve the issue.

Identified: We are continuing to work on a fix for this issue.

Identified: Since 2025/08/24 09:50 CEST, Dedibox servers in DC2, room 101, rack G34, block M, are unreachable. This may impact the availability of Dedibox servers in this location.

Resolved: This incident has been resolved.

Investigating: We are continuing to investigate this issue.

Investigating: We are continuing to investigate this issue.

Investigating: The IP/smtpauth.online.net 51.159.208.135 is blacklisted by Microsoft. We are currently working on it.

Investigating: We are continuing to investigate this issue.

Investigating: We are continuing to investigate this issue.

Investigating: Creation and updates are currently unavailable for .APP, .DEV, .PAGE, .SOY, .DAY, and .HOW. We are actively working on a resolution. Other domain types are not affected.

Resolved: This incident has been resolved.

Investigating: The RPN switch in DC2 lost network connectivity at 14:05. The team is currently investigating.

Identified: We are continuing to work on a fix for this issue.

Identified: Since 2025/08/24 09:50 CEST, Dedibox servers in DC2, room 101, rack G34, block M, are unreachable. This may impact the availability of Dedibox servers in this location.

Investigating: Our team is currently conducting an investigation to determine the root cause of this downtime.

Identified: Some of our email redirection servers are currently experiencing deliverability issues with the free.fr domain. Our technical teams are actively working to resolve this issue.