Scaleway Status

Investigating: S3 checksum error in some client/sdk, we alway return the full object checksum even when the request is ranged, causing issue in some high-level wrappers that try to compare the range's checksum with the full object one.

Full integrity of the objects aren't impacted, and all computed checksum are valid.

Identified: - Current progress on this incident:

Most of the disk attach are fixed, we still have some timeouts remaining Detach will no longer fail on those half-attached volumes Disks which were not attached properly during the incident wont be fixed automatically, you can detach/attach them Root cause not known

• Next actions on Instance side:

We will deploy a second release this morning with an even higher timeout value We'll wait 1-2 hours to ensure everything is OK before passing the status to monitoring as we don't know the root cause

Investigating: Since Nov 6th at 10 PM (FR), some databases have failed to start, and users are unable to connect to their databases. The issue is linked to the incident detailed at https://status.scaleway.com/incidents/n7k8t96f4cjy.

Investigating: We are continuing to investigate this issue.

Investigating: Starting on 2024/11/14 17:47 UTC, clients might experience issues pushing/pulling images from rg.fr-par.scw.cloud due to a certificate issue.

As a result, users from Serverless Functions/Containers using an image hosted on rg.fr-par.scw.cloud might also get impacted, as their functions/containers won't be able to start. Impacts for Serverless Functions/Containers are:

• instances that cannot start (scale up might be impossible)
• requests timeouting or 5xx errors if no instances are available.

We are currently investigating this issue, and apologize for the inconvenience.

Resolved: The root cause has been identified. A limit was reached, preventing new domain challenges to be created since 2025-05-26 06:42 UTC, thus leading to errors for new domains creation. This affected only a small part of the fr-par infrastructure (~20% of users).

We are going to fix the limit, and add monitoring, so it doesn't happen again.

Sorry again, and thanks for your understanding

Monitoring: The issue has been fixed. We have not found the root cause yet, but it's still under investigation. However, users should be able to create new custom domains.

Sorry again for any inconvenience.

Identified: A bug prevents users to create new custom domains on their functions/containers. It affects around 20% of the clients in fr-par. For these users, custom domains end up in error after around 3 minutes, even if CNAME configuration is correct. Already created custom domains are not affected. We are on it. Sorry about the inconvenience.

Resolved: This incident has been resolved.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished. We are monitoring the results.

Identified: We are implementing a fix.

Investigating: Our Secret Manager product is currently out of service in the fr-par region since 05h18 UTC. the ‘CreateSecretVersion’ and ‘AccessSecretVersion’ methods are unusable.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Identified: The IP 51.159.208.135 of our Webhosting platform is blacklisted by Microsoft. We are currently working on it.

Resolved: This incident has been resolved.

Monitoring: Since 9:47:08 UTC users may experience errors when trying to connect to their Grafana or manage their Grafana Users. Ingestion / Alerting is not impacted Incident is no longer live since 13:35 UTC Situation under investigation

Investigating: We are currently investigating this issue.

Resolved: This incident has been resolved.

Investigating: Customers can not access Grafana.

Resolved: The issue appeared at 3pm, after a new release. We rollbacked to the latest stable state and we will be merging a fix for what happened as soon as possible.

Investigating: Due to internal technical updates, the Organization Settings and Support Tickets pages are currently unavailable. We appreciate your patience and understanding as we work to restore access as soon as possible.

Resolved: This incident has been resolved.

Identified: Phone validation isn't possible on the Scaleway console due to a technical issue. Our team is currently investigate this issue.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished.

We are monitoring the results.

Identified: We are implementing a fix

Investigating: Our KeyManager product is currently out of service in the fr-par region since 05h18 UTC.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Resolved: This incident has been resolved.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished. We are monitoring the results.

Identified: We are implementing a fix.

Investigating: Our Secret Manager product is currently out of service in the fr-par region since 05h18 UTC. the ‘CreateSecretVersion’ and ‘AccessSecretVersion’ methods are unusable.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Identified: - Current progress on this incident:

Most of the disk attach are fixed, we still have some timeouts remaining Detach will no longer fail on those half-attached volumes Disks which were not attached properly during the incident wont be fixed automatically, you can detach/attach them Root cause not known

• Next actions on Instance side:

We will deploy a second release this morning with an even higher timeout value We'll wait 1-2 hours to ensure everything is OK before passing the status to monitoring as we don't know the root cause

Investigating: Since Nov 6th at 10 PM (FR), some databases have failed to start, and users are unable to connect to their databases. The issue is linked to the incident detailed at https://status.scaleway.com/incidents/n7k8t96f4cjy.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently experiencing an issue affecting the instances infrastructure across all availability zones in FR-PAR.

Impact:

• Starting and stopping instances with local storage may experience significant delays
• Snapshot tasks are currently stuck and unable to progress

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: We are aware of issues affecting customers of the gemma-3-27b-it model since 15:00. We are currently investigating theses issues.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished.

We are monitoring the results.

Identified: We are implementing a fix

Investigating: Our KeyManager product is currently out of service in the fr-par region since 05h18 UTC.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Resolved: This incident has been resolved.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished. We are monitoring the results.

Identified: We are implementing a fix.

Investigating: Our Secret Manager product is currently out of service in the fr-par region since 05h18 UTC. the ‘CreateSecretVersion’ and ‘AccessSecretVersion’ methods are unusable.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Resolved: The root cause has been identified. A limit was reached, preventing new domain challenges to be created since 2025-05-26 06:42 UTC, thus leading to errors for new domains creation. This affected only a small part of the fr-par infrastructure (~20% of users).

We are going to fix the limit, and add monitoring, so it doesn't happen again.

Sorry again, and thanks for your understanding

Monitoring: The issue has been fixed. We have not found the root cause yet, but it's still under investigation. However, users should be able to create new custom domains.

Sorry again for any inconvenience.

Identified: A bug prevents users to create new custom domains on their functions/containers. It affects around 20% of the clients in fr-par. For these users, custom domains end up in error after around 3 minutes, even if CNAME configuration is correct. Already created custom domains are not affected. We are on it. Sorry about the inconvenience.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently experiencing an issue affecting the instances infrastructure across all availability zones in FR-PAR.

Impact:

• Starting and stopping instances with local storage may experience significant delays
• Snapshot tasks are currently stuck and unable to progress

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: We are aware of issues affecting customers of the gemma-3-27b-it model since 15:00. We are currently investigating theses issues.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished.

We are monitoring the results.

Identified: We are implementing a fix

Investigating: Our KeyManager product is currently out of service in the fr-par region since 05h18 UTC.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Resolved: This incident has been resolved.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished. We are monitoring the results.

Identified: We are implementing a fix.

Investigating: Our Secret Manager product is currently out of service in the fr-par region since 05h18 UTC. the ‘CreateSecretVersion’ and ‘AccessSecretVersion’ methods are unusable.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Resolved: The root cause has been identified. A limit was reached, preventing new domain challenges to be created since 2025-05-26 06:42 UTC, thus leading to errors for new domains creation. This affected only a small part of the fr-par infrastructure (~20% of users).

We are going to fix the limit, and add monitoring, so it doesn't happen again.

Sorry again, and thanks for your understanding

Monitoring: The issue has been fixed. We have not found the root cause yet, but it's still under investigation. However, users should be able to create new custom domains.

Sorry again for any inconvenience.

Identified: A bug prevents users to create new custom domains on their functions/containers. It affects around 20% of the clients in fr-par. For these users, custom domains end up in error after around 3 minutes, even if CNAME configuration is correct. Already created custom domains are not affected. We are on it. Sorry about the inconvenience.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently experiencing an issue affecting the instances infrastructure across all availability zones in FR-PAR.

Impact:

• Starting and stopping instances with local storage may experience significant delays
• Snapshot tasks are currently stuck and unable to progress

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: We are aware of issues affecting customers of the gemma-3-27b-it model since 15:00. We are currently investigating theses issues.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: We are currently investigating this issue.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished.

We are monitoring the results.

Identified: We are implementing a fix

Investigating: Our KeyManager product is currently out of service in the fr-par region since 05h18 UTC.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Resolved: This incident has been resolved.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: A fix has been implemented since 7:15UTC / 9:15 CET and the service has been reestablished. We are monitoring the results.

Identified: We are implementing a fix.

Investigating: Our Secret Manager product is currently out of service in the fr-par region since 05h18 UTC. the ‘CreateSecretVersion’ and ‘AccessSecretVersion’ methods are unusable.

We apologize for the inconvenience and our engineers are working hard to resolve this quickly.

Resolved: The root cause has been identified. A limit was reached, preventing new domain challenges to be created since 2025-05-26 06:42 UTC, thus leading to errors for new domains creation. This affected only a small part of the fr-par infrastructure (~20% of users).

We are going to fix the limit, and add monitoring, so it doesn't happen again.

Sorry again, and thanks for your understanding

Monitoring: The issue has been fixed. We have not found the root cause yet, but it's still under investigation. However, users should be able to create new custom domains.

Sorry again for any inconvenience.

Identified: A bug prevents users to create new custom domains on their functions/containers. It affects around 20% of the clients in fr-par. For these users, custom domains end up in error after around 3 minutes, even if CNAME configuration is correct. Already created custom domains are not affected. We are on it. Sorry about the inconvenience.

Resolved: This incident has been resolved.

Identified: All the servers are back up and running on block M. Our engineers are continuing their investigations on block N.

Thank you for your patience and understanding.

Investigating: We have detected a switch down in DC2 room S101 G22 bloc N. Servers in that rack currently have no <public/private> network access and are unreachable.

=================== 25.05.25 05h32 UTC Issue has been forwarded to our team for resolution.

Investigating: We have detected a switch down in DC5 room S1 Rack C34. Servers in that rack currently have no <public/private> network access and are unreachable.

===================

26.05.25 09h41 UTC Issue has been forwarded to our team for resolution.

Resolved: This incident has been resolved.

Monitoring: The servers are starting to recover on this switch.

Thank you for your patience.

Investigating: We have detected a switch down in AMS rack EE22 room hall2. Servers in that rack currently have no <public/private> network access and are unreachable.

===================

24.05.25 12.13 UTC Issue has been forwarded to our team for resolution.

Resolved: This incident has been resolved.

Investigating: We are currently investigating this issue.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: We are currently experiencing a technical issue with the installation of Ubuntu 24.04 LTS, our teams are currently working on it, we will keep you informed.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: We are currently experiencing a technical issue with the installation of Ubuntu 20.04, our teams are currently working on it, we will keep you informed.

Identified: The IP 51.159.208.135 of our Webhosting platform is blacklisted by Microsoft. We are currently working on it.

Investigating: Several classic webhosting customers may experiencing 500/error SSL certificate errors. Our team is already working on it.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: We are continuing to work on a fix for this issue.

Identified: An issue regarding VPS servers management is currently preventing customers from managing their instances.

Deployed servers are unaffected.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: VPS management page and several endpoints are returning a 504 Gateway Time-out error. Our engineers are doing everything they can to resolve this quickly.

Monitoring: Swith has been replaced per a new one and all the customer servers seems back for the business.

Investigating: We have detected that public switch was down in DC2 Room 101 Rack A20. Servers in that rack currently have no network access and are unreachable.

Resolved: This incident has been resolved.

Identified: All the servers are back up and running on block M. Our engineers are continuing their investigations on block N.

Thank you for your patience and understanding.

Investigating: We have detected a switch down in DC2 room S101 G22 bloc N. Servers in that rack currently have no <public/private> network access and are unreachable.

=================== 25.05.25 05h32 UTC Issue has been forwarded to our team for resolution.

Investigating: We have detected a switch down in DC5 room S1 Rack C34. Servers in that rack currently have no <public/private> network access and are unreachable.

===================

26.05.25 09h41 UTC Issue has been forwarded to our team for resolution.

Resolved: This incident has been resolved.

Monitoring: The servers are starting to recover on this switch.

Thank you for your patience.

Investigating: We have detected a switch down in AMS rack EE22 room hall2. Servers in that rack currently have no <public/private> network access and are unreachable.

===================

24.05.25 12.13 UTC Issue has been forwarded to our team for resolution.

Resolved: This incident has been resolved.

Investigating: We are currently investigating this issue.

Resolved: This incident has been resolved.

Investigating: We are currently investigating this issue.

Resolved: This incident has been resolved.

Identified: Due to circumstances beyond our control, support will be exceptionally unavailable from 6 PM tonight until 10 PM. We apologize for this inconvenience.