Scaleway Status

Identified: We did discover that the trouble is broader than we thought. As a first mitigation we do recommend that users wait before upgrading from aws v1 (we do recommand any version strictly prior to aws-cli 1.37.0 or boto3 < 1.36.0). If aws v2 (that we do not recommend for the moment) is mandatory, we do recommend using aws-cli < 2.23.0 We are working actively to support those new versions, and plan to release fixes for the end of next week. The details of the bug are : support of CRC64NVME does not work properly Tranfer-Encoding: chunked header does not work

Identified: Everyone using aws-cli versions versions v1: >=1.37.0 and v2: >=2.23.0 received a 400 errors on PUT and POST requests.

Investigating: aws-cli now enforces a CRC64NVME integrity checksum on all PUT and POST requests. This concerns the versions v1: >=1.37.0 and v2: >=2.23.0. We do not currently support this checksum, we are working on how best to handle it on our end. In the meantime, you can choose one of these options to keep your aws-cli request working:

• Use the --checksum-algorithm option with one of our supported checksums:
  • SHA1
  • SHA256
  • CRC32
  • CRC32C
• Use an older version of aws-cli, this way it will not enforce the CRC64NVME checksum

Resolved: This incident has been resolved.

Monitoring: The problem reappeared briefly, but was immediately monitored and fixed by our engineers.

Monitoring: Problem as been found and fixed.

We are seeing the error rate getting lower, we are monitoring until full recovery

Investigating: Object Storage is unavalaible in pl-waw as of right now.

Our engineers are investigating this.

Identified: - Current progress on this incident:

Most of the disk attach are fixed, we still have some timeouts remaining Detach will no longer fail on those half-attached volumes Disks which were not attached properly during the incident wont be fixed automatically, you can detach/attach them Root cause not known

• Next actions on Instance side:

We will deploy a second release this morning with an even higher timeout value We'll wait 1-2 hours to ensure everything is OK before passing the status to monitoring as we don't know the root cause

Investigating: Since Nov 6th at 10 PM (FR), some databases have failed to start, and users are unable to connect to their databases. The issue is linked to the incident detailed at https://status.scaleway.com/incidents/n7k8t96f4cjy.

Investigating: We are continuing to investigate this issue.

Investigating: Starting on 2024/11/14 17:47 UTC, clients might experience issues pushing/pulling images from rg.fr-par.scw.cloud due to a certificate issue.

As a result, users from Serverless Functions/Containers using an image hosted on rg.fr-par.scw.cloud might also get impacted, as their functions/containers won't be able to start. Impacts for Serverless Functions/Containers are:

• instances that cannot start (scale up might be impossible)
• requests timeouting or 5xx errors if no instances are available.

We are currently investigating this issue, and apologize for the inconvenience.

Monitoring: In the time range from 20/03/25 4:30pm Paris time to 7:30pm some Scaleway Metrics and Logs may have been lost or misrouted to custom DataSources on all regions. In the time range from 20/03/25 4:30pm Paris time to 21/03/25 11am some Scaleway Metrics and Logs may have been lost on fr-par region only. During those time frames, gaps can appear in your data/dashboards. The issue has been fixed, but impact is still under investigation.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: We have detected an incident this Thursday, March 20th between 17:05 PM UTC and 18:20 PM UTC that could potentially prevent our customers from viewing their function/container logs. Our teams have already resolved the situation.

Resolved: This incident has been resolved.

Investigating: Product monitoring is experiencing increased latency.

Monitoring: In the time range from 20/03/25 4:30pm Paris time to 7:30pm some Scaleway Metrics and Logs may have been lost or misrouted to custom DataSources on all regions. In the time range from 20/03/25 4:30pm Paris time to 21/03/25 11am some Scaleway Metrics and Logs may have been lost on fr-par region only. During those time frames, gaps can appear in your data/dashboards. The issue has been fixed, but impact is still under investigation.

Monitoring: We are continuing to monitor for any further issues.

Monitoring: We have detected an incident this Thursday, March 20th between 17:05 PM UTC and 18:20 PM UTC that could potentially prevent our customers from viewing their function/container logs. Our teams have already resolved the situation.

Resolved: This incident has been resolved.

Investigating: We are currently investigating an issue with some emails being marked as spam for Orange users. We are currently in contact with Orange to resolve the situation. Start time: 26/03/2025 9h00

Investigating: We are currently investigating an issue with some emails being marked as spam for Orange users. We are currently in contact with Orange to resolve the situation. Start time: 26/03/2025 9h00

Resolved: This incident has been resolved.

Investigating: Some customers, especially with high traffic have sending latencies. The emails are not sent in time.

Identified: - Current progress on this incident:

Most of the disk attach are fixed, we still have some timeouts remaining Detach will no longer fail on those half-attached volumes Disks which were not attached properly during the incident wont be fixed automatically, you can detach/attach them Root cause not known

• Next actions on Instance side:

We will deploy a second release this morning with an even higher timeout value We'll wait 1-2 hours to ensure everything is OK before passing the status to monitoring as we don't know the root cause

Investigating: Since Nov 6th at 10 PM (FR), some databases have failed to start, and users are unable to connect to their databases. The issue is linked to the incident detailed at https://status.scaleway.com/incidents/n7k8t96f4cjy.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently experiencing an issue affecting the instances infrastructure across all availability zones in FR-PAR.

Impact:

• Starting and stopping instances with local storage may experience significant delays
• Snapshot tasks are currently stuck and unable to progress

Resolved: This incident has been resolved.

Investigating: Some customers, especially with high traffic have sending latencies. The emails are not sent in time.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently experiencing an issue affecting the instances infrastructure across all availability zones in FR-PAR.

Impact:

• Starting and stopping instances with local storage may experience significant delays
• Snapshot tasks are currently stuck and unable to progress

Resolved: This incident has been resolved.

Investigating: Some customers, especially with high traffic have sending latencies. The emails are not sent in time.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently experiencing an issue affecting the instances infrastructure across all availability zones in FR-PAR.

Impact:

• Starting and stopping instances with local storage may experience significant delays
• Snapshot tasks are currently stuck and unable to progress

Resolved: This incident has been resolved.

Investigating: Some customers, especially with high traffic have sending latencies. The emails are not sent in time.

Resolved: This incident has been resolved.

Monitoring: The problem reappeared briefly, but was immediately monitored and fixed by our engineers.

Monitoring: Problem as been found and fixed.

We are seeing the error rate getting lower, we are monitoring until full recovery

Investigating: Object Storage is unavalaible in pl-waw as of right now.

Our engineers are investigating this.

Resolved: This incident has been resolved.

Monitoring: The problem reappeared briefly, but was immediately monitored and fixed by our engineers.

Monitoring: Problem as been found and fixed.

We are seeing the error rate getting lower, we are monitoring until full recovery

Investigating: Object Storage is unavalaible in pl-waw as of right now.

Our engineers are investigating this.

Resolved: This incident has been resolved.

Monitoring: The problem reappeared briefly, but was immediately monitored and fixed by our engineers.

Monitoring: Problem as been found and fixed.

We are seeing the error rate getting lower, we are monitoring until full recovery

Investigating: Object Storage is unavalaible in pl-waw as of right now.

Our engineers are investigating this.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: Some website and their database are unavailable. The webhosting team has been alerted and will investigate it as soon as possible.

Resolved: This incident has been resolved.

Identified: The PSUs were faulty but the chassis is still unreachable, we are working on it

Investigating: We have detected a switch down in DC2 ;room 101; g30 bloc J. Servers in that rack currently have no public and private network access and are unreachable.

===================

28.03.2025 08h44 UTC The issue has been forwarded to our team for resolution.

Resolved: start: 14:01:27 UTC end: 14:06:14 UTC

Resolved: This incident has been resolved.

Monitoring: RPN was unavailable today between 13:20:34 UTC and 13:27 UTC in DC2/S203b/J5

Resolved: This incident has been resolved.

Identified: The issue has been identified, and a fix is being implemented. start: 13:20 UTC end: 13:41 UTC

Investigating: We are currently investigating this issue.

Identified: The PSUs were faulty, we are replacing them

Identified: We have detected a switch down in AMS1/Hall2/E21 bloc L Servers in that rack currently have no public and private network access and are unreachable.

===================

28.03.2025 08h44 UTC The issue has been forwarded to our team for resolution.

Resolved: This incident has been resolved.

Monitoring: We reseat the interface, and the rpn link is stable so far. We are monitoring the link.

Identified: RPN uplink is unstable since today Mar 28 08:43:40 UTC