Phrase Status

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: We are currently investigating an issue affecting the Translation Center in Phrase Strings (EU and US regions), where strings are failing to load. Our engineering team has identified a potential cause and is actively working to resolve it. We will provide further updates as our investigation progresses.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Investigating: We are currently investigating an issue affecting the Translation Center in Phrase Strings (EU and US regions), where strings are failing to load. Our engineering team has identified a potential cause and is actively working to resolve it. We will provide further updates as our investigation progresses.

Resolved: Orchestrator (EU) is available again since 5:32 PM CEST. All pending executions are being processed. This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: The issue has been identified and a fix is being implemented.

Investigating: We are continuing to investigate this issue.

Investigating: We are currently investigating an issue affecting Phrase Orchestrator. We are working to identify the root cause and will provide further updates as soon as more information is available.

Identified: We have successfully deduplicated metrics consumption for all blocked organizations. We are continuing work to develop per-organization deduplication tooling to ensure accurate billing cycle data, with the full team scaling these fixes tomorrow morning.

Monitoring: We have stopped the progression of the issue and are now working to identify the full scope of impact, including which organizations are affected or blocked. Affected accounts will be prioritized for resolution through a deduplication process, though this will take some time to complete.

Identified: We have identified the root cause as instability on a specific broker, which is causing repeated consumer group rebalancing in our Metrics service and leading to duplicate processing of metric updates. This may result in some customers experiencing incorrect consumption calculations or unexpected account blocking. We continue working on a fix.

Investigating: We are currently investigating an issue affecting our Metrics service that may result in duplicated metrics consumption for some customers. Our team is actively working to identify the affected components and determine the root cause. We will provide further updates as more information becomes available.

Identified: We have successfully deduplicated metrics consumption for all blocked organizations. We are continuing work to develop per-organization deduplication tooling to ensure accurate billing cycle data, with the full team scaling these fixes tomorrow morning.

Monitoring: We have stopped the progression of the issue and are now working to identify the full scope of impact, including which organizations are affected or blocked. Affected accounts will be prioritized for resolution through a deduplication process, though this will take some time to complete.

Identified: We have identified the root cause as instability on a specific broker, which is causing repeated consumer group rebalancing in our Metrics service and leading to duplicate processing of metric updates. This may result in some customers experiencing incorrect consumption calculations or unexpected account blocking. We continue working on a fix.

Investigating: We are currently investigating an issue affecting our Metrics service that may result in duplicated metrics consumption for some customers. Our team is actively working to identify the affected components and determine the root cause. We will provide further updates as more information becomes available.