Check Point Services Status

Resolved: This incident has been resolved.

Investigating: We are currently investigating this issue.

Postmortem: # Tenant View & Access Issues for New Users (Dec 1–2, 2025)

Incident Duration:

~21 hours — 01 Dec 2025, 19:30 UTC to 02 Dec 2025, 16:30 UTC

‌

Affected Services:

Infinity Portal — Production EU & US regions

‌

Impact Scope:

Partial — Users newly added to tenants between 30 Nov 2025 and 01 Dec 2025

‌

Overview

Between December 1st and 2nd, newly added users in the Infinity Portal experienced view and access issues to their assigned tenants. This occurred due to an unintended removal of valid user-tenant associations during a database cleanup procedure.

All affected data has since been fully restored, and customers have confirmed successful access.

Business Impact

Users added to tenants during 30 Nov 2025 – 01 Dec 2025 in the EU and US production environments were unable to view or access the tenants to which they were assigned. No other users or services were affected.

Root Cause

During an effort to correct inaccurate user access records, a cleanup process unintentionally removed valid access entries for users added between 30 Nov–01 Dec. As a result, those users temporarily lost the ability to view and access their assigned tenants.

Detection

The issue was detected during the deletion procedure when it became clear that legitimate entries were unintentionally removed.

Resolution

All missing user access records were restored using snapshot data and validated with customers. Normal access has been fully restored.

‌

Timeline (All Times in UTC)

‌

30 Nov 2025 – 09:30 UTC

• A planned update was deployed as part of ongoing improvements to our user and tenant management system.

01 Dec 2025 – 15:00 UTC

• We identified irregularities in user access records that required correction to ensure accuracy and security.

01 Dec 2025 – 16:15 UTC

• A backup of the relevant databases was taken before applying corrections.

01 Dec 2025 – 16:30–19:30 UTC

• A cleanup operation was performed to remove inaccurate access records.

01 Dec 2025 – 19:30 UTC

• We discovered that the cleanup unintentionally removed valid access for users added between 30 Nov–01 Dec, causing them to lose visibility and access to their assigned tenants.

01 Dec 2025 – 19:30–21:30 UTC

• Initial restoration efforts began, and a portion of the affected access was recovered.

02 Dec 2025 – 07:30–16:30 UTC

• Full data recovery continued using database backups, restoring all missing access records.

02 Dec 2025 – 16:30 UTC

• All access issues were resolved. Customers confirmed successful restoration, and the incident was marked as complete.

‌

Preventive Actions & Next Steps

• Enhanced validation workflows before performing sensitive database operations.
• Enhanced our internal monitoring to ensure user access data stays accurate and consistent across our systems.
• Added safeguards preventing deletion of recently created associations without confirmation.
• Implementing auditing and monitoring for unexpected user-tenant association changes.

Resolved: This incident has been resolved.

Identified: The issue has been identified and a fix is being implemented.

Postmortem: # Tenant View & Access Issues for New Users (Dec 1–2, 2025)

Incident Duration:

~21 hours — 01 Dec 2025, 19:30 UTC to 02 Dec 2025, 16:30 UTC

‌

Affected Services:

Infinity Portal — Production EU & US regions

‌

Impact Scope:

Partial — Users newly added to tenants between 30 Nov 2025 and 01 Dec 2025

‌

Overview

Between December 1st and 2nd, newly added users in the Infinity Portal experienced view and access issues to their assigned tenants. This occurred due to an unintended removal of valid user-tenant associations during a database cleanup procedure.

All affected data has since been fully restored, and customers have confirmed successful access.

Business Impact

Users added to tenants during 30 Nov 2025 – 01 Dec 2025 in the EU and US production environments were unable to view or access the tenants to which they were assigned. No other users or services were affected.

Root Cause

During an effort to correct inaccurate user access records, a cleanup process unintentionally removed valid access entries for users added between 30 Nov–01 Dec. As a result, those users temporarily lost the ability to view and access their assigned tenants.

Detection

The issue was detected during the deletion procedure when it became clear that legitimate entries were unintentionally removed.

Resolution

All missing user access records were restored using snapshot data and validated with customers. Normal access has been fully restored.

‌

Timeline (All Times in UTC)

‌

30 Nov 2025 – 09:30 UTC

• A planned update was deployed as part of ongoing improvements to our user and tenant management system.

01 Dec 2025 – 15:00 UTC

• We identified irregularities in user access records that required correction to ensure accuracy and security.

01 Dec 2025 – 16:15 UTC

• A backup of the relevant databases was taken before applying corrections.

01 Dec 2025 – 16:30–19:30 UTC

• A cleanup operation was performed to remove inaccurate access records.

01 Dec 2025 – 19:30 UTC

• We discovered that the cleanup unintentionally removed valid access for users added between 30 Nov–01 Dec, causing them to lose visibility and access to their assigned tenants.

01 Dec 2025 – 19:30–21:30 UTC

• Initial restoration efforts began, and a portion of the affected access was recovered.

02 Dec 2025 – 07:30–16:30 UTC

• Full data recovery continued using database backups, restoring all missing access records.

02 Dec 2025 – 16:30 UTC

• All access issues were resolved. Customers confirmed successful restoration, and the incident was marked as complete.

‌

Preventive Actions & Next Steps

• Enhanced validation workflows before performing sensitive database operations.
• Enhanced our internal monitoring to ensure user access data stays accurate and consistent across our systems.
• Added safeguards preventing deletion of recently created associations without confirmation.
• Implementing auditing and monitoring for unexpected user-tenant association changes.

Resolved: This incident has been resolved.

Identified: The issue has been identified and a fix is being implemented.

Postmortem: ### Summary

On Friday, December 5th, 2025, between 15:45 and 18:00 UTC, Infinity Portal Data Services experienced a temporary disruption affecting data ingestion and query operations.

Incident Timeline

• 15:45 – Alerts triggered for Infinity Portal Data Services; investigation began immediately.
• 16:00 – War room initiated with engineering teams.
• 16:15 – Issue narrowed down to a backend network security component impacting service communication.
• 17:00 – Fix identified and implemented in staging for validation.
• 17:45 – Fix successfully tested and rolled out to production environments.
• 18:00 – All services fully restored and stable.

Root Cause

The disruption was caused by an internal network security component misconfiguration that required adjustments to restore proper communication between services. The issue has been fully resolved.

Next Steps

We are improving monitoring and automation for these critical components to prevent similar issues in the future.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: The issue has been identified and a fix is being implemented.

Investigating: Starting at 15:45 UTC on December 5th, we are experiencing a cross-region issue affecting data ingestion from the Infinity Portal. Our team is actively investigating and will provide updates as soon as the issue is resolved.

Postmortem: ### Summary

On Friday, December 5th, 2025, between 15:45 and 18:00 UTC, Infinity Portal Data Services experienced a temporary disruption affecting data ingestion and query operations.

Incident Timeline

• 15:45 – Alerts triggered for Infinity Portal Data Services; investigation began immediately.
• 16:00 – War room initiated with engineering teams.
• 16:15 – Issue narrowed down to a backend network security component impacting service communication.
• 17:00 – Fix identified and implemented in staging for validation.
• 17:45 – Fix successfully tested and rolled out to production environments.
• 18:00 – All services fully restored and stable.

Root Cause

The disruption was caused by an internal network security component misconfiguration that required adjustments to restore proper communication between services. The issue has been fully resolved.

Next Steps

We are improving monitoring and automation for these critical components to prevent similar issues in the future.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: The issue has been identified and a fix is being implemented.

Investigating: Starting at 15:45 UTC on December 5th, we are experiencing a cross-region issue affecting data ingestion from the Infinity Portal. Our team is actively investigating and will provide updates as soon as the issue is resolved.

Postmortem: ### Summary

On Friday, December 5th, 2025, between 15:45 and 18:00 UTC, Infinity Portal Data Services experienced a temporary disruption affecting data ingestion and query operations.

Incident Timeline

• 15:45 – Alerts triggered for Infinity Portal Data Services; investigation began immediately.
• 16:00 – War room initiated with engineering teams.
• 16:15 – Issue narrowed down to a backend network security component impacting service communication.
• 17:00 – Fix identified and implemented in staging for validation.
• 17:45 – Fix successfully tested and rolled out to production environments.
• 18:00 – All services fully restored and stable.

Root Cause

The disruption was caused by an internal network security component misconfiguration that required adjustments to restore proper communication between services. The issue has been fully resolved.

Next Steps

We are improving monitoring and automation for these critical components to prevent similar issues in the future.

Resolved: This incident has been resolved.

Monitoring: A fix has been implemented and we are monitoring the results.

Identified: The issue has been identified and a fix is being implemented.

Investigating: Starting at 15:45 UTC on December 5th, we are experiencing a cross-region issue affecting data ingestion from the Infinity Portal. Our team is actively investigating and will provide updates as soon as the issue is resolved.