Security Operations Platform
Incident Report for Exabeam
Resolved
All the Exabeam regions and applications are now on-line and fully functional. We have further carried out all successful validations.

Webhook Cloud Collectors, if used, require an update after your region is on-line (available for all regions). Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

We are monitoring closely as some backlogs continue to be processed.

We sincerely appreciate your patience and understanding. If you have any questions or concerns, please contact Exabeam technical support.
Posted Nov 20, 2024 - 13:06 UTC
Update
All regions - On-line and fully functional

Webhook Cloud Collectors, if used, require an update after your region is on-line (available for all regions). Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

We are monitoring closely as some backlogs continue to be processed.
Posted Nov 20, 2024 - 05:14 UTC
Monitoring
Engineering continues to work on getting all regions on-line.

US-West, US-East, EU-W3, SG - On-line and fully functional. Some backlogs are still being processed.

JP, AU, CA, EU-W6 - On-line with some application restarts continuing. Some backlogs are still being processed.

During the restart for some applications, issues were encountered which needed to be resolved to restore full functionality. The restoration is taking longer than expected. All current known issues have been resolved.

Webhook Cloud Collectors, if used, require an update after your region is on-line (now available for all regions). Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

Backlog processing will continue beyond the time when a region comes on-line. We are monitoring closely and estimate backlog processing to be completed by midnight EST, November 19.
Posted Nov 19, 2024 - 23:23 UTC
Update
Engineering continues to work on getting all regions on-line.

US-West, US-East, EU-W3, SG - On-line and fully functional. Some backlogs are still being processed.

JP, AU, CA - On-line with some application restarts continuing. Some backlogs are still being processed.

EU-W6 - Restart is still in the process as system encountered some issues and is not yet available.

During the restart for some applications, issues were encountered which needed to be resolved to restore full functionality. The restoration is taking longer than expected. All current known issues have been resolved.

Webhook Cloud Collectors, if used, require an update after your region is on-line (US-W, US-E, EU-W3, SG, JP, AU, CA at this time). Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

Backlog processing will continue beyond the time when a region comes on-line.
Posted Nov 19, 2024 - 15:43 UTC
Update
Engineering continues to work on getting all regions on-line.

US-West, EU-W3, SG - On-line with some backlog still being processed.

US-East, JP, AU, CA - On-line with some application restarts continuing. Some backlog still being processed.

EU-W6 - Beginning the restart process and is not yet available.

During the restart for some applications, issues were encountered which needed to be resolved to restore full functionality. The restoration is taking longer than expected. All current known issues have been resolved.

Webhook Cloud Collectors, if used, require an update after your region is on-line (US-W, US-E, EU-W3, SG, JP at this time). Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

Backlog processing will continue beyond the time when a region comes on-line.
Posted Nov 19, 2024 - 04:27 UTC
Update
The engineering team continues to work to bring all regions on-line.

US-West, EU-W3, SG - On-line with some backlog still being processed.

US-East, JP, AU, CA - On-line with some applications restarting. Backlog processing is on-going.

EU-W6 - Beginning the restart process and is not yet available.

Some regions are taking longer than expected to restart. Individual applications within each region will continue to come on-line, and some functionality may not be available until the region has all applications fully functional.

Webhook Cloud Collectors, if used, require an update after your region is on-line (US-W, EU-W3, SG at this time). Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

Backlog processing will continue beyond the time when a region comes on-line.
Posted Nov 19, 2024 - 00:20 UTC
Update
The engineering team continues to work to bring all regions on-line.

US-West, EU-W3, SG - On-line with some backlog still being processed.

US-East, JP, AU, CA - On-line with some applications restarting. Backlog processing is on-going.

EU-W6 - Beginning the restart process and is not yet available.

Some regions are taking longer than expected to restart. The estimate for all regions to be fully functional is 4PM PT Monday November 18th. Individual applications within each region will continue to come on-line, and some functionality may not be available until the region has all applications fully functional.

Webhook Cloud Collectors, if used, require an update after your region is on-line (US-W, EU-W3, SG at this time). Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

Backlog processing will continue beyond the time when a region comes on-line.
Posted Nov 18, 2024 - 17:01 UTC
Update
The engineering team continues to work to bring all regions on-line.

We expect to have all regions on-line by 9AM PT Monday November 18th. Backlog processing will continue beyond the time when the region comes on-line.
Posted Nov 18, 2024 - 13:47 UTC
Update
The engineering team continues to work to bring all regions on-line.

Regions on-line – US West; Europe

All other regions – application restart process in-progress – expected completion 9AM PT Monday November 18th

Webhook Cloud Collectors, if used, require an update after your region is on-line. Details on how to update webhooks are here: https://community.exabeam.com/s/article/Webhook-Reconfiguration-Guide

The expected completion time is an estimate based on how the process went for the first region. Updates will be posted if there is a change in the estimate or when regions come on-line. The backlog will be processed once the region is on-line.

During the restart process, prior to a notification that the region is on-line, applications will be coming on-line and will be available but all functionality will not be available. We appreciate your patience during the restart process.
Posted Nov 18, 2024 - 04:11 UTC
Update
The engineering team continues to work to bring all regions on-line.

US West – All applications are running with the exception of WebHook connections. We continue working to reestablished the webhook capability.
Europe – application restart process in-progress – expected completion of restart is 11PM PT

All other regions – application restart process in-progress – expected completion 9AM PT

The expected completion time is an estimate based on how the process has gone for the first region. Updates will be posted if there is a change in the estimate or when the region is on-line.

During the restart process, prior to a notification that the region is on-line, applications will be coming on-line and will be available but all functionality will not be available. We appreciate your patience during the restart process.
Posted Nov 18, 2024 - 01:11 UTC
Update
The US West is now operational. The backlog of logs is being processed. Webhooks connections are still being reestablished.

We continue to work on bringing additional regions on-line. We will continue to provide updates.

To identify your deployment region, simply look at your Exabeam URL:

.exabeam.cloud - Your Region is US West
.use1.exabeam.cloud - Your Region is US East
.ca.exabeam.cloud - Your Region is Canada
.eu.exabeam.cloud - Your Region is Europe
.euw6.exabeam.cloud - Your Region is Switzerland
.au.exabeam.cloud - Your Region is Australia
.jp.exabeam.cloud - Your Region is Japan
.sg.exabeam.cloud - Your Region is Singapore
Posted Nov 17, 2024 - 18:23 UTC
Update
The engineering team continues to make progress in restoring the system. Each application within the platform must be restarted. The process of restarting applications has begun in some regions. During the restart process, some functionality may be available but not all product functionality is available. We appreciate your patience during the restart process.

As regions become fully operational updates will be provided. You should expect the next update within the next 4 hours.
Posted Nov 17, 2024 - 13:06 UTC
Update
The engineering team continues to make progress in restoring the system. Each application within the platform must be restarted. The process of restarting applications has begun in some regions. During the restart process, some functionality may be available but not all product functionality is available. We appreciate your patience during the restart process.

As regions become fully operational updates will be provided. You should expect the next update within the next 8 hours.
Posted Nov 17, 2024 - 05:17 UTC
Update
Our global engineering teams are continuing to make progress on restoring our Security Operations Platform. We are actively working through the issues to restore full functionality as quickly as possible. While incoming logs are not currently being processed, this functionality will resume when the system is restored.

We understand how important our service is to your operations, and we sincerely apologize for any inconvenience this may cause. Please rest assured that resolving this matter remains our highest priority, and we are making every effort to minimize the downtime.

While full system restoration is in-progress, you may be able to login but not all product features will be available. Please check back for updates on our progress to restoring full functionality.

We will continue to provide updates on our progress, and you should expect the next update within the next 4 hours.
Posted Nov 17, 2024 - 01:40 UTC
Update
We are continuing to make progress on restoring our Security Operations Platform. All our global engineering teams are actively working through the issues to restore full functionality as quickly as possible. While incoming logs are not currently being processed, this functionality will resume when the system is restored.

We understand how important our service is to your operations, and we sincerely apologize for any inconvenience this may cause. Please rest assured that resolving this matter remains our highest priority, and we are making every effort to minimize the downtime.

While full system restoration is in-progress, you may be able to login but not all product features will be available. Please check back for updates on our progress to restoring full functionality.

We will continue to provide updates on our progress, and you should expect the next update within the next 4 hours.
Posted Nov 16, 2024 - 22:01 UTC
Update
Our Security Operations Platform is currently experiencing an unexpected outage. This outage is due to error encountered during a routine system upgrade. The engineering team is actively working to resolve the issue and restore full functionality as quickly as possible. Currently, incoming logs are not being processed. Processing of the logs will resume when the system is restored.

We understand how important our service is to your operations, and we sincerely apologize for any inconvenience this may cause. Please rest assured that resolving this matter is our highest priority, and we are making every effort to minimize the downtime.

We will continue to provide updates on our progress, and you should expect the next update within the next 4 hours.
Posted Nov 16, 2024 - 18:34 UTC
Update
Exabeam along with the Vendor Google continues to work on the remediation of the issue. We apologize for any inconvenience. Next update to come as we gather more information.
Posted Nov 16, 2024 - 16:31 UTC
Update
Exabeam along with the Vendor Google continues to work on the remediation of the issue. We apologize for any inconvenience. Next update to come as we gather more information.
Posted Nov 16, 2024 - 11:55 UTC
Update
Exabeam along with the Vendor Google have identified the action plan to fix the issue and remediation is in progress.
We apologize for any inconvenience. Next update to come as we gather more information.
Posted Nov 16, 2024 - 09:48 UTC
Update
We still continue to investigate the service outage with Security Operations Platform with the assistance of our vendor Google. We apologize for any inconvenience. Next update to come as we gather more information.
Posted Nov 16, 2024 - 08:10 UTC
Update
While we are continuing to work toward resolving this issue, platform access has been restored in the following regions:

- us-west1
- us-east1
- europe-west3
- europe-west6
- me-central2
- SG

We have further involved our cloud Vendor Google and have raised a high priority case. Exabeam and Google continues to investigate the issue. We apologize for any inconvenience and will post another update as we make progress on restoring all services.
Posted Nov 16, 2024 - 06:56 UTC
Update
While we are continuing to work toward resolving this issue, platform access has been restored in the following regions:

- us-west1
- us-east1
- europe-west3
- europe-west6
- me-central2

We apologize for any inconvenience and will post another update as we make progress on restoring all services.
Posted Nov 16, 2024 - 04:44 UTC
Update
We are continuing to work toward resolving this issue. We apologize for any inconvenience and will post another update as we make progress on restoring services.
Posted Nov 16, 2024 - 03:33 UTC
Update
We are continuing to work toward resolving this issue. We apologize for any inconvenience and will post another update as we make progress on restoring services.
Posted Nov 16, 2024 - 02:12 UTC
Update
We continue to work toward resolving this issue. We apologize for any inconvenience and will post another update as soon as we learn more.
Posted Nov 16, 2024 - 01:11 UTC
Update
We’re experiencing a service outage with Security Operations Platform where both ingestion and access are currently impacted. We've root caused the issue and our team is working diligently to restore services as fast as possible. We apologize for any inconvenience. Next update to come as we gather more information.
Posted Nov 16, 2024 - 00:25 UTC
Identified
We’re experiencing a service outage with Security Operations Platform. Our team is currently working to restore services. We apologize for any inconvenience. Next update to come as we gather more information.
Posted Nov 15, 2024 - 23:53 UTC
Investigating
We’re currently investigating reports of a potential service interruption with Security Operations Platform. We apologize for any inconvenience and will post another update as soon as we learn more.
Posted Nov 15, 2024 - 23:22 UTC
This incident affected: asia-southeast1 (Singapore) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), europe-west3 (Germany) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), us-west1 (USA) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), australia-southeast1 (Australia) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), northamerica-northeast1 (Canada) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), asia-northeast1 (Japan) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), us-east1 (USA) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), europe-west6 (Switzerland) (Action Editor, Alert Triage, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service), and me-central2 (KSA) (Action Editor, APIs, Auto Parser Generator, Cloud Collectors, Context Management, Correlation Rule Builder, Dashboards, Log Stream, Outcomes Navigator, Search, Security Operations Platform, Service Health and Consumption Dashboard, Site Collectors, Threat Center, Threat Intelligence Service).