Exabeam Site Collector for SaaSExabeam Site Collector

Exabeam Site Collector

Exabeam SaaS Cloud uses site collectors to enable you to upload log data from your data centers or VPCs to Exabeam. Site collectors in the Exabeam SaaS Cloud were designed to support most data centers with a single Site Collector.

You can configure more than one site collector for your deployment. For example, if you need to collect logs from multiple data centers or VPCs and upload them to Exabeam.

Note

When cloud collector agents are in use, data does not flow through a site collector.

Important

Exabeam's legacy Site Collector (version 2020.04.x) will no longer be supported as of September 1, 2020. Please migrate your site collectors to the Exabeam SaaS Site Collector before the deadline.

Site Collector Architecture

At a high level, Exabeam's site collector involves three main processes:

  • Message collection

  • Data in transit persistence and upload

  • Cloud connectivity

site collector for saas

First, Exabeam collects messages from external servers, systems, data centers, and other machines via syslog and/or Exabeam collectors (including Windows, File, and GZip).

Note

For more information on configuring agent-based or server-side collectors, please refer to the Exabeam Collector GuideExabeam Data Lake Collector Guide.

Then, the site collector queues the messages on disk before uploading to Exabeam SaaS Cloud. The site collector continuously uploads messages from the queue to Exabeam SaaS Cloud. Data is encrypted in transit to the cloud and at rest (in the cloud). Data is compressed to 300-500% of the original source before uploading to Exabeam SaaS Cloud.

Finally, the cloud connectivity feature maintains a healthy connection to Exabeam SaaS Cloud to allow it to connect to customer assets such as AD for context and authentication, access API for log repositories, and any Incident Responder actions.

The site collector includes the following:

  • Kafka for message bus

  • Logstash for message processing

  • OpenVPN for client connectivity

  • Zookeeper for Kafka management

Site Collector Specifications

The Exabeam SaaS Site Collector can be deployed in two capacities, Essential and Enterprise. Please review the specifications in the table below that applies to your environment:

Maximun EPS

Minimum CPU and Memory

Agent Collectors

5k

4 CPU, 8 GB RAM

100

20k

8 CPU, 16 GB RAM

200

30k

16 CPU, 32 GB RAM

500

Table 1. Site Collector Ingestion Capacities


Additionally, please ensure the following storage requirements and permissions are met:

  • CentOS/RedHat 7.x or later

  • / must have a minimum 50 GB is required for SC operations

  • /tmp must have full root permissions

  • Ensure / and /opt are configured for disk usage

  • /data is storage for Kafka data (sizing is based on the Site Collector Specifications above) with 300 GB or higher per EPS

  • Default local retention is 24 hours or available free disk space in /data allocation

The following table is a guide for the Essential type SaaS Collector. It is based on calculations for:

  • 1500 bytes average message size

  • EPS rated for Essential Tiny, Small, Medium, and Large are 1k, 2k, 3.5k, 5k EPS respectively

  • Extra space reserved and calculated from ratings of thousands of events per second (EPS) per day to be stored locally

  • We recommend and ship SC with 24 hours local retention by default so that you have time to remedy issues which might occur.

Retention Hours

1k EPS

(GB)

5k EPS

(GB)

10k EPS

(GB)

20k EPS

(GB)

30k EPS

(GB)

2

50

250

500

1000

1500

4

100

500

1000

2000

3000

6

150

750

1500

3000

4500

8

200

1000

2000

4000

6000

12

300

1500

3000

6000

9000

24

600

3000

6000

12000

18000

Table 2. Essential Type Site Collector - Storage Capacities by Retention


Important

For capacity specifications that are not shown, please contact your Exabeam technical representative for assistance in calculating retention and EPS rates.

Default retention is 24 hours.

Where possible we recommend there is at least 2 SCs deployed behind a load balancer for high availability performance. You can deploy as many SCs as required for your logs processing. One SC must have OpenVPN if your ingestion is to support LDAP polling, database logs, eStreamer logs and fetching by Advanced Analytics or Incident Responder accessing local endpoints.

Here are some deployment examples based on log ingestion rates:

Collector Type

CPU, Memory, and Storage

Essential, 2.5k EPS

4 CPU, 8 GB RAM, 1500 GB

Enterprise, 10k EPS

2 SCs with 4 CPU and 8 GB RAM each, 6000 GB in total (recommended)

or

1 SC with 8 CPU and 16 GB RAM, 6000 GB in total

Enterprise, 20k EPS

SC 8 CPU, 16 GB RAM, 12000 GB

Enterprise, 30k EPS

SC 16 CPU, 32 GB RAM, 18000 GB

Enterprise, 75k EPS

4 SCs, 8 CPU and 16 GB RAM each, 45000 GB in total

or

3 SCs, 16 CPU and 32 GB RAM each, 45000 GB in total

Table 3. Specification Examples


Required Services

  • A NTP client must be active.

  • Hosts running SELinux mode must be set to permissive

  • Please ensure firewalld is installed and present on the system.

Note

The firewalld service automatically installs if it is not already present on the system.