Skip to main content

Site CollectorSite Collector Administration Guide

Site Collector Overview

Site Collector provides a cloud-native data platform for ingesting data from on-premises log sources and active directories into the Exabeam Security Operations Platform with easy onboarding process, monitoring, and management through centralized cloud user interface.

Site Collector contains Site Collector core, Site Collector app, and Collectors to:

  • Streamline data onboarding by using common templates and use cases

  • Apply data compression before logs are uploaded to Exabeam cloud

  • Provide clarity for data processing and processed log volume reporting

  • Offer proactive monitoring and data altering

Site Collector Architecture

Site Collector collects messages, transfers, persists, and securely uploads event data; and connects to the Exabeam Security Operations Platform. Site Collector is the managed entry point for logs to be routed to other processing tools, such as Advanced Analytics, Data Lake, Incident Responder, and Case Manager in the Exabeam Security Management Platform. Site collectors gather logs from external servers, systems, and data centers.

Architecture_diagram_NGSC_Updated_1.png

Component

Description

Site Collector

Ingests data from on-premises log sources and on-premises active directory (AD) into Exabeam Cloud with a simple onboarding process, basic monitoring, and management over centralized cloud user interface (UI).

Site Collector Core

Helps to:

  • Set up various collectors to onboard logs and context data

  • Upload logs to Exabeam Security Operations Platform

  • Perform continuous monitoring of each collector and collected log volumes

Site Collector App

Enables end to end log collector setup, configuration, and monitoring. The application is integrated with Site Collector Core, and other Exabeam Cloud applications and products.

Collector

Includes Site Collector flows, out-of-the-box processors and groups, custom processors, and other components and integrations developed for receiving logs or context from various sources, and sending the logs or context to the Exabeam Security Operations Platform.

There are two types of collectors: agent collectors and server-side collectors.

  • Agent collectors are the collectors that you install on the remote Windows or Linux devices. For example, Windows Event Log Collector, Windows File Collector, and Linux Archive Collector.

  • Server-side collectors are the collectors that are not installed on remote devices and that run directly on the Site Collector instances. Examples of server-side collectors include Syslog, Splunk, Oracle, MSSQL, MySQL, Qradar, and Estreamer collectors.

Windows Active Directory (formerly known as LDAP) Collector

Includes Site Collector flows, built-in processors, groups, custom processors, other components, and integrations aimed at pulling context data from your active directory (AD), and pushing the data to the Exabeam Security Operations Platform for contextual data processing.

Splunk Collector

Pulls logs from your Splunk server and pushes the logs to the Exabeam Security Operations Platform.

At collector level, a single Splunk collector instance can ingest 7k EPS events at 2.5KB average size. For enterprise specifications, you can simultaneously run more than two collector instances at combined 30k EPS.

It is recommended to use the shortest fetch interval such as 30 seconds, for optimized performance.

Syslog Collector

Includes Site Collector flows, out-of-the-box processors, groups, custom processors, other components, and integrations aimed at getting syslog logs from various sources and pushing the logs to Exabeam Security Operations Platform. For a successful use case implementation in Exabeam Cloud and easy monitoring of log sources, the Syslog collector helps to enable log onboarding from various sources. Ingests up to 10k EPS events per Site Collector instance with the average log size of 2.5k.

For Syslog collector instances, each site Collector instance can ingest up to:

  • 10k EPS with the average log size of 2.5 kb over TCP with no data loss

  • 11k EPS with the average log size of 1 kb over UDP with no data loss

  • 11k EPS with the average log size of 2.5 kb over UDP with 0.05% data loss

If you have multiple collector instances in the same Site Collector instance, the EPS will be split between the collector instances. To achieve higher EPS, create multiple Site Collector instances to split the traffic.

Windows Event Log Collector

Includes a set of Site Collector flows, pre-built processors, groups, custom processors, other components, and integrations that pull logs in XML format from your Windows server and push the logs to Exabeam Security Operations Platform. The collector provides flexible template configuration capabilities to collect Windows events.

A Windows Event Log Collector instance runs on its own Windows sever and can support up to 500 EPS at 10% CPU utilization, and 2k EPS at 40% CPU utilization.

A single Site Collector instance runs up to 100 Windows Event Log Collector instances on a VM with common specifications.

A single Site Collector instance runs up to 500 Windows Event Log Collector instances on a VM with enterprise specifications.

It is recommended to set up one Windows Event Log Collector instance at a time, and not to set up multiple collector instances simultaneously.

Note

  • Site Collectors app maintains a balanced compression ratio and speed, and uses gzip compression mode. To enable you to enhance data throughput in the network by decreasing the data size, Site Collectors use 4-10x data compression rate for log data and 3-4x data compression rate for context data. The compression ratio varies per log source type.

  • Using an external load balancer, the Syslog data can be load balanced to ensure that the EPS is evenly distributed, and log collection is not impacted during scheduled updates. It is recommended that you deploy at least 2 site collector instances per load balancer for optimum performance. TCP load balancing is required for syslog.

  • Data at rest is encrypted.