Skip to main content

Site CollectorSite Collector Administration Guide

Prerequisites to Install Site Collector

Warning

Ensure that you complete all the prerequisites before installation to avoid installation failure.

Before you install the Site Collector, you must complete the following prerequisites:

  1. Ensure that you have administrative permissions.

  2. Fulfill the environment requirements.

  3. Configure the required network ports.

  4. Mount /*_repository drivers with read/write/exec permissions for root.

  5. Ensure that you have the required software on your VM for one node installation.

  6. Configure your VM for deployment for RedHat 7, 8, and 9 or Ubuntu 18.04/20.04. For detailed instructions about setting up a virtual machine, see Set up Virtual Machine on your Cloud Platform.

  7. Ensure that you do not mount the /tmp folder because Site Collector Core installation and upgrade activities are launched from the /tmp folder.

  8. Run prechecks before you set up the VM. See Run Prechecks to Validate VM Configuration.

Note

  • Site Collectors do not support proxy authentication.

  • Site Collector is installed under User Identifier (UID): 9786.

  • Network File Share (NFS) is not supported.

  • SELinux is disabled by default. Using the enforced mode can affect the operation of the docker. Hence you must carefully configure SELinux in the enforced mode before enabling this mode.

  • Ensure that you disable the IPv6 protocol that is not supported.

  • Ensure that you set the binding on your VM by referring to check host.txt.

  • Site Collectors does not support ingestion of all data events with a raw message size greater than 3MB. For performance optimization and error prevention, through guardrails, Site Collectors trims all data events between 1MB and 3MB to 1MB. Ensure that you ingest up to 1MB of data per log event.

Required Permissions, Operating System, and Packages

Permissions

Root Permissions

Time synchronization

NTP, Chrony

Operating System

  • RedHat 7, 8, and9

  • Ubuntu 18.04, 20.04, 22.04, and 23.04

Note

  • Updating packages in RHEL OS via YUM RedHat Package Manager may result in automatic docker removal. To prevent automatic docker removal, and keep the docker packages locked, add the command exclude=docker* in the /etc/yum.conf file.

  • The supported OS version for Windows Event Log Collector, Windows Archive Collector, and Windows File Collector is Windows server 2016.

Packages

  • bash v4.0 or later to support dictionaries

  • tmux v1.9 or later

  • docker-ce v20.10.10 or later for RHEL and Ubuntu

    Note

    Ensure that you install docker-ce and not other versions of Docker Engine such as Docker Snap. Site Collector runs from /opt/exabeam and uses filesystem for logs whereas Docker Snap accesses files only from the home directory.

    For more information, see Install Docker Engine on Ubuntu and Install Docker Engine on RHEL. When you install the docker-compose-plugin, it installs the docker compose library.

  • docker compose (docker-compose-plugin)

  • curl

  • gawk

  • jq

  • ntpstat

  • openssl

  • rsync

  • sed

  • systemd

  • tar

  • ntp (or chrony)

Site Collector Specifications

To install Site Collector, refer to the following minimum specifications for the virtual machine.

Common and Enterprise Specifications for Site Collector

Storage

Common Specifications

Enterprise Specifications

Disks partition sizes and availability

Create separate physical SSD partitions of 350 GB data in total as follows. Ensure that you allocate the specified free space for each partition.

  • /tmp: 10 GB

  • /var: 50 GB

  • /opt: 15 GB

  • /flowfile_repository: 25 GB

  • /content_repository: 200 GB

  • /provenance_repository: 50 GB

For optimum Site Collector performance, it is recommended to use physical partitions and not logical partitions.

Create separate physical SSD partitions of 900 GB data in total as follows.

Ensure that you allocate the specified free space for each partition.

  • /tmp: 10 GB

  • /var: 50 GB

  • /opt: 15 GB

  • /flowfile_repository: 75 GB

  • /content_repository: 600 GB

  • /provenance_repository: 150 GB

CPU

4 Core

Recommended clock speed: 2.8 GHZ or higher

16 Core

Recommended clock speed: 2.8 GHZ or higher

Memory

16 GB

32 GB

Supported High Level EPS / Throughput

10k EPS/25Mbps

30k EPS/75Mbps for multiple collectors and 10k EPS for an individual collector

Note

  • For optimum Site Collector performance, it is recommended to use physical partitions and not logical partitions.

  • Disk requirements do not include Operating System requirements or requirements for any other application. Hence it is recommended to allocate ~50GB for /root for Site Collector operations, or more space based on your internal policy requirements, in addition to the 350 GB space that is required for physical SSD partitions.

  • All the physical SSD partitions require read, write, and execute permissions.

  • The /tmp directory requires exec permissions for installation and upgrade of Site Collector instances.

  • You can install AV/EDR agents if they do not contradict prerequisites to set up VM for Site Collector. If you use AV/EDR agents, exclude /opt/exabeam/* a directory where Site Collector is installed by default.

  • The content repository partition with a size of 200GB supports data retention up to 18 hours if the GCS bucket connection is interrupted. This supports an average EPS of 8.5k and an average message size of 2.5KB. To increase the data retention time, scale up the disk size for the content repository partition considering that approximately 11GB disk space is required for every 76GB of uncompressed data.

Site Collector Network Ports

The following table displays the network ports that are required for communication with the Site Collector.

Source

Destination

Port

Protocol

Description

All Site Collectors

DNS server

53

DNS

DNS Lookup

All Site Collectors

NTP Server

123

NTP

NTP Server

Administrator Network

All Site Collectors

22

SSH

Administrator command line access to host via an encrypted connection

All Site Collectors

  • oauth2.googleapis.com

  • www.googleapis.com

  • storage.googleapis.com

  • pubsub.googleapis.com

  • accounts.googleapis.com

  • auth.cloud.exabeam.com

  • api.cloud.exabeam.com

443

HTTPS

Port for enabling logs and context upload to Google Cloud Storage/Pub-Sub and for management and monitoring flow

All Site Collectors

  • auth.<region>.exabeam.cloud

  • api.<region>.exabeam.cloud

443

HTTPS

Site Collector management. It is used for heartbeat exchange with the Site Collector app to communicate operational statics, commands, and health information.

Note

The endpoints accounts.google.com and *.auth0.com are no longer required. Update your network configuration to remove these endpoints.

The following table lists the regions that Site Collectors support.

Exabeam Region

GCP Region

Organization URL

Exabeam API Base URL

US West

us-west1

https://org-name.exabeam.cloud/

https://api.prod.exabeam.cloud

US East

us-east1

https://org-name.use1.exabeam.cloud/

https://api.use1.exabeam.cloud

Canada

northamerica-northeast1

https://org-name.ca.exabeam.cloud/

https://api.ca.exabeam.cloud/

Europe

europe-west3

https://org-name.eu.exabeam.cloud/

https://api.eu.exabeam.cloud/

Singapore

asia-southeast1

https://org-name.sg.exabeam.cloud/

https://api.sg.exabeam.cloud/

Japan

asia-northeast

https://org-name.jp.exabeam.cloud/

https://api.jp.exabeam.cloud/

Australia

australia-southeast1

https://org-name.au.exabeam.cloud/

https://api.au.exabeam.cloud/

Note

Based on the Organization URL, find your API Base URL that Site Collector uses to communicate with the backend services.

Additional Ports for Specific Configurations

For deploying additional services, refer to the following table to configure the ports that match your environment.

Source

Destination

Port

Protocol

Description

All Site Collectors

Splunk Server

8089

HTTPS

Default Splunk port. There must be access to the remote Splunk server by its IP/Hostname and port. Ensure that the Splunk server is accessible from each Site Collector node.

All Site Collectors

LDAP (AD) Server

389/636

HTTP/HTTPS

Default LDAP ports. There must be access to the remote LDAP (AD) server by its IP/Hostname and port. Ensure that the LDAP (AD) server is accessible from each Site Collector node.

Any Syslog Client

Site Collector host

Any of 1024 - 49151

TCP

Syslog collector ports. While setting up a Syslog Collector (listener) on Site Collector, configure the Syslog collector port. Open the port for Syslog client. Ensure that you do not use the ports 514, 515, and 601 that are not supported for configuration.

Ports for Windows Event Log Collector Configuration

For deploying Windows Event Log Collector, refer to the following table to configure the ports that match your environment.

Source

Destination

Port

Protocol

Description

Windows Event Log Collector

Site Collector Host

8080

HTTPS

Use this port to enable access from Windows server (not external network access) to Site Collector host.

Windows Event Log Collector

Site Collector Host

8899

HTTPS

Use this port to push data from Windows Event Log Collector to Site Collector.

Windows Event Log Collector

Site Collector Host

8880

HTTPS

Use this port for C2 Server for communication between Minifi (Windows Event Log Collector) and Nifi (WEP).

Windows Event Log Collector

Site Collector Host

9875

HTTP

Use this port to download the Windows Collector installer and installation script.

Windows Event Log Collector

Site Collector Host

9876

HTTPS

Use this port to monitor installation progress from Windows Event Log Collector to Site Collector backend.

Windows Event Log Collector

Site Collector Host

9877

HTTPS

Management Windows

Windows Event Log Collector

Site Collector Host

9878

HTTPS

Use this port to perform remote actions such as upgrade and deletion.

Windows Event Log Collector

Site Collector Host

9879

HTTPS

Use this port to perform remote actions such as upgrade and deletion.

Site Collector Inter-Component Communication Ports

Source

Destination

Port

Protocol

Description

All Site Collectors

nifi.web.http.port

8080

HTTPS

Site Collector web interface port which should be opened for internal communication.

All Site Collectors

nifi.cluster.node.protocol.port

8484

HTTPS

Site Collector cluster port. It is used for the cross-cluster communication between Site Collector nodes. It must be open for internal communication.

All Site Collectors

nifi.cluster.load.balance.port

9093

HTTPS

Site Collector load balancing communication port is used by Site Collector to balance event processing within the cluster. It must be open for the internal communications.

Note

Based on your network security standards, ensure that you block the inter-component communication ports to disable external access. These ports are meant for local communication only and not required to be externally accessible.