Skip to main content

Site CollectorSite Collector Administration Guide

Set Up Syslog Collector

Syslog Collector is a set of Site Collector flows, out-of-the-box processors, groups, custom processors, other components, and integrations aimed at getting syslog logs from various sources such as Linux OS logs, FTP, and network, and pushing the logs to Exabeam Security Operations Platform for further processing. For a successful use case implementation in Exabeam Cloud and easy monitoring of log sources, the Syslog collector helps to enable log onboarding from various sources.

Note

The Site Collectors service supports ingestion of single line and multi-line syslog events. The events can be of various Syslog protocols such as RFC 5424, RFC 5425, RFC 5427, and RFC 5848.

To set up the Syslog collector:

  1. Log in to the Exabeam Security Operations Platform with your registered credentials.

  2. Navigate to Collectors > Site Collectors.

  3. Ensure that Site Collector is installed and in running state.

  4. On the Site Collector page, click the Collectors Library tab, then click Syslog.

    Syslog_1_Oct_23.png

  5. In the Definition section, enter the required information as follows.

    Syslog_config_1.png

    • Collector Name – Specify a name for the Syslog collector.

      Note

      Ensure that you specify different names for Site Collector instance and the Syslog collector.

    • Site Collector Instance – Select the site collector instance for which you want to install the Syslog collector.

    • Port – Enter the TCP port number of the Syslog server, from the supported port range 1024 - 49151.

      Note

      If you use port 514, forward the port on the Site Collector VM and set up the Syslog Collector with port 1514 or any other port from the supported port range.

  6. Click Next.

  7. In the Authentication section, select the required protocol from the list: UDP or TCP.

    • UDP – For faster log collection and reduced transmission overhead, considering lower network costs for logs with increased variability, select UDP. A huge amount of Syslog data within the same network with minimal packet loss is transmitted over UDP.

      Syslog UDP supports maximum 1024 Byte message size because of Syslog UDP RFC, and 10k EPS with minimal data loss.

      Use UDP when the log size is less than 1Kb because based on the UDP protocol, UDP packets are of 1Kb size and there is minimal data loss for the log size that is less than 1kb. To optimize data loss prevention for Syslog UDP, the updated configuration changes related to operating system have been implemented. The Operating System level socket receive buffer size is configured as follows:

      • net.core.rmem_default = 17039360

      • net.core.rmem_max = 17039360

    • TCP – If you need reliable and ordered data transmission between networks, select TCP, to transmit Syslog data over TCP securely. To configure a secure syslog ingestion, select TCP and enable the Secure Connection option.

      Syslog_config_2.png

      To enable secure and encrypted TLS communication between your syslog server and the Syslog Collector, download the Exabeam generated default security certificates, or upload your custom certificates. You can use default or custom certificates.

      Click DEFAULT CERTIFICATE, and click Download Certificate. Then save and apply the downloaded certificates on your syslog server.

      Custom_certificate.png

      To use custom certificates signed by Certificate Authority (CA), click CUSTOM CERTIFICATE, and click Select File. Upload the certificate files: cert.pem, key.pem, and (optional) ca.pem that are bundled in a .tar.gz archive file. If you upload the optional ca.pem, all the subsequently uploaded certificates that are signed by this Certificate Authority (CA) will be accepted. To know how to generate the cert.pem, key.pem, and ca.pem files, see Generate Custom Certificate Using OpenSSL.

      Custom_certificate_2.png

      In addition to uploading the certificates in .tar.gz format, you can paste the content of your custom certificates that are in key.pem, cert.pem, or ca.pem format into the relevant text box.

      syslog_Collector_paste_certificate.png

  8. In the Event type section, select the required event type in which you want the collector to read logs: SINGLE LINE or MULTI-LINE.

    Syslog_config_3.png

    • If you select SINGLE LINE, the collector reads ingested logs as one message. The message is split by line breaks.

    • If you select MULTI-LINE, specify a regex pattern (regular expressions) to define where a message starts and ends which extends across multiple lines, by clicking +Pattern.

      To test the accuracy of the regular expressions, click Check Patterns.

      Syslog_config_4.png

  9. To set up the Syslog collector, click Setup.

    The configuration for Syslog Collector is complete. The collector is set up and ready to receive the logs pushed by syslog sources.

    Syslog_step3.png

While configuring the Syslog Collector, if you need assistance for managing ports, refer to the following sections on managing ports. For more information, refer to Set up virtual machine on your cloud platform.

Generate Custom Certificate Using OpenSSL

Using the open-source command-line toolkit OpenSSL that allows you to perform SSL and TLS protocols related tasks, you can create custom certificates to ensure secure and encrypted TLS communication between your syslog server and the Syslog Collector.

To generate an SSL certificate using the OpenSSL, perform the following steps.

To generate a private key for the Certificate Authority (CA), use the following command.

openssl genrsa 4096 > ca-key.pem

To generate the X509 certificate for the CA, use the following command.

openssl req -new -x509 -nodes -days 365 \
   -key ca-key.pem \
   -out ca.pem \
   -subj "/CN=ca"

To generate the private key and certificate request, use the following command.

openssl req -newkey rsa:4096 -nodes -days 365 \
   -keyout key.pem \
   -out req.pem

To generate the X509 certificate for the server, use the following command.

openssl x509 -req -days 365 -set_serial 01 \
   -in req.pem \
   -out cert.pem \
   -CA ca.pem \
   -CAkey ca-key.pem \
   -subj "/CN=server"

Manage Ports with Firewall on Ubuntu

If you use Ubuntu, use Uncomplicated Firewall (ufw) that is a frontend for iptables. To enable ufw firewall, allow ssh access, enable logging, and check the status of the firewall by using the following commands.

  1. Before enabling the firewall for Ubuntu, use the following command to allow SSH.

    sudo ufw allow ssh
sudo ufw enable;sudo ufw reload;sudo ufw status

  2. To ensure that the destination port is enabled for TCP traffic, use the following command:

    sudo ufw allow exposed_port/tcp

sudo ufw allow effective_port/tcp

  3. To edit the UFW configuration, use the following command:

    sudo vi /etc/ufw/before.rules

  4. Before * filter section, insert the following commands, modify the parameters exposed_port and effective_port with the port number on which you want to enable the requests from outside and port number of the port on which the service is listening respectively.

    *nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -p tcp --dport exposed_port -j REDIRECT --to-port effective_port
COMMIT

  5. To enable and reload the firewall, use the following command:

    sudo ufw enable
sudo ufw reload

  6. To check the status, run sudo ufw status and get the expected output as follows:

    Status: active

To                         Action      From
--                         ------      ----
exposed_port/tcp           ALLOW       Anywhere                  
effective_port/tcp         ALLOW       Anywhere                  
exposed_port/tcp (v6)      ALLOW       Anywhere (v6)             
effective_port/tcp (v6)    ALLOW       Anywhere (v6)

Manage Ports with Firewall on RHEL

If you use RHEL, before enabling the firewall for RHEL, use the following command to allow SSH.

sudo firewall-cmd --permanent --add-service ssh
sudo firewall-cmd --reload

You must see 'success' after you run the above commands. Following is an example of the output that you receive.

[exabeam@ngsc ~]$ sudo firewall-cmd --permanent --add-service ssh
success
[exabeam@ngsc ~]$ sudo firewall-cmd --reload
success

To validate your changes after reloading, use the following command.

sudo firewall-cmd --list-services

You can now see ssh and any other services you have configured here. Following is an example of the output that you receive. This allows users to SSH into the VM after firewall is enabled, to execute shell command for installing Site Collector.

[exabeam@ngsc ~]$ sudo firewall-cmd --list-services
ssh

To setup port forwarding, run the following commands:

sudo firewall-cmd --add-forward-port=port=514:proto=tcp:toport=1514 --permanent

sudo firewall-cmd --reload