CollectorsCloud Collectors Administration Guide

Before you configure the AWS CloudWatch Cloud Collector, ensure that you complete the prerequisites.

Log in to the Exabeam Security Operations Platform with your registered credentials as an administrator.

Navigate to Collectors > Cloud Collectors.

Click New Collector.

Click AWS CloudWatch.

Enter the following information for the cloud collector.

• NAME – Specify a name for the Cloud Collector instance.

• Format – Displays JSON format in which the logs are collected from AWS Firehose.

(Optional) SITE – Select an existing site or to create a new site with a unique ID, click manage your sites. Adding a site name helps you to ensure efficient management of environments with overlapping IP addresses.

By entering a site name, you associate the logs with a specific independent site. A sitename metadata field is automatically added to all the events that are going to be ingested via this collector. For more information about Site Management, see Define a Unique Site Name.

(Optional) TIMEZONE – Select a time zone applicable to you for accurate detections and event monitoring.

By entering a time zone, you override the default log time zone. A timezone metadata field is automatically added to all the events that are going to be ingested via this collector.

Click Install.

A confirmation message informs you that the new Cloud Collector is created. Record the authentication token to use it while creating Amazon Data Firehose. Ensure that you record the token immediately after configuration because this token cannot be retrieved later.

The confirmation message window also displays the endpoint /firehose to which logs are sent. Record the link to use it while creating an Amazon Data Firehose.

On Amazon Data Firehose console, click Create Firehose stream.

Select the Source as Direct PUT and the Destination as HTTP Endpoint.

Specify a name for the Firehose stream.

In the Destination settings section, paste the HTTP endpoint URL that you obtained after configuring the AWS CloudWatch Cloud Collector.

In the Access key box, paste the token that you obtained while configuring the collector.

Retain the default settings for Content encoding. By default, CloudWatch sends logs to Data Firehose in GZIP compressed format.

In the Retry duration specify the duration in which Firehose retries to send data to the selected endpoint in case of failure in reaching the endpoint. After the specified time interval, if the error persists, the Destination Error Logs section in Firehose shows the error details.

In Buffer hints section, specify the buffer size for the data to be collected and specify the buffer interval in seconds until which Firehose waits for data collection from CloudWatch.

In the Backup settings section, specify if you want to push only failed data or all the data to the S3 bucket. It is recommended that you select Failed data only to send the data that failed to reach the HTTP endpoint from the firehose to the S3 bucket.

In S3 backup bucket, browse to the folder where your S3 bucket is configured, in which you want to back up the data.

Retain the other default settings and click Create Firehose stream.

For more information, see Create a Firehose stream in the AWS documentation.

The Firehose stream is created. Click the firehose that you created to view details.

Proceed to create a CloudWatch Log group.

Create an IAM role named CWLtoFirehoseRole that grants CloudWatch Logs permission to stream data into your AWS Data Firehose delivery stream.

To create a role using a custom trust policy, on the AWS Management Console, navigate to Roles.

Select Create role.

Select the Custom trust policy role type and paste the following policy details. For more information, see Creating an IAM role and Creating IAM policies in the AWS documentation.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "Service": "logs.<REGION>.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

In the Add permission policies section select the name of the policy that you created such as PermissionsForCWL.

Review the policy details and in the Role Name section specify a name: CWLtoFirehoseRole.

Click Create Role.

Use this role while creating a Subscription Filter.

On Amazon CloudWatch console, navigate to Log groups and click Create log group.

In the Log group details section, specify a name for the log group, and enter the required details.

Click Create.

The log group that you created is listed in the Log groups section. You can monitor the logs generated via Lambda or CloudWatch listed in the log group. For more information, see Working with log groups and log streams in the AWS documentation.