Data LakeData Lake Administration Guide

Table of Contents

Exabeam Cloud Telemetry Service

Exabeam telemetry service provides valuable quality and health metrics to Exabeam. System events, metrics, and environment health data are collected and sent to Exabeam Cloud, enabling insight into system issues, such as processing downtime (such as processing delays and storage issues) and UI/application availability.

Learn about the different types of telemetry data, possible telemetry data, and disabling this feature.

Note

If you do not wish to send any data to the Exabeam Cloud, please follow the opt-out instructions listed in the How to Disable Exabeam Cloud Telemetry Service.

Prerequisites

For Exabeam to successfully collect telemetry data, please ensure the following prerequisites are met:

  • Advanced Analytics I48.4 or later with a valid license

  • Data Lake I32 or later with a valid license

  • Access to *.cloud.exabeam.com over HTTPS port 443.

Types of Telemetry Data in Exabeam Cloud Telemetry Service

At a high level, telemetry data falls into one of three categories:

  • Metrics (for example, CPU, events-per-second, and processing delay)

  • Events (for example, machine restart, user login, and configuration changes)

  • Environment (for example, versions, products, nodes, and configuration)

IP addresses and hostnames are masked before being sent to Exabeam Cloud. For example, {"host": "*.*.0.24"}.

Metrics

The example below shows the metrics data sent from the master node to the telemetry service in Exabeam Cloud:

Note

The example below is only a partial example and does not show the full payload.

{ "metrics": [ {"points":[[1558614965, 0.29]], "name": "tm.plt.service_cpu.exabeam-web-common-host1"}, {"points": [[1558614965, 0.3457]], "name": "tm.plt.service_memory.exabeam-web-common-host1"}, {"points": [[1558614965, 0.77]], "name": "tm.plt.service_cpu.mongodb-shard-host1"}, {"points": [[1558614965, 0.04947]], "name": "tm.plt.service_memory.mongodb-shard-host1"} ] }

Events

The example below shows the events data sent from the master node to the telemetry service in Exabeam Cloud:

Note

The example below is only a partial example and does not show the full payload.

{ "events": [ "dateHappened": 1558614965, "title": "Device /dev/shm S.M.A.R.T health check: FAIL", "text": "S.M.A.R.T non-compatible device" ] }

Environment

The example below shows the environment data sent from the master node to the telemetry service in Exabeam Cloud:

Note

The example below is only a partial example and does not show the full payload.

{"environment": { "versions": { "uba": { "build": "4", "branch": "I46.2"}, "common": { "build": "7", "branch": "PLT-i12.5"}, "exa_security": { "build": "33", "branch": "c180815.1"} }, "hosts": { "host3": { "host": "*.*.0.24","roles": ["oar","cm"]}, "host2": {"host": "*.*.0.72","roles": ["uba_slave"]}, "host1": {"host": "*.*.0.70","roles": ["uba_master"]} }, "licenseInfo": { "customer": "EXA-1234567", "gracePeriod": 60, "expiryDate": "10-11-2021", "version": "3", "products": ["User Analytics","Entity Analytics"], "uploadedAt": 1557740839325 } }

Data Collected by Exabeam Cloud Telemetry Service

Exabeam telemetry service provides valuable quality and health metrics to Exabeam. System events, metrics, and environment health data are collected and sent to Exabeam Cloud, enabling insight into system issues, such as processing downtime (such as processing delays and storage issues) and UI/application availability. The table below lists the possible metrics, events, and environment telemetry data.

Note

You can also view a full list of product metrics and events sent to the Exabeam cloud (including when the requests were made and the full payload) by accessing the audit log file located at /opt/exabeam/data/logs/common/cloud-connection-service/telemetry.log.

Environment

Name

Description

Frequency

Inventory

Nodes, masked IPs, and roles of each node.

Once a day

Product Version

Versions of each product in your deployment.

Once a day

License information

License information for each product in your deployment.

Once a day

Metrics for Advanced Analytics

Name

Description

Frequency

tm.aa.processing_delay_sec

An Advanced Analytics processing delay (if applicable) in seconds.

5 mins

tm.plt.service_status.<service-name>

Per-service status.

5 min

tm.plt.ssh_logins

Number of SSH logins.

5 min

tm.plt.service_memory.<service-name>

Per-service memory.

5 min

tm.plt.service_cpu.<service-name>

Per-service CPU.

5 min

tm.plt.load_avg_1m

tm.plt.load_avg_5m

tm.plt.load_avg_10m

Load average (CPU) per 1-minute, 5-minute, and 10-minute period.

5 min

tm.aa.compressed_logs_bytes

Log volume of the last hour.

1 hour

tm.aa.compressed_events_bytes

Events volume of the last hour.

1 hour

tm.aa.notable_users

Notable users.

5 min

tm.plt.disk_usage.mongo

tm.plt.disk_usage.data

tm.plt.disk_usage.root

Disk usage per partition.

5 min

tm.plt.total_users

Total users.

1 hour

tm.plt.total_assets

Total assets.

1 hour

Metrics for Data Lake

Name

Description

Frequency

tm.plt.service_status.<service-name>

Per-service status.

5 min

tm.plt.ssh_logins

Number of SSH logins.

5 min

tm.plt.service_memory.<service-name>

Per-service memory.

5 min

tm.plt.service_cpu.<service-name>

Per-service CPU.

5 min

tm.plt.load_avg_1m

tm.plt.load_avg_5m

tm.plt.load_avg_10m

Load average (CPU) broken per 1-minute, 5-minute, and 10-minute period.

5 min

tm.plt.disk_usage.mongo

tm.plt.disk_usage.data

tm.plt.disk_usage.root

tm.plt.disk_usage.es_hot

tm.plt.disk_usage.kafka

Disk usage per partition.

5 min

tm.plt.total_users

Total users.

1 hour

tm.plt.total_assets

Total assets.

1 hour

tm.dl.es.cluster_status tm.dl.es.number_of_nodes tm.dl.es.number_of_data_nodes tm.dl.es.active_shards tm.dl.es.active_primary_shards

Elasticsearch cluster status.

5 min

tm.dl.kafka.total_lag

A Kafka delay if detected.

5 min

tm.dl.kafka.connectors_lag

A Kafka connector lag if detected.

5 min

tm.dl.avg_doc_size_bytes

Average document size.

15 min

tm.dl.avg_msg_size_bytes

Average message size.

5 min

tm.dl.index_delay

Index delay if detected.

5 min

tm.dl.connectors_send_rate_bytes

Total connector ingestion rate in bytes.

5 min

tm.dl.ingestion_queue

Kafka topic delay if detected.

5 min

tm.dl.indexing_rate

Average indexing rate.

5 min

tm.dl.shards_today

Elasticsearch shards today.

5 min

tm.dl.shards_total

Elasticsearch shards total.

5 min

How to Disable Exabeam Cloud Telemetry Service

Hardware and Virtual Deployments Only

Cloud Telemetry Service will be enabled by default, following the installation of the relevant product versions. Exabeam highly recommends to connect to the Telemetry Service, in order to enjoy the benefits of future enhancements that will be built using this data.

If you do not wish to send any data to the Exabeam Cloud, the steps required vary depending on your deployment scenario:

  • Product Upgrade or Patch Installation

  • Product Installation

  • Any time after Product Upgrade

Disabling Telemetry Before Product Upgrade or Patch Installation

To disable the hosting of telemetry data in the Exabeam Cloud before upgrading your Exabeam product(s):

  1. Access the Cloud Connection Service (CCS) configuration files at:

    /opt/exabeam/config/common/cloud-connection-service/custom/application.conf
  2. Add a new line:

    cloud.plugin.Telemetry.enabled = false
  3. Perform the upgrade steps described in the Upgrade an On-Premises or Cloud Exabeam ProductUpgrade an On-Premises or Cloud Exabeam Product section.

Disabling Telemetry During a Product Installation

To disable the hosting of telemetry data in the Exabeam Cloud while installing your Exabeam product(s):

  1. Perform the installation steps described in the product installation section, but do not upload the product license. You will upload the product license later in this process.

  2. Access the Cloud Connection Service (CCS) configuration files at:

    /opt/exabeam/config/common/cloud-connection-service/custom/application.conf
  3. Add a new line:

    cloud.plugin.Telemetry.enabled = false
  4. Restart CCS by running the following command:

    . /opt/exabeam/bin/shell-environment.bash
    cloud-connection-service-stop && cloud-connection-service-start
  5. Upload the product license by following the steps provided in the Download an On-premises or Cloud Exabeam License and ??? sections.

Disabling Telemetry After Product Upgrade

To disable the hosting of telemetry data in the Exabeam Cloud after upgrading your Exabeam product(s):

  1. Access the Cloud Connection Service (CCS) configuration files at:

    /opt/exabeam/config/common/cloud-connection-service/custom/application.conf
  2. Add a new line:

    cloud.plugin.Telemetry.enabled = false
  3. Restart CCS by running the following command:

    . /opt/exabeam/bin/shell-environment.bash
    cloud-connection-service-stop && cloud-connection-service-start