Advanced AnalyticsExabeam Advanced Analytics Administration Guide

Peer Groups

Peer groups can be a team, department, division, geographic location, etc. and are defined by the organization. Exabeam uses this information to compare a user's behavior to that of their peers. For example, when a user logs into an application for the first time Exabeam can evaluate if it is normal for a member of their peer group to access that application. When Dynamic Peer Grouping is enabled, Exabeam will use machine learning to choose the best possible peer groups for a user for different activities based on the behaviors they exhibit.

Executives

Exabeam watches executive movements very closely because they are privileged and have access to sensitive and confidential information, making their credentials highly desirable for account takeover. Identifying executives allows the system to model executive assets, thereby prioritizing anomalous behaviors associated with them. For example, we will place a higher score for an anomaly triggered by a non-executive user accessing an executive workstation.

Service Accounts

A service account is a user account that belongs to an application rather than an end user and runs a particular piece of software. During the setup process, we work with an organization to identify patterns in service account labels and uses this information to classify accounts as service accounts based on their behavior. Exabeam also adds or removes points from sessions based on service account activity. For example, if a service account logs into an application interactively, we will add points to the session because service accounts should not typically log in to applications.

Workstations & Servers

Assets are computer devices such as servers, workstations, and printers. During the setup process, we will ask you to review and confirm asset labels. It is important for Exabeam to understand the asset types within the organization - are they Domain Controllers, Exchange Servers, Database Servers or workstations? This adds further context to what Exabeam sees within the logs. For example, if a user performs interactive logons to an Exchange Server on a daily basis, the user is likely an Exchange Administrator. Exabeam automatically pulls in assets from the LDAP server and categorizes them as servers or workstations based on the OS property or the Organizational Units they belong to. In this step, we ask you to review whether the assets tagged by Exabeam are accurate. In addition to configuration of assets during setup, Exabeam also runs an ongoing classifier that classifies assets as workstations or servers based on their behavior.

Network Zones

Network zones are internal network locations defined by the organization rather than a physical place. Zones can be cities, business units, buildings, or even specific rooms. For example, "Atlanta" can refer to a network zone within an organization rather than the city itself (all according to an organization's preference). Administrators can upload information regarding network zones for their internal assets via CSV or add manually one at a time.

Asset Groups

Asset Groups are a collection of assets that perform the same function in the organization and need to be treated as a single entity from an anomaly detection perspective. An example of an asset group would be a collection of Exchange Servers. Grouping them this way is useful to our modeling processing because it allows us to treat an asset group as a single entity, reducing the amount of false positives that are generated when users connect to multiple servers within that group. As a concrete example, if a user regularly connects to email exchange server #1 then Exabeam builds a baseline that says this is their normal behavior. But exchange servers are often load-balanced, and if the user then connects to email exchange server #2 we can say that this is still normal behavior for them because the exchange servers are one Asset Group. Other examples of asset groups are SharePoint farms, or Virtual Desktop Infrastructure (VDI).

Exabeam supports Common Access Card (CAC) authentication. CAC is the principal card used to enable physical spaces, and it provides access to computer networks and systems. Analysts have CAC readers on their workstations to read their Personal Identity Verification (PIV) credentials and authenticate them to use various network resources.

Note the following restrictions:

Customers are able to control the responsibilities and activities of their SOC team members with Role-based Access Control (RBAC). Local users, LDAP users or SAML authenticated users will be assigned roles within Exabeam.

Each user can be assigned one or more roles and the responsibilities of those roles are determined by the permissions their role allows. If users are assigned more than one role, that user receives the permissions of both roles.

To access the Roles page, navigate to Settings > User Management > Roles.

Users are the analysts that have access to the Exabeam UI to review and investigate activity. These analysts also have the ability to accept sessions. Exabeam supports local authentication or authentication against an LDAP server.

Understand the difference between Roles and Users. Configure the analysts that have access to the Exabeam User Interface, add the analyst's information, assign them roles, and set up user permissions and access based on your organization's needs.

With SAML authentication, Exabeam can integrate with SSO vendors such as Okta, Ping, Google, and Microsoft Active Directory Federation Services. You can also configure Exabeam for SSO through a custom or generic identity provider (IdP). SSO integration eliminates the need for users to manage separate sign-on credentials for Exabeam.

Logs tell Exabeam what the users and entities are doing while context tells us who the users and entities are. These are data sources that typically come from identity services such as Active Directory. They enrich the logs to help with the anomaly detection process or are used directly by the risk engine layer for fact-based rules. Regardless of where these external feeds are used, they all go through the anomaly detection layer as part of an event. Examples of context information potentially used by the anomaly detection layer are the location for a given IP address, ISP name for an IP address, and department for a user.

Analysts are able to view and edit Exabeam's out-of-the-box context tables as well as create their own custom tables. They can select a specific table, such as Executive Users, Service Accounts, etc. and see the details of the table and all of the objects within the table. Edits can be performed on objects individually or through CSV uploads.

Exabeam provides several filters and lookups to get your security deployment running immediately. However, there may be assets and users within your organization that need particular attention and cannot be fully addressed out of the box. Custom context tables allow you the flexibility to create watchlists or reference lists for assets, threat intelligence indicators, and users/groups that do not fit in the typical deployment categories. Custom context tables let you put parts of your organization under extra monitoring or special scrutiny, such as financial servers, privileged insiders, and high-level departed employees.

Within Advanced Analytics, you can create watchlists using context tables. When creating the table, the Label attribute allows you to attach tags to records that match entries in your context table. This provides quick access to query your results and/or focus your tracking using a global characteristic.

You can also build rules based on entries in your context tables. Set up alerts, actions, or playbooks to trigger when conditions match records, such as access to devices in a special asset group.

Context Data

Prepare Context Data

You can upload data as CSV files with either key and value columns or key-only column. All context tables include a Label to tag matching records into groups during parsing and filtering.

Key-value CSV –Two-field data file with a header row. This lookup lists correlations between the two fields, such as:

Key Fieldname	Value Fieldname
AC1Group	Accounts Receivable
AC2Group	Accounts Payable

Key-only CSV – Single-field data file with no header row. Items on this list are compared to as being present or not during data filtering. For example, a watchlist context table, SpecialGroup, consists of user groups of special interest:

“Accounts Receivable”

“Accounts Payable”

“Accounting Database Admin”

You can create a correlation rule that sends an alert when the monitoring data contains a user having the group name that matches any in the SpecialGroup table.

Label – The named tag associated with a record. This allows you to filter groups of records during parsing or filtering. You can also use labels to assemble watchlists based on groupings rather than by individual asset or user record.

Note

You can opt not to use labels by selecting No Label during table creation. Otherwise, labels are associated with tables and its records. For key-value context tables, the Label is drawn from the value field of the matching context table entry. For key-only context tables, the Label is the table attribute you enter in the Manual Assignment field during table creation and is used to tag all matching records.

Import Data into a Context Table Using CSV

This is the most flexible method to create unconventional context tables as the CSV file can contain any category or type of data that you wish to monitor.

1. Select your desired context table.

2. Select the Upload Table icon.

3. Click Upload CSV. From your file system, select the CSV file you wish to import, then select Next.

   Note

   Key and value (2 fields) tables require a header first row. Do not include a header for keys-only CSV files (1 field). Table names may be alpha-numeric with no blank spaces. (Underscore is acceptable.)

4. Inspect the contents that will be added to your table. Select Apply Changes, when you are done.

Once context has been integrated, it is displayed in the table. You can use the lookup tables in rules as required.

For assistance in creating custom context tables, contact Exabeam Customer Success by opening a case at Exabeam Community.

You can configure static mappings from hosts to IP addresses, and vice versa. This is especially useful for mapping domain controllers (DCs). Since DCs do not often change IPs, you can tie the DC hostname to a specific IP address. Additionally, if there is user activity that isn't tied to a hostname but is tied to an IP address, then you can map the user to their specific, static IP address. This helps maintain and enrich information in events that may be lost or unknown since the system cannot tie events to dynamic IP addresses.

Map IP addresses to hosts

Add them to the file: /opt/exabeam/data/context/dynamic_objects/static_ip_host_mapping.csv

CSV Format: [ip], [host]

Map hosts to IP addresses

Add them to the file: /opt/exabeam/data/context/dynamic_objects/static_host_ip_mapping.csv

CSV Format: [host], [ip]

Proxy and other generic sequence events (such as, web, database, file activity, endpoint) as well as some security and DLP alerts may generate logs that contain only machine names or IP addresses without the user names. In Advanced Analytics , you can automatically associate these events with users by IP/host-to-user mapping.

UserPresentOnHostIf {
 kerberos-logon = {
  Condition = "not (EndsWith(user, '$') OR InList(user, 'system', 'local service', 'network service','anonymous logon'))"
  UserPresentOn = ["dest_host", "dest_ip"]
 }
 remote-logon = {
  Condition = "not (EndsWith(user, '$') OR InList(user, 'system', 'local service', 'network service','anonymous logon'))"
  UserPresentOn = ["dest_host", "src_host", "dest_ip", "src_ip"]
 }
 remote-access = {
  Condition = "InList(ticket_options, '0x40800000', '0x60810010') && not (EndsWith(user, '$') OR InList(user, 'system', 'local service', 'network service', 
'anonymous logon'))"
  UserPresentOn = ["src_host", "src_ip"]
 }
}

exabeam-analytics-stop
exabeam-analytics-start

HostToUserMerger {
 Enabled = true
 EventTypes = [
  {
   EventType = "web-activity-allowed"
   MergeFields = ["src_host", “src_ip”]
  },
  {
   EventType = "web-activity-denied"
   MergeFields = ["src_host"]
  }
 ]
} 

exabeam-analytics-stop
exabeam-analytics-start

You can create and display a custom login message for your users. The message is displayed to all users before they can proceed to login.

To display a custom login message:

Your custom login message is now shared with all users before they proceed to the login screen.

You can configure the maximum search result limit when using Threat Hunter’s search capabilities. By default, the result limit is set to 10,000 sessions.

The default result limit is located in the application_default.conf file at /opt/exabeam/config/tequila/default/application_default.conf.

All changes should be made to

/opt/exabeam/config/tequila/custom/application.conf.

To configure the default result limit, enter an acceptable value in place of 10000 at tequila.data.criteria:

finalQueryResultLimit = 10000

There is no restriction on the limit value, however, for very large intermediate results you should input at least 30,000 sessions.

Dates and times may appear slightly different between Advanced Analytics, Case Manager, and Incident Responder.

Machine Learning (ML) algorithms require a different infrastructure than regular deployments. This infrastructure is necessary to run data science algorithms. ML infrastructure will install two new docker-powered services: Hadoop YARN and Advanced Analytics API.

Installation is only supported on EX4000 powered single- or multi-node deployments running Advanced Analytics i35 or later due to the high system resources needed for these jobs. ML infrastructure is a requirement for algorithms that drive the Personal Email Detection, Daily Activity Change Detection, and Windows Privileged Command Monitoring features.

EventStore.Enabled = true
EventStore.UseHDFS = true

/opt/exabeam/ds-server/config/default/script.confEventStoreHDFSPathTemplates = [
"hdfs://hadoop-master:9000/opt/exabeam/data/input/(YYYY-MM-dd)/(HH).*.evt.gz"
]
/opt/exabeam/config/custom/custom_exabeam_config.conf
LogFetcher {
UseHDFS = true
LogDir = "/opt/exabeam/data/input"
# this values by default, you don’t have to override it in this config
HDFSHost = "hadoop-master"
HDFSPort = 9000
}

{
EventStoreHDFSPathTemplates = [
"hdfs://hadoop-master:9000/opt/exabeam/data/output/(YYYY-MM-dd)/(HH).[type]-events-{m,s?}.evt.gz",
"hdfs://hadoop-master:9000/opt/exabeam/data/output/(YYYY-MM-dd)/(HH).[type]-events-{m,s?}.[category].evt.gz"
]
EventStore {
#Event type. Can be Raw, Container or Any
Type = "Container"
#Event category. All available categories are in event_categories.conf
Categories = ["all"]
}

{
EventStoreHDFSPathTemplates = [
"hdfs://hadoop-master:9000/opt/exabeam/data/input/(YYYY-MM-dd)/(HH).*.[category].evt.gz"
]
EventStore {
#Event category. All available categories are in event_categories.conf
Categories = ["all"]
}

LogFetcher {
UseHDFS = true
# this path to events folder in HDFS should be the same as in
# script.conf EventStoreHDFSPathTemplates, main difference
LogDir = "/opt/exabeam/data/input"
# this values by default, you don’t have to override it in this config
HDFSHost = "hadoop-master"
HDFSPort = 9000
}

EventStoreHDFSPathTemplates = [
"hdfs://hadoop-master:9000/opt/exabeam/data/input/(YYYY-MM-dd)/(HH).*.[category].evt.gz"
]
LogDir = "/opt/exabeam/data/input"

personal-email-identification {
...
EventStore {
Type = “Container”
Categories = ["alerts"]
}
...
}

personal-email-identification {
...
EventStore {
Categories = ["alerts"]
}
...
}

ds-server-stop
aa-api-stop
hadoop-yarn-stop

Advanced Analytics now detects users who visit suspected phishing websites. Phishing often starts with a domain name string that has the look-and-feel of a legitimate domain, but is not. Phishers target the Internet's most recognizable domain names (google.com, yahoo.com, etc.) and make slight changes on these domain names in order to fool unassuming eyes. Phishing detection uses lexical analysis to identify whether a domain is a variant of popular domain names. In addition, it also checks URLs against a white-list of popular legitimate domains and a blacklist of identified suspicious domains. It also uses substring searches to identify domains that contain the domain name of a popular site as a substring within the suspect domain. For example, www.gmail.com-hack.net contains the recognizable "gmail.com" within the top level domain.

Associated Rules:

Configuration

Configuration should be made in /opt/exabeam/config/custom/custom_exabeam_config.conf.

To enable Phishing Detection, set PhishingDetector.Enabled = true.

Support Information

Supported in single and multi-node environments with EX2000 and EX4000.

Administrators typically need to restart the Analytics Engine when configuration changes are made to the system such as adding new log feeds to be analyzed by Exabeam or changing risk scores to an existing rule.

Exabeam will store time-based records in the database for recovery and granular reprocessing. The histograms and the processing state of the Exabeam Analytics Engine are time-stamped by week and stored in the database. This allows the Exabeam Analytics Engine to be able to go back to any week in the past and continue processing.

To illustrate, let's say that the Exabeam Analytics Engine started processing logs from January 1, 2016, and is currently processing today, April 15, 2016. The administrator would like to ingest new Cloud application log feeds into Exabeam and start reprocessing from a time in the past, say March 30, 2016. The administrator would stop the Exabeam Analytics Engine and then restart processing from March 30, 2016. The system will go back to find the weekly boundary where the state of the nodes and the models are consistent - which might mean a few days before March 30, 2016 - and start processing all the configured log feeds from that point in time.

Navigate to Settings > Admin Operations > Exabeam Engine.

Upon clicking Restart Processing, the Processing Feeds page appears. You can choose to:

If you have made configuration changes then the system will check for any inadvertent errors in the configuration files before performing the restart. If the custom configuration validation does identify errors in the config files then it will list the errors and not perform the restart. Otherwise, it will restart the analytics engine as usual.

So what exactly is a rule anyway? There are two types of Exabeam rules:

Model-based rules rely on a model to determine if the rule should be applied to an event in a session, while fact based rules do not.

For example, a Fireye malware alert is fact based and does not require a model in order to be triggered. On the other hand, a rule such as an abnormal volume of data moved to USB is a Model-based rule.

Model-based rules rely on the information modeled in a histogram to determine anomalous activities. A rule is triggered if an event is concluded to be anomalous, and points are allocated towards the user session in which the event occurred. Each individual rule determines the criticality of the event and allocates the relevant number of points to the session associated with that event.

Taken together, the sum of scores from the applied rules is the score for the session. An example of a high-scoring event is the first login to a critical system by a specific user – which allocates a score of 40 to a user’s session. Confidence in the model must be above a certain percentage for the information to be used by a rule. This percentage is set in each rule, though most use 80%. When there is enough reliable information for the confidence to be 80% or higher, this is called convergence. If convergence is not reached, the rule cannot be triggered for the event.

Since anomaly-based rules depend on models, it is helpful to have a basic understanding of how Exabeam's models work.

Our anomaly detection relies on statistical profiling of network entity behavior. Our statistical profiling is not only about user-level data. In fact, Exabeam profiles all network entities, including hosts and machines, and this extends to applications or processes, as data permits. The statistical profiling is histogram frequency based. To perform the histogram-based profiling, which requires discrete input, we incorporate a variety of methods to transform and to condition the data. Probability distributions are modeled using histograms, which are graphical representations of data. There are three different model types – categorical, numerical clustered, and numerical time-of-week.

Categorical is the most common. It models a string with significance: number, host name, username, etc. Where numbers fall into specific categories which cannot be quantified. When you model which host a user logs into, it is a categorical model.

Numerical Clustered involves numbers that have meaning – it builds clusters around a user’s common activities so you can easily see when the user deviates from this norm. For example, you can model how many hosts a user normally accesses in a session.

Numerical Time-of-Week models when users log into their machines in a 24-hour period. It models time as a cycle so that the beginning and end of the period are close together, rather than far apart. For example, if a user logs into a machine Sunday at 11:00 pm, it is closely modeled to Monday at 12:00am.

Over time, models built in your deployment naturally become outdated. For example, if an employee moves to a different department or accepts a promotion and they do not adhere to the same routines, access points, or other historical regularities.

We automatically clean up and rebuild all models on a regular basis (default is every 16 weeks) to ensure your models are as accurate and up-to-date as possible. This process also enhances system performance by cleaning out unused or underutilized models.

Exabeam has an internal Rule ID naming convention that is outlined below. This system is used for Exabeam created rules and models only. When a rule is created or cloned by a customer, the system will automatically create a Rule ID for the new rule that consists of customer-created, followed by a random hash. For example, a new rule could be called, customer-created-4Ef3DDYQsQ {.

The Exabeam convention for model and rule names is: ET-SF-A/F-Z

ET: The event types that the model or rule addresses. For example,

SF: Scope and Feature of the model. For example,

A/F: For rules only

Z : Additional Information (Optional). For example,

The Advanced Editor is a JSON style editor and is what an administrator would use if they wanted to edit one of Exabeam's existing rules, or edit a cloned rule. All of Exabeam's out-of-the-box rules can be edited only via the Advanced Editor.

This editor shows the entire rule as it exists in the configuration file. The Rule ID is the only field that cannot be changed. See the Rule Naming Convention section in this document for more information about Exabeam's naming convention. When an administrator makes any changes to a rule, the rule is validated during the save operation. If the rule has incorrect syntax, the administrator is prompted with the error and the details of the error. Once a rule is edited and saved using the Advanced Editor, the rule cannot be viewed via the Simple Editor.

Before configuring Threat Intelligence Service, ensure your deployment meets the following prerequisites:

The communication between Threat Intelligence Service and Advanced Analytics occurs over a secure HTTPS connection.

If connections from your organization do not make use of a web proxy server, you may skip this section. Threat Intelligence Service is available automatically and does not require additional configuration.

If connections from your organization are required to go through a web proxy server to access the Internet, you will need to provide the configuration as shown below.

Before configuring Threat Intelligence Service, ensure your deployment meets the following prerequisites:

As soon as the deployment can successfully connect to Threat Intelligence Service, threat intelligence feed data is pulled and saved as context tables and is also viewable on the Threat Intelligence Feeds settings page.

To view threat intelligence feeds, navigate to Settings > Cloud Config > Threat Intelligence Feeds.

The page displays details related to all threat intelligence feeds provided by the cloud-based Exabeam Threat Intelligence service, such as:

If you want to collect data from some of the threads in other context tables, see the next section on Assigning a Threat Intelligence Feed to a New Context Table.

Some of the feeds are pre-assigned to context tables. Click the arrow of a threat intelligence feed to expand and view additional details, including:

Click the edit icon of a single threat intelligence feed to assign or unassign it to one or more context tables.

Click the view icon to view existing indicators in the context table.

Select multiple threat intelligence feeds to bulk assign or unassign them from context tables.

To view the current context tables provided by Threat Intelligence Service:

To view the current status of the ExaCloud connector service:

The service availability icon shows the current health of the Cloud connector service that is deployed on your Exabeam product.

The two-cluster scenario employs an Active-Passive Disaster Recovery architecture with asynchronous replication.

With this approach, you maintain an active and secondary set of Advanced Analytics (and additional Case Manager and Incident Responder) clusters in separate locations. In cases of a failure at the active site, you can fail over to the passive site.

At a high level, when Disaster Recovery is set up between two Advanced Analytics clusters, the active cluster is responsible for fetching the logs from SIEM or receiving the logs via Syslog. Once the logs have been parsed into events, the events are replicated from the active cluster to the passive cluster every five minutes.

Optionally, the raw logs can be replicated from the active to the passive cluster. (This allows reprocessing of logs, if needed. However, replication will generate great bandwidth demands between nodes.) If the active cluster goes down, then the passive cluster becomes the active until such time as the downed site is recovered.

screen -LS dr_setup
/opt/exabeam_installer/init/exabeam-multinode-deployment.sh

Please select the type of cluster:
1) This cluster is source cluster (usually the primary)
2) This cluster is destination cluster (usually the dr node)
3) This cluster is for file replication (configuration change needed)

screen -LS dr_setup
/opt/exabeam_installer/init/exabeam-multinode-deployment.sh

Please select the type of cluster:
1) This cluster is source cluster (usually the primary)
2) This cluster is destination cluster (usually the dr node)
3) This cluster is for file replication (configuration change needed)

What is the IP of the source cluster?

The source cluster's SSH key will replace the one for this cluster. How do you want to pull the source cluster SSH key?
1) password
2) SSH key

What is the path to the private key file?

This section includes instructions on how to failover to the passive site when the previously active site goes down. It also covers how to failback when you are ready to bring the restored site back online.

Exabeam's recommended policy for failback is to demote the failed cluster as the new passive cluster going forward. For example, Cluster A is the active cluster and Cluster B is the passive. Cluster A fails and Cluster B becomes active. When Cluster A is ready to come back online, it rejoins as a passive cluster until complete data synchronization and is then promoted to an active cluster again.

sos; docker ps

replicator-socks-stop; replicator-stop

screen -LS dr_failover
/opt/exabeam_installer/init/exabeam-multinode-deployment.sh

sos; everything-stop

docker-start

everything-start

File replication across clusters leverages Advanced Analytics and Incident Responder disaster recovery functionality, which replicates entire cluster configurations, context, user generated data, logs/events, and HDFS files (for Incident Responder).

In certain customer scenarios, clusters are situated in remote areas with considerable bandwidth restraints. In these rare scenarios, you can configure Advanced Analytics and/or Incident Responder to only replicate and fetch specific files. For example, you can configure your deployment to replicate only compressed event files across clusters.

screen -LS dr_setup
/opt/exabeam_installer/init/exabeam-multinode-deployment.sh

Please select the type of cluster:
1) This cluster is source cluster (usually the primary)
2) This cluster is destination cluster (usually the dr node)
3) This cluster is for file replication (configuration change needed)