Security ContentExabeam How Content Works Guide

Exabeam Enrichment

Enrichment refers to the part of the analytics engine pipeline that adds 'contextual' information to an event that is otherwise not already parsed or not available in the log.

Note

The Advanced Analytics pipeline is as follows:

parsing > event building > enrichment > session building > modeling > rule triggering

Most enrichment adds new values, modifies them, or creates new fields based on existing fields or context table lookups.

Types of Enrichment

There are two types of enrichment, system- and user-defined.

System-Defined

This type of enrichment is done automatically by Advanced Analytics in the backend, and can be slightly tuned by custom_exabeam_config.conf.

  • Host-Ip Mapping – If a user or hostname is detected without the other, this enrichment feature populates the missing field based on previously seen data.

  • Security/Dlp-Alerts-to-User Mapping – When security or DLP alerts do not have the user information, this enrichment feature populates the user field based on previously seen data.

User-Defined

This type of enrichment can be granularly controlled by the user.

  • Context Enrichment – Performs a lookup from a context table to populate a field.

  • Event Enrichment – Modifies/adds/removes fields. This is the most common type of enrichment, defined the same way context enrichment is defined. All logical expressions available in the analytics engine, excluding model/session expressions, can be used in the Event Enricher.

  • Event Duplicator – Duplicates an event for the purpose of adding to a different user/asset timeline.

Enrichment Use Cases

It is possible to gather a lot of information about the log from values within the log itself that are not specifically parsed. You can then act on the contextual items seen in the logs, such as in modeling or rule triggering.

Track Specific Field Values

In the following example, we create a new field called 'win_command_count' to allow a rule the ability to "count" within a sequence/session specifically on certain values for process_name.

count-win-command {
  EventTypes = ['process-created','privileged-object-access']
  Condition = "exists(process_name) &&
 ((InList(toLower(process_name),'net.exe') && InList(toLower(arg),'start','user','time','view','use','localgroup','group','config','share')) || (InList(toLower(process_name),'netsh.exe') &&
InList(toLower(arg),'advfirewall')) ||
(InList(toLower(process_name),'tasklist.exe','ver.exe','ipconfig.exe','systeminfo.exe','netstat.exe','whoami','qprocess.exe','query.exe','type.exe','at.exe','reg.exe','wmic.exe','wusa.exe','sc.exe','rundll32.exe','psexesvc.exe', 'icacls.exe', 'arp.exe', 'route.exe')))"
  Map = [
    {
      Field = "win_command_count"
      Value = """'1'"""
    },
    {
      Field = "win_critical_command"
      Value = """process_name"""
    }
  ]
}

The analytics engine "count" expressions allow us to count against a certain field, such as "process_name," but the expression does not let us specify for what values of that field we should count on. The below enricher allows us to create a field that only ever exists when it is or contains the values we want to count on. Now we can count how many times the following command was used.

Sum(win_command_count, 'process-created')

Note

When you need to know the number of times a set of conditions are satisfied in a session, you can create a field and assign a value 1 to it to indicate the condition was satisfied for the event. This gives you the capability to implement the abnormal number based use cases.

A rule can use the field to know the unique number of processes that satisfied the condition.

DistinctCount(win_critical_command, 'process-created')

Context Retrieval

Many of the event enrichers perform context enrichment. Enrichers that do this use a parsed field value as a key to a context table to extract the value into a new field.

user-email {...
      Map = [
        {
          Field = "user"
          Value = """GetValue('email_user',toLower(user_email))"""
        }...

In the above example the 'user' field is created from the user_email context table. When only the 'user_email' is parsed from the log, you can fetch from this context table the AD 'user' value mapped to the 'user_email' which will stitch the event to the user timeline.

New Field Based on Information Within the Event

Most of the time in enrichment you can take fields from the message to create new fields based on some conditions using logical expressions.

Example 1:

local-user {
      EventTypes =
['batch-logon','file-delete','file-read','file-write','privileged-access','privileged-object-access','process-created','service-logon','workstation-locked','workstation-unlocked','local-logon','remote-access','remote-logon','account-password-change','account-password-reset','account-lockout','account-unlocked','account-enabled','account-disabled','account-deleted','account-creation','member-added','member-removed']
      Condition = "exists(domain) && ((exists(dest_host) && dest_host = domain) || InList(toLower(domain),'workgroup', 'window manager', 'font driver host')) && vendor='Microsoft Windows' && !InList(event_code,'4648','4769','673','676','552') and not EndsWith(user, '$') and !InList(toLower(user),'system','local service','network service','anonymous logon')"
      Map = [
        {
          Field = "user_type"
          Value = """'local'"""
        },
        {
          Field = "user"
          Value = "concat(user, ' (', dest_host, ')')"
        }
      ]
    }

In the above example a user_type field is created, which holds an attribute about the user, such as whether the user is a local user.

Example 2:

security-alert-local_asset {
      EventTypes = 
['security-alert','dlp-alert','process-alert','network-alert','database-alert']
      Condition = "exists(src_host) || exists(src_ip) || exists(dest_host) || exists(dest_ip)"
      Map = [
        {
          Field = "local_asset"
          Value =
"""if(isSiteLocal(src_ip),first(src_host,src_ip),if(isSiteLocal(dest_ip),first(dest_host,dest_ip),first(src_host,src_ip,dest_host,dest_ip)))"""
        }
      ]
    }

In the above example, we look at alert based event types and determine the field to be made the local asset based on priority and isSiteLocal() function.

Field Modification

In the following example, we modify an existing field to create a field that can be used for detection by existing Advanced Analytics content:

bytes-domain {
      EventTypes = 
['dlp-email-alert-out','dlp-email-alert-out-failed','dlp-alert','usb-insert','usb-write','usb-read','dlp-email-alert-in','share-access','print-activity','file-write','file-delete']
      Condition = "exists(bytes_unit) && !exists(bytes)"
      Map = [
        {
          Field = "bytes_num" 
          Value = """replaceAll(bytes_num, ",","")"""
        },
        {
          Field = "bytes"
          Value = """Multiply(bytes_num,ReturnIf(ToLower(bytes_unit)='kb',1024,ReturnIf(ToLower(bytes_unit)='mb',1048576,ReturnIf(ToLower(bytes_unit)='gb',1073741824,0))))"""
        }
      ]
    }

Advanced Analytics content related to data transfer sizes operate using bytes (not kilobytes, megabytes, or gigabytes). So, when we parse a value from a log that is not in bytes representation, we modify the parsed value accordingly, multiplying the parsed bytes value (bytes_num) by 1024 when we see the bytes_unit value is kilobytes, multiplying it by 1024*1024=1048576 when the bytes_unit value is megabytes, and so on.

Required Model/Rule Field

When you want to accurately track how often a specific activity has occurred, you can track the count pairs of field values. You can also use this information to determine if different field values are better stored in a single field value.

Many enrichers use the 'concat' logical expression to put two field values together.

unix-target-id {
      EventTypes = 
['account-deleted','account-password-change','account-password-reset']
      Condition = """!exists(target_user) && exists(target_user_id) && exists(dest_host) && vendor='Unix'"""
      Map = [
        {
          Field = "target_user"
          Value = "ReturnIf(target_user_id = '0', concat('root (', dest_host, ')'), concat(target_user_id, ' (', dest_host, ')'))"
        }
      ]
    }

In the above example, we create a field called target_user that either begins with 'root' if target_user_id = '0' or begins with the actual target_user_id field if the value is not equal to 0.

netflow-scanhost {
      EventTypes = ['netflow-connection']
      Condition = "exists(src_host)"
      Map = [
        {
          Field = "src_host_time"
          Value = """concat(src_host, '-', take(time,9))"""
        }
      ]
    }

In the above example we concatenate src_host and the time field. We can concentrate with time to track whether multiple events relating to a certain activity happened within a single second.

A rule can now use:

"""DistinctCountByIf(dest_host, src_host_time, src_locality = 'internal', 'netflow-connection') = 20"""

to track whether a host (src_host) reached out to another host (dest_host) 20 times within a second. This works because the time value is concatenated to the src_host value, and because src_host will not change, we can rely on that if the entire field does not change, then the time is the same for the different events.

Event Enricher Configurations

Here is the syntax for an event enricher:

EventTypes = ["event_type"]         ### what event types should be evaluated against this enricher. Using [] means the enricher will apply to all event types.

Condition = "exists(certain_field) OR endsWith(some_field, '.bat')" .              ### conditions based on Exabeam's logical expressions

Map = [ { field = "new_field", value = "anything I want" }, { field = "new_field_2", value = """'100'""" } ]         ### new field definitions
  • First, restrict what events are enriched, and then define the fields and values to be created.

  • Restriction is done using the 'EventType' and 'Condition' field.

  • Make sure you only look at events that are of the type(s) specified in the 'EventTypes' field, expressed as an array of event_types.

  • Further restrict what gets enriched by adding the analytics engine expressions in the conditions parameter. In the above example, we only enrich the event if the field 'some_field' ends with '.bat' or we enrich the event if the field 'certain_field' already exists.

Additional Enrichment Guidelines

Within an enricher, you can enrich multiple fields.

You can even enrich on a field created higher up in the enricher.