Security ContentExabeam How Content Works Guide

Exabeam Event Building

The event builder stage is the part of the analytics engine pipeline that categorizes a parsed message into an Exabeam event type. Event types are the basic units that are used by the rest of the processing engine (enrichment, models, rules, and UI components).

Note

The Exabeam Advanced Analytics pipeline is as follows:

parsing > event building > enrichment > session building > modeling > rule triggering

Every parser is matched to an event builder definition. If there is no event builder for a parser, nothing is done with the parsed output of that log event. Parsed messages that do become events are written into evt.gz files.

The event building stage introduces several advantages:

  • Reduces the number of parsers – Different event types can be created based on a single parser. For example, a Windows login event will indicate the successful or failed outcome of a login in a field, which will have the value 0x0 in case of a successful login and another value in case of a failed login. This value can be parsed and conditioned on the event builder to create a local-logon or a failed-logon event. This eliminates the need for two parsers to capture these two event types.

  • Combines information in two logs – Some log sources provide all the information needed in an Exabeam event in two separate log events. For example, one VPN log could indicate the user's session and the source IP, and another log could provide the user's session ID, user name, and assigned IP. In order to create a meaningful vpn-login event, the information from both logs have to be combined into a single event. This can be achieved in the event builder based on the session ID field that would be identical in both logs.

  • Complex combination of multiple logs – Some email sources can generate hundreds of logs for a single email. In order to combine information in all these messages into a single event, a complex logic is needed which can be defined in the event builder.

Match Parsers to Event Builders

Different event builders can match the output of the same parser to create different event types. This is done to create different event types based on what is seen in the log.

There is also no issue with having a single event builder match the output of several parsers. This is done for example when there are several formats of an event that require several parsers (for example, a Windows event is collected with Snare and Beats from different systems). In fact, this will reduce the number of event builders and make them easier to manage.

However, unlike in the parsing stage where a parser cannot match with an event if the event has already been matched with another parser, multiple event builders can apply to a single message. This results in the creation of multiple events for a single log. Therefore, it is important to make sure that no more than one event builder will match a parsed message. This is usually done by creating a condition on the name of the desired parser as well as some of the data in the message, if necessary. There may be special cases in which more than one event should be created for a single log.

Event Builder Definition

Here is an example event builder definition containing a number of common fields, such as name, output-type, source, and vendor.

netskope-file-write = {
   input-message = [{
     expression = "InList(type, 'netskope-activity','s-netskope-activity','cef-netskope-file-operation-1') and InList(toLower(activity),'edit','move','create')"
   }]
   name = netskope-file-write
   output-type = file-write
   source = Netskope Active Platform
   vendor = Netskope Active Platform
}

Event Builder Field Descriptions

All of the fields below are required.

Field

Description

Event Builder ID

netskope-file-write

This can be any value but it must be identical to the name parameter.

Input-message

This contains the expression(s) that should be considered by this event builder. Similar parsers that should create the same type of event based on the same conditions seen across logs can be grouped into a single event builder as shown in the above Event Builder Definition example.

"InList(type, 'parser_name_1','parser_name_2')"

This expression matches a parser to an event builder. The type field contains the name of the parser that created the message.

Note

This type refers to the parser message type, which happens to be the parser name.

Everything parsed by the parsers defined in this expression only get evaluated by this event builder definition (unless the parser also exists in another event builder) and set of conditions.

"InList(toLower(activity),'edit','move','create')"

The rest of the conditions involve using logical expressions that check against parsed fields to decide whether a message becomes an event.

Name

This is the name of this specific event builder definition. It must be the same as the key given to the entire config/hocon block. If not, the analytics engine will not start. If another event builder contains the same name further down the file, the first one will be overwritten.

Output-Type

This is the Exabeam event type assigned to this event. It has to match one of Exabeam events and will determine how this event will be handled by downstream analytics.

Note

This type refers to the event type.

Source

The specific software/OS/product name that generates the log. This field will typically be visible in the UI.

Vendor

The name of the vendor of the system that generated the log.

Table 2. 


Event Stitching

Sometimes multiple logs are needed to build an event, meaning all the data needed for a single event are spread out over multiple logs. Event stitching allows the event builder to extract all the relevant pieces of information from different parsed logs and create a single event. Different parsers extract information from the relevant logs detailing different pieces of the same logical activity, for example a vpn login, to create a single event.

Two types of event builders can be used to combine information from multiple messages to a single event:

  • VariableMessageMultiEventTracker – Used to combine information from a variable number of messages into a single event.

  • ContivityMultiEventTracker – Used to combine information from exactly two messages into a single event. Here is an example Postfix email event builder:

    postfix-email-in = {
      input-message = [ ### for 'ContivityMultiEventTracker' ebuilders there should only be two objects in this field array, as 'ContivityMultiEventTracker' uses exactly two messages
       {
        expression = "type = 's-postfix-dlp-email'"    ### the expression that contains what parser to catch and other logical expression as the conditions.
        output-fields = "msg_id,src_ip,src_host,sender,recipients,recipient,host,subject"  ### fields to keep from the parsed message that will be apart of the event
        type = s-postfix-dlp-email   ### a name given to this specific message extraction
       },
       {
          expression = "type = 's-postfix-dlp-email-1'"
          output-fields = "dest_ip,dest_host"
          type = s-postfix-dlp-email-1
       }
      ]
      key-fields = "msg_id" ### the field that ties together the two messages and should be present in both parsed messages
      name = postfix-email-in ### the name of the event builder, same as the hocon block name
      output-type = dlp-email-alert-in ### event-type
      source = Postfix  ### product name
      tracker = ContivityMultiEventTracker #### special type of event builder, other value for tracker is 'VariableMessageMultiEventTracker'
      vendor = Postfix ### vendor name
    }

Types of Event Type Fields

When an event builder creates an event from a log, information from that log is mapped to three types of fields: required, extended, and informational.

An event builder always creates an event of a specific type. Each event type has unique fields that correlate to certain information in a log. There are three types of fields: required, extended, and informational.

For an event builder to create an event, a log must contain information that matches an event type's required fields. Information that maps to extended and informational fields is optional to create an event, but is still useful to process and display the event.

Required Event Type Fields

An event type's required fields ensure that an event has the minimum set of meaningful data for other components to process.

Components, like rules, machine learning algorithms, and Smart Timelines™, need a few basic data points to properly process an event. To ensure that an event contains these data points, Event Builder creates an event from a log only if that log contains the required data for a specific event type.

For example, the process-created event type has a process_name required field. To create an process-created event, a log must contain information about the process name.

Extended Event Type Fields

Information contained in extended event type fields help rules and models detect anomalies.

When Event Builder creates an event from a log, it matches certain information in the log to an event type's extended fields, if the information exist. Risk Engine uses the information contained in the extended fields to train models and evaluate the event against rules.

For example, the vpn-login event type has an os extended field. The VPN29 - VPN Operating Systems model trains on this os information. If the model considers the os anomalous, it may trigger the VPN32 - First VPN from OS rule.

Informational Event Type Fields

Informational event type fields enrich Advanced Analytics events and data in Data Lake with contextual information.

When an event builder creates an event from a log, it matches certain information in the log to an event type's informational fields, if the information exists. In both Advanced Analytics and Data Lake, these fields are used to correlate important data, like host and IP addresses, and enrich events with contextual information so you can easily search for logs, events, users, or assets.

For example, user_sid is an informational field for the event type kerberos-login. Data Lake maps user_sid to account_id so you can search for either user_sid or account_id and find the same log.

In Advanced Analytics, Smart Timeline™ events also display certain information based on these informational fields. If the event builder can't find the information for informational fields in the log, the informational fields appear blank in the Smart Timeline.