- Introduction to How Content Works
- Understanding the Log
- Exabeam Parsers
- Exabeam Event Building
- Exabeam Enrichment
- Exabeam Persistence and Templates
- Exabeam Models
- Exabeam Rules
- A. Appendix
Understanding the Log
Exabeam can ingest logs directly from the source, fetch logs from SIEM log repositories, or ingest logs via Syslog as well as Exabeam Data Lake. Logs provide insight into the activity of users and entities (such as servers and workstations) and security issues across your enterprise. Context sources give our platforms additional information outside of the log data to make sense of the logs.
Once logs have been collected, they can be parsed in Data Lake and Advanced Analytics. This section will help you determine when to parse the logs and how to identify fields within the logs.
Do You Need a Parser?
In order to determine whether you need a parser, first determine whether your log qualifies for security processing and analysis. This is especially pertinent when parsing for Data Lake. For Advanced Analytics, all ingested logs must be parsed.
A parser extracts values from a log and maps those values to the appropriate Exabeam fields. See Exabeam Parsers for more information.
Parsing for Data Lake
Data Lake is designed to work with any log source and does not require a parser for much of its functionality. Data Lake can ingest, index, and search all logs even if they are not parsed.
Since parsing is a complex and resource intensive piece of the Data Lake pipeline, it may have some performance implications . Therefore, you should only use parsers where necessary . It is important to note that you can always go back in Data Lake and reparse old logs.
In Data Lake, you can do the following without a parser:
Send logs to Data Lake (ingest and index)
Set log retention
Perform string-based searches
Create rules on your data
Create certain reports and dashboards
You are limited in the types of rules, reports, and dashboards you can create without a parser since you do not get the benefit of field values.
If the data is parsed, these additional features are available:
Field specific reports and dashboards
Field specific visualizations
Field specific rules
Add context to logs, which make them searchable
Parsing for Advanced Analytics
Conversely, in Advanced Analytics, only parsed logs can be ingested. However, only logs related to an event type that Advanced Analytics can process and analyzes should be parsed. For a list of possible Advanced Analytics events and the content in which they are used, please refer to the Advanced Analytics Content Guide.
In logs that support Advanced Analytics event types, only the specific fields that are used for display or for processing need to beparsed. For example, for Entity Analytics logs like network connection, events do not need a user, and only IP, host, port, and similar information should be parsed.
If logs are required for Advanced Analytics (UBA) , they must have a user or a way to identify the user, such as a badge to a user list. Without a user, Advanced Analytics cannot process the log and it does not make sense to create a parser.
In Entity Analytics, events without a user can be processed, such as network connection status and netflow connection. Any event type can go on an asset timeline as long as it has a hostname. It is still necessary to make sure the log falls into one of the Advanced Analytics security related event types.
Minimum Log Data for Processing in Advanced Analytics
Once you have determined the log is security related, the next question to ask yourself is, does the log event have the minimum required information to be processed in Advanced Analytics.
For an event to be processed in Exabeam, it needs either:
A username (typically AD account), or information from which it can be derived, such as an email or distinguished name badge ID
Information about the originating or receiving asset (hostname or IP)
In addition to the minimum information, it also helps for the log to have a full timestamp and information about the host (hostname or IP) which produced the log. In Exabeam, these values will be parsed as "time" and "host", respectfully. The time information must include the year, month, day, hour, minutes, seconds, and preferably a timezone of when the log was generated.
Identify the Exabeam Event
Once an event matches the minimum criteria the next question would be which Exabeam event type best describes the event. An event type can be for example "remote-logon", "app-activity”, etc. For a list of Advanced Analytics event types, please refer to the Advanced Analytics Content Guide.
The log may not match an Exabeam event type exactly. In that case, you should use the Exabeam event that represents the log most closely.
Log Process Example
The following is an example of decision process for choosing logs for Advanced Analytics. This includes general questions to ask, and answers based on this example data:
"Aug 14 22:13:03 10.130.168.57 vendor=Forcepoint product=Security product_version=8.3.0 action=permitted severity=1 category=1913 firstname.lastname@example.org src_host=10.130.164.49 src_port=49265 dst_host=host.com dst_ip=188.8.131.52 dst_port=443 bytes_out=0 bytes_in=4805 http_response=0 http_method=CONNECT http_content_type=- http_user_agent=Mozilla/5.0_(Windows_NT_6.1;_WOW64)_AppleWebKit/537.36_(KHTML,_like_Gecko)_Chrome/59.0.3071.109_Safari/537.36 http_proxy_status_code=200 reason=%<reasonString> disposition=1028 policy=Exceptions_and_Filter_Updates**BasicBlocking role=1807 duration=4 url=https://exampledomain.com/download/virus.exe"
Determine whether the log has security significance. In this case, the log is from a web proxy product which shows access to websites, and therefore has security significance.
Answer the following questions to perform an initial analysis of the log:
Can the log be tied back to a user or device?
Yes, the log can be tied back to a user (
Does the log have a complete time field?
No, the log time is missing the year. We will use exabeam_time (
Aug 14 22:13:03).
What product or vendor produced these logs?
vendor=Forcepoint)It would also help to know the product, but it is not crucial in this case.
Perform the secondary analysis, asking whether the log can be mapped to an Exabeam Event Type.
The event type for this log is "web-activity-allowed". We expect "web-activity-allowed" logs to have "web_domain" and user information, which this log includes.
Based on these questions, we can conclude that Exabeam will provide value by ingesting this log.