How to Run Query Searches in Exabeam Data Lake

Data Lake can be customized to search for variations and combinations in the captured data to suit needs and circumstances. The Search UI offers an input box for customers to apply their own criteria.

Complex or heavily used queries can be saved to the local library for re-use.

Note

Here are additional methods to consider when handling large data volumes:

Filtered Searches -- Narrow the amount of data to search, you can apply filters using context tables to optimize your queries.Filtered Searches in Data Lake
Cross-cluster Searches -- In a multi-cluster deployment, you can perform searches simultaneously across all log ingesting clusters.

Syntax

The following table shows the accepted syntax for querying in Data Lake . Data Lake query semantics applies a limited subset of Lucene.

Note

Note that AND, TO, NOT, and OR are case-sensitive operators (i.e. all upper-case only).

Types	Description	Example
Terms	Alpha-numeric text to search for	`”error”` Look for records with string `error`.
Fields	Data type or category name (i.e. key within [key,value] of structured data)	Search any field by field name followed by a colon "`:`" and string to search for. `status:”error”` Look for records with string `error` in category status.
Operators	Joining of two or more criteria
`AND` or `+`	Both terms must exist	`user:"joe" AND host:"201.45.34.24"` Look for records with both `joe` and `201.45.34.24` in their respective fields.
`OR`	Either term may exist	`user:"joe" OR country:"jane"` Look for records with either in category user`jane` or `joe`.
`NOT` or `-`	Term must not exist	`user:"joe" NOT country:"US"` Look for records with `joe` but without `US` in their respective fields. Note The `NOT` operator cannot be used with just one term but must have a core search to apply the `NOT` condition against. (i.e. The above example could not run as just `NOT country:"US"`.)
`TO` `>` `<` `>=` `<=`	Range of values with lower and/or upper limits, expressed as numeric values `field_name: low TO high` `field_name: >low` `field_name: <high` `field_name: >=low` `field_name: <=high`	`num_hit: [10 TO 50]` `num_hits: >50` `logon_date: [2018-10-31 TO 2018-12-31]` `date: [* TO 2012-01-01]` `indexTime: [* TO 2018-10-05T23:48:00.000]` `indexTime: [* TO 2018-10-05T23:43]`
Grouping `( )`	Multi-term search processed first by criterium set in parentheses	`error* (joe OR jane)` Look for records with leading string `error` in that contain either `joe` or `jane`.
Wildcards `?` `*`	[empty] Single character variation search (Cannot be used as leading character.) [empty] Multi-character variation search (Cannot be used as leading character.)	`user:jo?` Look for records with string `jo` with a single trailing character (e.g. `jo2`). `user:jo*` Look for records with string `jo` with any trailing characters.
Special Characters `\`	Characters `+ – & \|\| ! ( ) { } [ ] ^ ” ~ * ? : _` that used in query operations can be converted to be used search text by adding ‘`\`’ before the character	`user:”jo\+”` Look for records with string `jo+`. Alternative method: `user.keyword:”jo-anne”` Look for record with hyphenated string `jo-anne`.
Regular Expressions	Regular expression patterns can be embedded in the queries by wrapping them in forward-slashes ("`/`")	`field_name:/[regular-expression]/`
Tokenized Fields	System field names invoke parsing when standardized delimiters are encountered, such as	Sample search for "[email protected]" `user.keyword: string` `user:"string"`, where `string` contains `@`, `.` , or `-`. Yields results because `user.keyword` is non-tokenized `user: string` `user.keyword: "string"`, where `string` contains `@`, `.` , or `-`. No results because `user` is tokenized for full-text search, where, for example, `[email protected]` is parsed as `user`, `engineering`, `domain`, and `com`
`_exists_` and `!_exists_`	Determine whether fields that exist (have a value) or not	`_exists_:user` Yields logs where user field is populated `!_exists_:<exa_parser_name>` Yields logs where `<exa_parser_name>` field is empty
`<field>.keyword:"-"`	Search string qualifier when a keyword type field cannot be parsed. Do not use `<field>:”-”` even though this field is a text type as well. Otherwise, there will be no results returned.	`host.keyword:”-”` This search will return data with a non-parsable host field.

Note

The default operator in searches is OR unless you explicitly form your query to not apply it.

Time Parameters

Correctly searching and synchronizing time between log messages is critical to forming a timeline of events you are analyzing. There are multiple ways time information is stored in log messages. It is important to distinguish between them and use them accordingly.

Parameter	Description
@timestamp	This is a search value. It is the default time field that reflects the time when log message was received at the Data Lake ingestion layer.
indexTime	This is a search value. It is the time the Data Lake parser/enricher processed the log message for indexing.
exa_adjustedEventTime	This is a message log field. It is the time value derived from event itself with adjustments such as time zone, if present in log message and parsed out.
exa_rawEventTime	This is a message log field. It is the non-adjusted time value derived from log message itself. If log message does not have a time field, it defaults to ~indexTime.

Field Explorer

In addition to using manually created search strings, users have the option to filter data using out-of-the-box filters available in the Search UI.

The Field Explorer is the quick pick tool for viewing captured data in known categories (both out-of-the-box and custom filters). Click on the hyperlink for a given sub-category and menu of known values are listed to filter further. View field visualization can be selected to immediately visually organize data from the shown list.

Searches using Exabeam exa_category

Out-of-the-box filters are available in the Search UI. Once data, using preliminary parameters (e.g. time range) is gathered, a categorized Field Explorer appears below the Timeline. Information is separated by areas of focus such as: Account Management, Failed Logon and Lockout, Windows Authentication, and Default. Select links under each area to further filter data by sub-selection or field query. Events counts are listed in each linked category. Each activated filter is reflected in query syntax in the Search input field.

These categories are part of the "exa_category" set and there exist subcategories to narrow searches with. The queries are in the form:

exa_category:"<category>" AND <field>:"<value>"

Category (for exa_category)	Description	Field
Account Management	Events relating to creation, deletion, and modification of entity's computer accounts	account_name dest_host domain event_code host target_user user
Account Switch	Events indicating that user A is operating as user B (e.g. runas, sudo)	account dest_host event_code host user
Active Directory	Events related to Microsoft Active Directory	user object activity_type attribute object_class event_name event_code dest_host domain host
Application	Events relating to applications (e.g. pull/sync from a code repository)	activity app host src_ip user
Audit Change	Changes to the audit policy of a computer	event_code event_name host policy subcategory user
Authentication	Events related to connection credentials	user event_code auth_method failure_reason src_ip dest_ip dest_host domain host
Badge	Physical access log events	badge_id location_building location_door outcome user
Configuration Change	Events indicating the setting of a system has changed	event_code event_name host log_type src_type user
DHCP	Events from DHCP service	user dest_ip dest_host host
DLP	Events from a data leak protection system	alert_name external_domain host protocol src_ip user
DNS	Events from a DNS system	dest_ip dest_port query_id query_type src_ip src_port
Database	Change events for database endpoints	database_name db_operation dest_host dest_ip src_host src_ip user
Endpoint	Actions of interest at endpoints	command_line dest_host host process_name user
Failed Logons and Lockouts	Login failure events	dest_host dest_ip domain event_code host user
File	File access events	accesses dest_host file_name host user
Logout	Logout events	user event_code logon_type dest_host host domain
Network	Network traffic events	bytes_in bytes_out dest_ip dest_port host protocol rule src_ip src_port
Network Alert	Network access events	dest_ip dest_port host protocol src_ip src_port
Print Activity	Printing/Printer action events	event_code host outcome printer_name user
Privileged Access	Action events connected to highly restricted assets	dest_host event_code host privileges process_name user
Security Alerts	Actions for known malicious payloads	alert_name alert_type host malware_url src_host src_ip user
System Event	System-level events	event_name log_source host dest_host
VPN	VPN login events	failure_reason host src_ip src_translated_ip user
Web	Web-based access events of interest	user protocol action category web domain bytes out bytes in src_ip dest_ip method result code host
Windows Authentication	Microsoft Windows login-based events	dest_host dest_ip event_code host logon_type src_ip user

Searches using Exabeam Fields

Exabeam parses and categorizes different values for fast searching, using the query format:

<field>:"<value>"

Field	Description	Value
exa_activity_type	Actions that are considered behaviors of concern in general practice	authentication account-management account-management/user object-access alert account-management/user/enable authentication/remote-logon audit-log-change authentication/remote-access password-management object-access/read cve-notice netflow object-access/write account-management/user/create account-management/user/disable web-access password-management/change network-traffic process-creation audit-log-change/delete authentication/logout account-management/user/delete alert/dlp authentication/service-logon print password-management/reset config-change alert/file object-access/delete authentication/batch-logon authentication/local-logon email email/inbound
exa_addRiskToAsset	Incremental risk score changes marking milestone triggers	true false
exa_adjustedEventTime	Time offsets for event of interest. It is the time value derived from event itself with adjustments such as time zone, if present in log message and parsed out	milliseconds
exa_category	Exabeam core categories of interest in threat detection	See "Searches using Exabeam exa_category" section
exa_device_type	Device category	operating-system operating-system/file-system
exa_outcome	Milestone marker for event result triggers	success failed
exa_parser_name	Filter by parser name	parser name
exa_rawEventTime	Event time window of interest (UTC). It is the non-adjusted time value derived from log message itself. If the message does not have a time field, it defaults to ~indexTime.	@timestamp
exa_rule_category*	Filter by defined rule category	category name (See "Searches using Exabeam exa_category" section)
exa_rule_config_cardinality_field*		@timestamp
exa_rule_config_is_enabled*	Events when rule is enforced or disabled	true false
exa_rule_config_max_cardinality*		max value
exa_rule_config_num_events*	Threshold count for events of interest	count value
exa_rule_config_query_key*		user.keyword
exa_rule_config_realert*	Threshold count for recurring events	minutes:[integer]
exa_rule_config_terms_size*		minutes:[integer]
exa_rule_config_timeframe*	Time range for events of interest	minutes:[integer]
exa_rule_config_top_count_key*		@timestamp,user.keyword
exa_rule_description*		cardinality description
exa_rule_id*	Filter for events that trigger a specified rule, specified by rule ID	ID value
exa_rule_name*	Filter for events that trigger a specified rule, specified by rule name	rule name
exa_rule_search_query*		See query rules in "Syntax" section
exa_rule_severity*	Threshold trigger based on severity level	HighAlertSeverity MediumAlertSeverity LowAlertSeverity
exa_rule_type*		CardinalityRuleType FrequencyRuleType Aggregation
exa_security_alerts		alert_type alert_name alert_severity alert_id src_ip dest_ip src_host dest_host host user malware_url additional_info -- A field for providing event-specific information that cannot be mapped directly to any field, applying primarily to alert events.

* "exa_rule_" fields are parsed out of correlation rules triggered by Data Lake .

Data LakeExabeam Search Quick Reference Guide

Table of ContentsTable of Contents