Data LakeData Lake User Guide

Table of Contents

Exabeam Data Lake Search

One of the primary activities of an SOC is searching the log repository for specific events. For example, searching for the activities of a specific user in a given timeframe. Searching is the beginning of any investigation. It is where you access all your logs and filter through them, looking for events that match your criteria. Regardless of if you are building a visualization or a chart, you begin from a search.

You can interactively explore your data from the Search page. You have access to every event that matches the search query within the selected date and time range. You can submit search queries, filter the search results, and view event data. You can also see the number of events that match the search query and get field value statistics. The distribution of events over time is displayed in a histogram at the top of the page.

Exabeam Data Lake Search Page Overview

This section will isolate and identify the different sections of the Search page and give a brief overview of their functions.

DL-SearchUI-Timeline.jpg

Search Bar in Exabeam Data Lake

At the top of the page is a search bar where you can enter a simple text search or use the Lucene query syntax to search your data. The total number of matching events are shown above the toolbar. To the right is a drop-down time filter that you can use to filter logs based on various relative and absolute time ranges.

search bar

Next to the time filter is a SAVE button that allows you to save searches - this function saves both the query string and the currently selected index pattern.

Clicking on LIBRARY opens a drop-down box that contains all of your saved searches, visualizations, and dashboards. Clicking on a Saved Search will populate the Search Bar with the saved query and launch the search.

DL-SearchUI-SaveLibrary.jpg

For searches that have run a long period without timing out (for example, a search on a small data set does not complete in 5 minutes) or you have initiated in error, you can halt the query by clicking the cancel icon that appears when the search is started:

DL-Search-CrossClusterUI_-Stop.jpg

If your query string extends beyond the length of the search bar, the text will wrap to the next row. To view the whole expanded query, click ... located in the lower right corner of the search bar. Click elsewhere outside the search bar to collapse the expanded view.

multiline search bar UI

To initiate a query with hot-keys, use Control-Enter.

Note

In a cluster deployment, shards that are not available (such as replicate shards) do not block searches on the remainder of the cluster. Partial results will be returned.

Events Table from Exabeam Data Lake Search Results

When you submit a search request the Timeline, Events Table, and Fields list are updated to reflect the search results. The most recent events that match the query are listed in the Events Table. You can use this to look at individual log messages and display log data filter by fields. If no fields are selected, then the entire log messages are displayed. By default, the table shows the localized version of the time field configured for the selected index pattern and the event source. You can add fields to the Events Table from the Fields list. You can sort the listed events by any indexed alpha column field that’s included in the table.

data lake search enhanced view

At the top of each event is a gray highlighted section. These are the fields within the event that Exabeam considers likely to be of interest to the analyst.

To view the event data as a separate page, click the event record. You can bookmark and share this link to provide direct access to a particular event.

The Table view allows you to create your own table with fields of your choosing, to be arranged as you see fit.

data lake table view

While viewing in Table view, sort the results according to a single column or multiple columns by clicking SORT COLUMNS or by clicking the arrow next to any column name.

DL-SearchUI-TableView-Sort.jpg

The SORT COLUMNS menu lets you select the columns by which you want to sort data in the table. You can configure each column to sort by ascending or descending order. Additionally, you can sort search result tables on the Search page and data tables on the Visualizations by multiple columns.

DL-SearchUI-TableView-SortColumns.jpg

In the Raw view, each event shows the first four lines of the event log. Generally, the first four lines include the timestamp and the actual payload of the event. You can see the full event payload by clicking Show more and then contract it back to a truncated view by clicking Show less.

data lake raw log view

The Share button allows you to create a shortened URL which links directly the current search results table. You can COPY and share the snapshot link with members of your organization who have access to the Data Lake UI.

DL-SearchUI-Share.jpg

The Export button allows you to save your search results as a PDF or a CSV file. For CSV exports, set the number of search results to export (up to 1 million). Large CSV exports are split into multiple CSV files and then exported to a ZIP file.

DL-SearchUI-Export.jpg

For each log event, you can select the More Options icon to open the submenu for export options:

  • Copy Raw Log -- Cache the raw log text to your local buffer.

  • Copy Link -- Cache a shareable link to you local buffer.

  • Open in New Tab -- Present the parsed fields and raw log in a new tab in your web browser.

data lake search raw log export

Timeline View from Exabeam Data Lake Search Results

The Timeline View is a date histogram bar graph that shows the count of logs over time, matched by the search and time filters. You can click on the bars to narrow the time filter.

DL-SearchUI-Timeline - Closeup.jpg

To set a Time Filter from the histogram, do one of the following:

  • Click the bar that represents the time interval you want to zoom in on.

  • Click and drag to view a specific timespan. You must start the selection with the cursor over the background of the chart—the cursor changes to a plus sign when you hover over a valid start point.

The histogram lists the time range you’re currently exploring, as well as the interval range that is being used. To change the intervals, click the link and select an interval from the drop-down. The default behavior automatically sets an interval based on the time range.

To set a Time Filter from the histogram, do one of the following:

  • Click the bar that represents the time interval you want to zoom in on.

  • Click and drag to view a specific timespan. You must start the selection with the cursor over the background of the chart—the cursor changes to a plus sign when you hover over a valid start point.

The histogram lists the time range you’re currently exploring, as well as the interval range that is being used. To change the intervals, click the link and select an interval from the drop-down. The default behavior automatically sets an interval based on the time range.

DL-SearchUI-TimeRange-Quick.jpg

You can narrow your search further using the Time Picker tool. See Selecting a Timeframe.

You can collapse or display the Timeline by clicking the Open/Close icon.

data lake close chart
Selecting a Timeframe in Exabeam Data Lake Search Results

The time filter restricts the search results to a specific time period. You can set a time filter if your index contains time-based events and a time-field is configured for the selected index pattern. By default, the time filter is set to the last 15 minutes. You can use the Time Picker to change the time filter or select a specific time interval or time range in the timeline view at the top of the page.

For all time-based data, you can select the time span that you want to analyze in the current view at the top right of the window. There are multiple ways to get to the events you are interested in: either use the Quick tab to select a date range like Today or Last 1 hour or use the Relative and Absolute tabs to specify more specific time spans you want to look at.

After you select a time range, you will see a timeline view at the top of the screen, which will show the distribution of events over time.

To use the Time Picker, either drag your mouse over a specific span of time.

Timeline-SelectDrag.jpg

Or, you can select a bar on the timeline that represents the time interval you want to zoom in on.

Timeline-Select.jpg

Filtered Searches in Data Lake

In addition to using time constraints to narrow the amount of data to search, you can apply filters using context tables to optimize your queries. Filtered Searches can also be applied to dashboards, visualizations, and reports.

Note

Please consider the following:

  • PDF export of filtered searches is currently not supported

  • Filtered searches work only with key-only context tables of no greater than 10k records

  • One context table at a time can be applied per filtered search

  • Context table records must match the format of the field being queried

  • Do not use string values with numeric characters

  • Exact value matches will be applied

  1. To narrow the data that you will run the query against, click + Context table below the Search field to expand the menu to add a new filter to searches.

    filtered search add context table
  2. Select the Field, is or is not condition, and In context table that you want to apply. You can select from the drop down list or start typing in the fields to get possible matches displayed.

    filtered search context table menu
  3. Click ADD to apply the filter. The filter will appear below the Search field.

    filtered search filter
  4. Click SAVE to store the query to the library.

    data lake save query

You can click the filter to edit it. If the parameter has already been applied, a check mark will appear next to the record. Click UPDATE to apply the filter.

filtered search previously selected

For data formats supported in filtered searches, please see the Exabeam Search Quick Reference Guide.Exabeam Search Quick Reference Guide

Performing Searches in Exabeam Data Lake

Data Lake is built on top of Elasticsearch, which uses the Lucene query language. For more detailed information on syntax and search options, see Data Lake Search Quick Reference GuideExabeam Search Quick Reference Guide.

Note

In a cluster deployment, shards that are not available (such as when a node goes down) do not block searches on the remainder of the cluster. Partial results will be returned.

Types of Exabeam Data Lake Queries

Data Lake accepts searches in the Lucene query language. A query is broken up into terms and operators. There are two types of terms: Single Terms and Phrases. This section covers some of the basic operators for conducting searches.

  • A Single Term is a single word such as "test" or "hello".

  • A Phrase is a group of words surrounded by double quotes such as "hello world".

  • Multiple terms can be combined with Boolean operators to form a more complex query.

Text Searches

By default, the search box performs unstructured text searches. It searches for entries containing any of your search terms and a hyphen is considered a delimiter. This means that if no specific field is indicated in the search, the search will be done on all of the fields that are being analyzed. It will not tell you if your search has the wrong syntax.

Note

Text searches are not case sensitive. This means that 'category" and "CaTeGory" return the same results. Use double quotes (“”) to search for an exact match

Field Level Searches

The query language allows you to search inside any field, simply enter the name of the field and then a colon.

Some examples:

To just search inside a field named “lang”

lang:en

To search for the language English or Spanish in the "lang" field

lang:(en OR es)

Like the selected fields, the entered query will be persisted if you save your search.

You can search a range within a field. If you use brackets [], this means that the results are inclusive. If you use {}, this means that the results are exclusive.

Using the _exists_ prefix for a field will search the events to see if the field exists.

You cannot use wildcards inside of phrases.

For information on queries, see Data Lake Search Quick Reference Guide.

Logical Statements

Logical statements enable you to use more than one condition in a query. You can use parentheses to define complex logical statements and be sure that you use the proper format such as capital letters to define logical terms like AND or OR.

In some cases, you might want to compare the results of two separate queries. Data Lake can handle multiple queries by joining them with a logical OR.

To search for the logon event 4768 and the user Barbara Salazar:

event_code:4768 AND user:bsalazar

Search Type

Operator(s)

Example

Full Search

*

*

Literal String

""

"geo-address"

Single Field

<Field name>:

country:

Missing Field

missing:

missing:vpn

Present Field

_exists_:

_exists_:vpn

Wildcard

* for any number of characters

? for one character (Cannot be used in _type fields.)

*

Negative Terms

!;-;NOT

-VPN

!VPN

NOT VPN

Range Search

[number TO number]

user.listed_count:[0 TO 10]

For more detailed information about running searches, see Exabeam Search Quick Reference GuideExabeam Search Quick Reference Guide

Exabeam Data Lake Event Categorization

Data Lake supports multiple categorization attributes for each log or event type defined in the product. Different vendors use different fields and terms in their logs.

Categorizing events provides a consistent taxonomy for queries, reports, visualization, dashboard, search, and correlation rules. Our out-of-the-box compliance reports leverage this nomenclature.

For example, a log has the following value:

exa_activity_type: authentication/local_logon

This log will also be returned in the query:

exa_activity_type=authentication

Current categories are:

  • exa_category

  • exa_device_type

  • exa_activity_type

  • exa_outcome

Examples:

exa_activity_type = account-management/user/create

exa_device_type = operating-system/network/firewall

exa_outcome = success/allow

For a complete list of Exabeam event categories, see How to Run Query Searches in Exabeam Data Lake > "Searches using Exabeam exa_category".How to Run Query Searches in Exabeam Data Lake

Saved Searches in Exabeam Data Lake

Saving searches enables you to reload query results quickly and use them as the basis for Visualizations, Dashboards, and Reports. A search can be saved into a library by clicking on the SAVE. You can access the Search Library at any time to get a list of all your saved searches. Selecting a saved search will populate the search box with the query and launch the search.

To Save Current Search:

  • Click SAVE in the toolbar.

data lake save query
  • Enter a name for the search and click SAVE TO LIBRARY.

DLSearchUI-SavToLibrary.jpg

To Open a Saved Search:

  • Open the Search Library.

DL-SearchUI-Library.jpg
  • Select the search you want to open.

DLSearchUI-LibraryUI.jpg

To Edit or Delete a Saved Search:

The Search Library contains a list of all Saved Searches. To edit any Saved Search click on the vertical ellipsis to the right of the date field. This opens a drop-down menu and from here you can edit the search or delete it.

DLSearchUI-LibraryUI-EditDelete.jpg

How to Run an Exabeam Data Lake Cross-Cluster Search

In a multi-cluster deployment, you can perform searches simultaneously across all trusted clusters. Ensure you have permission to run a cross-cluster search and that the clusters of interest are available. You must have at least one remote cluster configured. For more information on setting up a searchable remote cluster, seeData Lake Administrator Guide > Configuring Data Lake > Cross-Cluster SearchCross-cluster Search in Exabeam Data Lake.

If you have permission to conduct cross-cluster searches, there will be a Local Cluster menu above the search field.

DL-Search-CrossClusterUI.jpg

Select the clusters you want to apply the search to. Compose the query and its parameters as you would for typical searches, following prescribed syntaxes (see Type of Queries). To configure a remote cluster, see the Data Lake Administration Guide > Configuring Data Lake >Cross Cluster Searches.Cross-cluster Search in Exabeam Data Lake

Warning

Cross cluster search results export is limited to 10,000 events per search query. For local clusters, up to 1,000,000 events per search query can be exported in CSV format.