Log Stream Features Introduced in 2024

October 2024

The following Log Stream features were introduced in October 2024:

Feature	Description
Support for Extracting Value from JSON Arrays	Log Stream now provides a manual option for entering JSON path expressions when extracting and mapping field values. For a simple field-level value, the JSON path expression must be in the following format: `$.field_name`. While this option can be used to map any field value, when you need to extract value from an array, the manual option is the only available method. Multiple types of path expressions are supported to facilitate extracting values from an array. These path expressions extract different levels of value from an array, ranging from returning the entire array object to returning specific nested elements. For more information, see Extract Mapped JSON Fields in the Log Stream Guide.
Improved Logic for Extracting Specific Values from JSON Fields	Log Stream now supports the use of conditional logic to extract more precise values from JSON fields when ingesting logs and mapping values to CIM fields. Two new columns have been added to the extraction table to facilitate this logic: Extraction Condition – If you want to apply conditions to the way a field value from the raw log is extracted for mapping to a CIM field, insert a logical condition in the Extraction Condition column that will determine in what scenarios the mapping should take place. For example, on row 1 in the image below, the value from the `eventCategory` field in the raw log will only be mapped to the `alert_name` CIM field if the raw log value contains the word "Data". Substring Regex – If you want to extract a portion of a string value from the raw log, insert a regular expression in the Substring Regex column that will extract just the specified portion of the string value and map it to a CIM field. For example, on row two in the image below, the condition in the Substring Regex column extracts a specific part of the `resources[1].type` field from the raw log. Specifically, it extracts only the `S3` part of the value and maps it to the `alert_type` CIM field. For more information, see Extract Mapped JSON Fields in the Log Stream Guide.

Feature

Description

Support for Extracting Value from JSON Arrays

Log Stream now provides a manual option for entering JSON path expressions when extracting and mapping field values. For a simple field-level value, the JSON path expression must be in the following format: $.field_name. While this option can be used to map any field value, when you need to extract value from an array, the manual option is the only available method.

Multiple types of path expressions are supported to facilitate extracting values from an array. These path expressions extract different levels of value from an array, ranging from returning the entire array object to returning specific nested elements.

For more information, see Extract Mapped JSON Fields in the Log Stream Guide.

Improved Logic for Extracting Specific Values from JSON Fields

Log Stream now supports the use of conditional logic to extract more precise values from JSON fields when ingesting logs and mapping values to CIM fields. Two new columns have been added to the extraction table to facilitate this logic:

Extraction Condition – If you want to apply conditions to the way a field value from the raw log is extracted for mapping to a CIM field, insert a logical condition in the Extraction Condition column that will determine in what scenarios the mapping should take place.
For example, on row 1 in the image below, the value from the eventCategory field in the raw log will only be mapped to the alert_name CIM field if the raw log value contains the word "Data".
Substring Regex – If you want to extract a portion of a string value from the raw log, insert a regular expression in the Substring Regex column that will extract just the specified portion of the string value and map it to a CIM field.
For example, on row two in the image below, the condition in the Substring Regex column extracts a specific part of the resources[1].type field from the raw log. Specifically, it extracts only the S3 part of the value and maps it to the alert_type CIM field.

For more information, see Extract Mapped JSON Fields in the Log Stream Guide.

July 2024

The following Log Stream features were introduced in July 2024:

Feature	Description
Support for Ingesting Certain Hybrid Logs with a JSON Parser	Certain types of hybrid JSON logs are now recognized as JSON-formatted logs for the purpose of building and managing parsers. The supported hybrid log format is a JSON message that is prefixed with text metadata from the Cloud Collector source, as in the following example: `destinationServiceName=Azure <JSON message>` This improvement provides a consistent experience across different types of parser during field extraction when creating parsers. Whether you are working with a regular expression parser, a native-JSON parser, or even a JSON parser with a non-JSON Cloud Collector prefix, extracted fields are clearly highlighted in various colors for easy viewing. For more information, see Extract Event Fields in the Log Stream Guide.
Support for Regex in Native JSON Parsers	If you need to extract values from a native JSON log that contains a non-JSON prefix, you can now add a Regex extraction when building your JSON parser. During the field mapping process, a new option is available that will allow you to include a Regex extraction. This option can be used to define values that cannot be extracted using JSON field mapping. For more information, see Extract Event Fields in the Log Stream Guide.
Multi-Log Event Building	Log Stream handling of multi-log event building has been improved to generate a single event from multiple logs across multiple event processing nodes. Leveraging predefined definitions from Exabeam, parsers that use multi-log event building can extract or combine information from key fields across different parsed logs coming from different processing nodes. Based on the parser definitions, Exabeam tracks the logs and stitches them together once conditions are met. This capability enables Exabeam to deliver richer events that provide a more complete picture of your organization's security posture. For more information, see Multi-Log Event Building in the Log Stream Guide.

Feature

Description

Support for Ingesting Certain Hybrid Logs with a JSON Parser

Certain types of hybrid JSON logs are now recognized as JSON-formatted logs for the purpose of building and managing parsers. The supported hybrid log format is a JSON message that is prefixed with text metadata from the Cloud Collector source, as in the following example:

destinationServiceName=Azure <JSON message>

This improvement provides a consistent experience across different types of parser during field extraction when creating parsers. Whether you are working with a regular expression parser, a native-JSON parser, or even a JSON parser with a non-JSON Cloud Collector prefix, extracted fields are clearly highlighted in various colors for easy viewing.

For more information, see Extract Event Fields in the Log Stream Guide.

Support for Regex in Native JSON Parsers

If you need to extract values from a native JSON log that contains a non-JSON prefix, you can now add a Regex extraction when building your JSON parser. During the field mapping process, a new option is available that will allow you to include a Regex extraction. This option can be used to define values that cannot be extracted using JSON field mapping.

For more information, see Extract Event Fields in the Log Stream Guide.

Multi-Log Event Building

Log Stream handling of multi-log event building has been improved to generate a single event from multiple logs across multiple event processing nodes. Leveraging predefined definitions from Exabeam, parsers that use multi-log event building can extract or combine information from key fields across different parsed logs coming from different processing nodes. Based on the parser definitions, Exabeam tracks the logs and stitches them together once conditions are met. This capability enables Exabeam to deliver richer events that provide a more complete picture of your organization's security posture.

For more information, see Multi-Log Event Building in the Log Stream Guide.

May 2024

The following Log Stream features were introduced in May 2024:

Feature	Description
Indicators for Enriched Fields in Live Tail	You can now easily identify enriched fields in the Log Line Details panel of Live Tail, thanks to new indicator icons and tooltips. Each enriched field displays an icon, signaling that it contains data enhanced by a specific type of enrichment data. Click on these icons to view a tooltip that describes the enrichment source. For more information, see Analyze and Troubleshoot in the Log Stream Guide.

Feature

Description

Indicators for Enriched Fields in Live Tail

You can now easily identify enriched fields in the Log Line Details panel of Live Tail, thanks to new indicator icons and tooltips. Each enriched field displays an icon, signaling that it contains data enhanced by a specific type of enrichment data. Click on these icons to view a tooltip that describes the enrichment source.

For more information, see Analyze and Troubleshoot in the Log Stream Guide.

March 2024

The following Log Stream features were introduced in March 2024:

Feature	Description
Native JSON Custom Parsers	Custom parsers can now be built using native JSON field extraction. In modern environments, cloud-based data sources often provide more complex logs in compound JSON formats. Because the JSON data is represented in key/value pairs, working with the native JSON simplifies the tasks of extracting event fields from sample logs and mapping them to the Exabeam common information model, especially when compared to using Regular Expressions. JSON data is also more flexible and elastic, so the order and completeness of fields in incoming logs becomes less of an issue. The Log Stream Parser Manager automatically detects whether your sample logs are in JSON format, and selects the appropriate format for field extraction. However, if you are creating a new parser, during the event field extraction step, you can toggle between the JSON and Regular Expression extraction methods. For more information, see Create a Custom Parser/Extract Fields in the Log Stream Guide.
Platform Notifications for Log Stream	Platform notifications now provide visibility into changes in parser package installations and availability. In this release, the Exabeam Security Operations Platform supports Log Stream as a new notification type. When enabled, notifications will be generated when parser packages are installed, uninstalled, or become available. For more information, see Log Stream in Platform Notifications in the Log Stream Guide.

Feature

Description

Native JSON Custom Parsers

Custom parsers can now be built using native JSON field extraction. In modern environments, cloud-based data sources often provide more complex logs in compound JSON formats. Because the JSON data is represented in key/value pairs, working with the native JSON simplifies the tasks of extracting event fields from sample logs and mapping them to the Exabeam common information model, especially when compared to using Regular Expressions. JSON data is also more flexible and elastic, so the order and completeness of fields in incoming logs becomes less of an issue.

The Log Stream Parser Manager automatically detects whether your sample logs are in JSON format, and selects the appropriate format for field extraction. However, if you are creating a new parser, during the event field extraction step, you can toggle between the JSON and Regular Expression extraction methods.

For more information, see Create a Custom Parser/Extract Fields in the Log Stream Guide.

Platform Notifications for Log Stream

Platform notifications now provide visibility into changes in parser package installations and availability. In this release, the Exabeam Security Operations Platform supports Log Stream as a new notification type. When enabled, notifications will be generated when parser packages are installed, uninstalled, or become available.

For more information, see Log Stream in Platform Notifications in the Log Stream Guide.

January 2024

The following Log Stream features were introduced in January 2024:

Feature	Description
Custom Landscape and Platform Names	Parser Manager has been enhanced to allow you to add custom landscape and platform names for any proprietary or previously unsupported log source.

Log StreamLog Stream Release Notes

Table of ContentsTable of Contents