Skip to main content

Log StreamLog Stream Guide

Manually enter a Regular Expression or Key

To enter a regular expression manually, follow the procedure described below.

  1. In the Regular Expression table at the top of the screen, click + New Regular Expression, if necessary, to add an empty row to the table.

  2. In the Regular Expression column, enter a regular expression that defines the field you want to extract value from.

    enter-regex.png

    Keep in mind the following:

    • You should include a value from the log in the regular expression. You must include the field name in the capturing group. If you enter a key, include it in the regular expression, outside the capturing group.

      For example, let's say you have an field called alert-name. In the sample logs, you have a key, foo=, and values 696 and 636. You can enter a regular expression that is more specific; for example, foo=({alert-name}6[39]6). Or, you can enter a regular expression that is more inclusive; for example, foo=({alert-name}\d+).

    • You can enter a regular expression that extracts multiple fields. Consider creating an expression like this if your log values are concatenated; your logs aren't formatted in key-value pairs, so you're identifying values based on location; or your parser covers multiple, slightly different log formats.

      For example, in the s-xml-4663 parser, a single regular expression extracts the process, directory, and process_name fields:

      ecp-apg-extractmultifield.png

    • In some cases, the host and time fields are required but you can't find appropriate values in your sample logs. If you retrieved your sample logs from a SIEM, like Advanced Analytics, you can enter a regular expression using special keys, exabeam_host and exabeam_time. For example, exabeam_host=({host}[\w.\-]+). Your SIEM may add a header to the raw log that includes host and time information, and these special keys parse that information.

  3. To save the mapping, click the check at the far right side of the row. The field mapping is saved.

  4. Check on the following after you save the mapping:

    • In the table of mapped regular expression fields at the top of the screen, ensure that the appropriate CIM field has been automatically populated in the Maps to Field column, based on the regular expression you just saved.

    • In the % LINES column of the same table, verify the percentage of log lines from which the regular expression extracted value.

    • In the Sample Log Lines section at the bottom of the screen, ensure that the mapped field extracts the appropriate value from the sample log. Mapped values appear highlighted in various colors.

    logline-regex-mapping.png

  5. Repeat the process to map another field on the next line using a regular expression and save the field mapping.

  6. Continue to add new regular expression rows until you have mapped all of the core fields and any of the detection, informational, or custom fields you think will be useful.

  7. After you have selected all of the necessary fields for extraction, ensure the fields are in the correct order, review the matching event type fields and log values, then continue building your custom parser by configuring event builders.