Skip to main content

Log StreamLog Stream Guide

Manually Enter JSON Path Expressions

To enter a JSON path expression manually, follow the procedure described below.

Note

If you are using this procedure to extract values from array objects, the examples of extracted expressions in the table below are based on this full input log.

  1. In the JSON Path Expression table at the top of the screen, click + New JSON Path Expression, if necessary, to add an empty row to the table.

  2. In the JSON Path Expressions column, enter a JSON path expression that defines the field you want to extract value from.

    To extract a simple, field-level value, the JSON path expression should be in the following format: $.field_name.

    You can also use this method of manually entering expressions to extract values from more complex array objects. The following table describes the path expressions that are currently supported.

    Tip

    Before you begin mapping complex values, consider whether you may need to create custom fields to map them to. To create custom fields, click the + New custom field option in the left side panel and follow the steps in Fields Panel.

    JSON Path Expression

    Description

    Expected Outcome

    $.rules

    Extracts the entire rules array object.

    [
  {
    "rule_source": "CR",
    "rule_id": "r1",
    "rule": "rule name",
    "rule_reason": "resolved rule reason",
    "rule_severity": "correction prority",
    "type": "Brute Force",
    "rule_usecases": [
      "use case1",
      "use case2"
    ],
    "tags": [
      "t1",
      "t2"
    ],
    "mitre_labels": [
      {
        "technique_key": "T1071",
        "technique": "Valid Accounts",
        "tactic": "Privilege Escalation",
        "tactic_key": "TA0004"
      },
      {
        "technique_key": "T1072",
        "technique": "Valid Accounts",
        "tactic": "Initial Access",
        "tactic_key": "TA0001"
      }
    ]
  },
  {
    "rule_source": "BEAM",
    "rule_id": "r2",
    "rule": "rule name",
    "rule_reason": "resolved rule reason",
    "rule_severity": "correction prority",
    "type": "Brute Force",
    "rule_usecases": [
      "use case1",
      "use case2"
    ],
    "tags": [
      "t1",
      "t2"
    ],
    "mitre_labels": [
      {
        "technique_key": "T1073",
        "technique": "Valid Accounts",
        "tactic": "Privilege Escalation",
        "tactic_key": "TA0004"
      },
      {
        "technique_key": "T1074",
        "technique": "Valid Accounts",
        "tactic": "Initial Access",
        "tactic_key": "TA0001"
      }
    ]
  }
]

    $.metadata_tags

    Extracts a list of metadata tag strings from rules array.

    ["Tag1","Tag2","Tag3"]

    $.rules[*].rule_id

    Extracts a list of rule ID strings from the rules array.

    ["r1","r2"]

    $.rules[*].mitre_labels

    Extracts the nested mitre label arrays from within rules array object.

    [
  [
    {
      "technique_key": "T1071",
      "technique": "Valid Accounts",
      "tactic": "Privilege Escalation",
      "tactic_key": "TA0004"
    },
    {
      "technique_key": "T1072",
      "technique": "Valid Accounts",
      "tactic": "Initial Access",
      "tactic_key": "TA0001"
    }
  ],
  [
    {
      "technique_key": "T1073",
      "technique": "Valid Accounts",
      "tactic": "Privilege Escalation",
      "tactic_key": "TA0004"
    },
    {
      "technique_key": "T1074",
      "technique": "Valid Accounts",
      "tactic": "Initial Access",
      "tactic_key": "TA0001"
    }
  ]
]

    $.rules[*].mitre_labels[*]. technique_key

    Extracts the technique key strings from the nested mitre label arrays

    ["T1071","T1072","T1073","T1074"]

    $.rules[*].mitre_labels[0]. technique_key

    Extracts a technique key string from the specific nested mitre label array for each rule in the rules array.

    ["T1071","T1073"]

    $.rules[0].mitre_labels[*]. technique_key

    Extracts technique key strings from each nested mitre label array for a specific rule in the rules array.

    ["T1071","T1072"]

  3. After entering a manual JSON path expression, use the drop down menu in the Select a Field column to select a CIM field that you want to map the expression to. The options are listed according to whether they are core, detection, or informational fields for the specified activity type. You can also click + New field at the bottom of the list and create a custom field.

  4. Optionally, if you want to apply conditions to the way a field value from the log is extracted for mapping to the selected CIM field, insert a logical condition in the Extraction Condition column.

    For more information about implementing this type of logic, see Add Logic to JSON Field Extraction.

  5. Optionally, you can also use the Substring Regex column to define a condition that extracts a specific value from within a string and maps only that value to a CIM field.

    For more information about implementing this type of logic, see Add Logic to JSON Field Extraction.

  6. To save the mapping, click the check at the far right side of the row. The field mapping is saved.

    manual-json-path-expression.png

  7. Check the Sample Log Lines section at the bottom of the page, to see if the mapped field extracts any value from the sample log. Extracted values appear above the log line and are highlighted in various colors.

    Note

    For mapped field values extracted from an array object, the extracted values appear listed above the log in various colors but are not highlighted in the sample log line itself.

    manual-array-token.png

  8. To map another JSON field, click + New JSON Path Expression. A new row appears in the table.

  9. Repeat the process to manually enter a JSON path expression, map it to a CIM field, and click the check to save the field mapping.

  10. Continue to add JSON path expression rows until you have mapped all of the core fields and any of the detection, informational, or custom fields you think will be useful.

  11. After you have selected all of the necessary fields for extraction, ensure the fields are in the correct order, review the matching event type fields and log values, then continue building your custom parser by configuring event builders.