Skip to main content

Log StreamLog Stream Guide

Add Conditions

Conditions are a string, or set of strings, that uniquely exist in specific logs.

Keep in mind that a log is evaluated against custom parsers first, then out-of-the-box parsers. If a log doesn't include all conditions, it wont match the parser.

Note

To avoid matching a log with the wrong parser, you must carefully choose conditions that uniquely exist in specific logs. Conditions can't be too general or strict.

Let's take this log as an example:

%{host} KAFKA_CONNECT_SYSLOG: <110>1 2020-04-01T02:08:22.073Z 485cafdca7ac Skyformation - 1192192023904365343 - CEF:0|Skyformation|SkyFormation Cloud Apps Security|2.0.0|sk4-audit-event|audit-event|0|cat=audit cfp3=34.0544 cfp3Label=latitude cfp4=-118.244 cfp4Label=longitude cs6Label=raw-event destinationServiceName=Okta flexString1=app.inbound_del_auth.login_success flexString1Label=application-action src=13.108.238.8 suid=system suser=barbara.salazar request=Success deviceInboundInterface=5d6baf21-742d-11ea-9f5a-7fa07153

If applied conditions are too general, logs that match these conditions may not parser correctly. For example, if you choose just [“CEF:0|Skyformation”] as a condition, the parser matches any log received from Exabeam Cloud Connectors. Choosing two conditions, like [“CEF:0|Skyformation”, “destinationServiceName=Okta”] is better, but the parser still matches any Okta log received from Exabeam Cloud Connectors.

If conditions are too strict, the parser won't cover all relevant logs. In general, don't use values for log variables like IP address, time, and host name as conditions. For example, if you choose [“CEF:0|Skyformation”, “destinationServiceName=Okta”, “src=13.108.238.8”] as your conditions, the parser only matches Okta logs from source IP address 13.108.238.8 and that were received from Skyformation; only very specific logs would match all these conditions.

To ensure that your conditions aren't too general or strict, it's best if you include conditions that indicate:

  • Vendor or product that generated the log—for example, "Windows", "Okta", "Cisco ASA v9.8".

  • Log format—for example, "CEF:0", "LEEF:1.0", "LEEF:2.0".

  • Event type—for example, "app.inbound_del_auth.login_success", "action=security-alert-high", "vpn-session-started".

If your sample logs don't contain this information, choose other conditions that are unique to your sample logs product or vendor, format, and event type. Let's take this log as an example:

<134>Aug 30 22:35:23 DNSLOG: src=192.25.5.12 spt=53 dst=156.140.56.11 dpt=28317 proto=UDP mid=59898 op=0 fl=|QR|RD|RA| rc=SRVFAIL cl=IN tp=PTR name=70.6.29.113.in-addr.arpa aa_log_source=DNSLOG_FROM_CISCO_UMBRELLA

You could select "DNSLOG:", "src=", "spt=", "dst=", "dpt=", "proto=", "mid=", "fl=", "rc=", and "name=" as your conditions because they are unique to your log and unlikely to appear in logs from other vendors or products.

As a last resort, before the log is ingested, append a value to the log so the parser can easily identify it. For example, append a value like "aa_log_source=DNSLOG_FROM_CISCO_UMBRELLA" to the end of the log and use it as your only condition:

<134>Aug 30 22:35:23 DNSLOG: src=192.25.5.12 spt=53 dst=156.140.56.11 dpt=28317 proto=UDP mid=59898 op=0 fl=|QR|RD|RA| rc=SRVFAIL cl=IN tp=PTR name=70.6.29.113.in-addr.arpa aa_log_source=DNSLOG_FROM_CISCO_UMBRELLA

To add conditions:

  1. Enter a value in the SELECT CONDITIONS bar, or in the list of raw log lines, highlight a string.

  2. Click Add Condition.

    The string is highlighted in yellow so you can see if all sample logs contain the string.

  3. Carefully review the logs that match the conditions you added.

    • To view the log lines that match every condition, click the Matches tab.

    • To view the log lines that don't match every condition, click the Non-matches tab.

  4. Click Next.

    The Parser Info page appears.