Correlation RulesCorrelation Rules Guide

When you create a correlation rule, you can adjust how long a rule is delayed from evaluating events so that it includes late-arriving events in its evaluation.

To determine for how long you should delay a rule, you must first understand how correlation rules queries events to evaluate. After events are built, they're stored in Search. A correlation rule queries Search for events every four minutes. When a rules queries Search, it evaluates a batch of events created in a four-minute window. The evaluation delay value determines from how long ago the rule queries for a batch, and the rule uses this timestamp as the upper bound of the four-minute evaluation window. By default, the evaluation delay value is seven minutes.

For example, if a rule evaluates a batch of events at 9:20 AM, the approxLogTime of those events is between 9:09 and 9:13 AM. 9:13 AM is the upper bound of a four-minute window beginning at 9:09 and is seven minutes before 9:20 AM. The rule evaluates another batch of events four minutes later, at 9:24 AM. The approxLogTime of those events is between 9:13 and 9:17 AM.

If an event takes more than seven minutes to appear in the Search store, the event won't be in the Search store when the rule queries it. Even if the event appears in the Search store later, the rule won't go back to evaluate it. Because the rule never evaluates the event, it misses a potential trigger.

To ensure a correlation rule evaluates and triggers against delayed events, you can adjust the evaluation delay value so it evaluates events in an earlier window. For example, let's say Event A was created at 9:45 AM and stored in the Search store at 10:00 AM; in other words, it was 15 minutes late. If a correlation rule with the default seven-minute evaluation delay value evaluates a batch of events at 10:00 AM, the events would have an approxLogTime between 9:49 and 9:53 AM. Event A isn't included in the batch and has missed the window for being evaluated. If the correlation rule has a 11-minute evaluation delay and evaluates a batch of events at 10:00AM, the events would have an approxLogTime between 9:45 and 9:49. Event A is included in this batch.

To determine for how long events are delayed and what evaluation delay value you should use, use the Log Delay Insights dashboard. Percentage of events arriving with in a time threshold per hour: By Activity Type is a column chart that counts the number of events that arrive within and outside your evaluation delay value and groups those events by activity type. Percentage of events arriving with in a time threshold per hour: By Vendor is a column chart that counts the number of events that arrive within and outside your evaluation delay value and groups those events by vendor. For either chart, adjust the Target Delay in Minutes parameter until all events of interest arrive within your evaluation delay value. Use the value of Target Delay in Minutes as your evaluation delay value.

You can set a delay between seven and 360 minutes.