Create a Log Source Policy

You can configure a log source policy that defines criteria for monitoring log sources and set up notifications for log sources that go silent. Based on the configured policy, you can easily monitor log sources and generate real-time alerts for any anomalies, ensuring timely threat detection and proactive incident response. With the help of log source policies you can identify unique log sources and detect the silent log sources.

Log in to the New-Scale Security Operations Platform with your registered credentials as an administrator.
On the New-Scale Security Operations Platform home page, click the Log Sources tile to view Log Source details. Alternatively, in the left pane, navigate to the Log Sources section.
Click New Policy.

On the New Policy page, enter the following information.

Note

In order to receive notifications for this policy, enable Notification Frequency. Setting a Silent Monitoring Threshold alone is not sufficient to generate notifications.

Policy Name – Specify a name for the log source policy.
Condition – Define criteria to discover log sources using conditions.
- Add Condition – Specify the values for Field and operator based on which you want to discover various log sources.
  You can use any CIM field as a Filter field. For example, to apply the filter for all Microsoft logs, configure the filter vendor contains 'Microsoft'.
  Note
  A log source is always matched to the same policy consistently. After a log source is matched to a specific policy, it will remain associated with that policy unless the policy is disabled or deleted. As for policy conditions, expressions like GetContextAttribute, and GetDynamicContextAttribute are not supported. Although these expressions are available when configuring enrichment rules, you cannot use them in log source policy conditions.
  The IsContextKey can be used with any Context Tables available in your environment.
  If you select the Operator Is Context Key, you can select a context table from the list of context table names.
- Add Group – Click Add Group to configure complex filters based on multiple conditions.

Log Source Identifier – Select up to two identifiers to serve as unique identifiers to discover log sources.

Most commonly, you can use a single identifier. For example, to identify Windows hosts, you can select the host option as the Log Source Identifier. In more complex cases, such as monitoring individual applications within each host, two Log Source Identifiers can be configured—host and product.

Note

If the identifier value of an event exceeds 255 characters, the Log Source Monitoring system excludes that event from log source identification and silence monitoring. Ensure that you set the log source identifier value below 255 characters in your Raw log.

The following table lists use cases to help you identify which identifiers to use for the intended results.

Use Case	Recommended Condition	Recommended Identifiers
Monitor logs from a specific vendor	(contains(vendor, 'VENDOR_NAME')) or (vendor = 'VENDOR_NAME')	vendor, product
Monitor a specific type of device (For example, Firewalls)	(product_category = 'firewall') or (containsAny(m_collector_name, 'palo_alto_fw', 'cisco_asa', ...))	host, product
Monitor Windows devices	(vendor = 'Microsoft') and (platform = 'Windows') or based on collector name for windows events.	host, product
Detect silent or 'flatline' log sources	A condition defining the set of hosts/collectors that should be reporting, e.g., (m_exa_sc_collector_type = 'splunk')	m_exa_sc_collector_name, host
Monitor specific log collectors (for example, Cloud Collectors)	(exists(m_collector_name)) or filter by a list of names (containsAny(toLower(m_collector_type), 'aws', 'azure', ...))	m_collector_name, m_collector_type
Monitor devices based on context (For example, critical servers)	(HasContextKey('CONTEXT_TABLE_NAME', host))	host, product
Monitor (generic monitoring) any log with a specific field	(exists(vendor))	vendor, product

Notification Settings – Set the threshold and frequency for notifications about silent log sources. See Configure Notification Settings for Log Sources for details.

Configure Notification Settings for Log Sources

Using the notification settings, you can configure when and how you get notifications about silent log sources in a log source policy.

Set Silent Monitoring Threshold

Under Silent Monitoring Threshold, in the Warn After Silent for field, select a duration between 30 minutes - 72 hours to set a condition for detecting a silent log source. You can define a period of inactivity after which a log source is considered silent. For example, if a certain log source remains inactive for the specified duration, it will be classified as silent.

Configure Notification Frequency

To manage notification frequency for detecting silent log sources, enable the toggle and select the following options based on your preference.

At most once every 24 hours (Default) – Select to receive in-app notifications. The first notification is sent when a log source first goes silent, with follow-up notifications every 24 hours while it stays silent. This is a default option that provides a daily summary of silent log sources.
Refer to the following example of an in-app notification.
Every time (receive all notifications) – Select to receive all notifications immediately each time a log source goes silent via your preferred notification channel based on the webhook that you configured.
You can create a notification channel and set the notification preferences by navigating to Settings > Global Notifications > Add Channel.
For more information about creating a notification channel for silent log sources, see Add a New Notification Channel in Manage Global Notifications Preferences.Manage Global Notifications Preferences
Refer to the following example of a notification that is sent to the notification channel that you configured.
```
{
  "application": "Log Sources",
  "subject": "Silent Log Sources Detected",
  "description": "An active Log sources is detected to be silent.",
  "severity": "Critical",
  "category": "Collectors",
  "message": "The status of the log source with identifiers {host=IyobdhiL, product=Sysmon} in policy Microsoft_Windows_Domain has changed to silent. Logs may not be ingesting.",
  "recommendation": "Please check your Log sources to troubleshoot."
}
```
Note
When you configure Global Notification options by navigating to Settings > Global Notifications > Delivery, ensure that the notifications for Silent Log Sources Detected and Silent Log Sources Summary are enabled. Enable or disable notification individually at the policy level.
Use the Global Notifications option to enable of disable receiving webhook notifications for Silent Log Sources Detected at subscription level and in-app notifications for Silent Log Source Summary at user level.
- Silent Log Sources Detected – Enable to receive notifications for each log source, every time a log source goes silent. These webhook notifications are sent if you selected the Every time (receive all notifications) option in the log source policy.
- Silent Log Source Summary – Enable to receive notifications for all associated log source policies. These in-app notifications are sent if you selected the At most once every 24 hours (Default) option in the log source policy.

Set the Duration of Log Retention –Time-to-live (TTL)

Set how long the log sources are retained before they expire and automatically deleted. You can set the Time-to-live (TTL) value that ranges from 7 days to 60 days while creating a log source policy. After the TTL expires, the system automatically deletes the data to optimize storage and performance.

To set the Time-to-live (TTL) value within a range of 7 days to 60 days, drag the slider, and click Save.

The Logsource TTL uses TTL value defined by you, to clean up inactive log sources. When a log source remains silent for the specified TTL duration, the log source becomes eligible for removal. Cleanup does not apply to log sources that are in the running state.

Edit a Log Source Policy

Use the following steps to edit the log source policy conditions in the existing policies.

On the Log Sources home page, navigate to the list of existing log source polices section.
Click . Then click Edit Policy Settings.
In the Edit Policy page, edit or add the conditions based on your requirement.
Note
Ensure that you create or edit the log source policy with unique identifiers and conditions. Two log source policies must not have the same identifiers and conditions.
After editing the notification frequency, and Time-to-live (TTL) configuration, click Save to save the updates.
After you update the policy all new log sources are identified based on the updated policy conditions. Based on the log source retention time that you defined in the TTL configuration, the log sources are retained or cleaned up.

Apply Date Filter to View Last Seen Log Sources

To view the last seen log sources of a particular log source policy, you can apply the date filter. You can select a date range to view the log sources for a specific time or clear the filter to view the records that were visible before.

For a particular log source policy in the LAST SEEN filter lets you select a date range to view the log sources for a specific time.

After you select a date range and apply filter, the log source records for the specified range are displayed.

Log SourcesLog Sources

Table of ContentsTable of Contents