Skip to main content

Responses are generated using AI and may contain mistakes.

CollectorsCloud Collectors Administration Guide

Table of Contents

Recover Stopped Log Ingestion for Microsoft 365 Exchange Admin Reports Cloud Collector

To recover missing data after an ingestion gap, first use the Message Trace v1 search in Cloud Collectors to identify exactly when the logs stopped. Once the timeframe is confirmed, re-create the cloud collector and adjust the Ingest From setting to the timestamp of the interruption to backfill the lost logs.

Use the following steps to recover stopped log ingestion.

  • Identification of Ingestion Discontinuance: Check when log ingestion is stopped.

  • Creation of a new collector with MessageTraceV2 data source, in addition to the existing collector for which log ingestion has been stopped.

Check When Log Ingestion Stopped

Use the following steps to access the Search page for the cloud collector instance for which log ingestion is stopped.

  1. Log in to the New-Scale Security Operations Platform with your registered credentials as an administrator.

  2. Navigate to Collectors > Cloud Collectors.

  3. On the Overview tab, search for the Microsoft 365 Exchange Admin Reports collector instance using the search box.

  4. Locate the Microsoft 365 Exchange Admin Reports collector instance that has log ingestion stoppage error and click the more options menu (...).

    Microsoft_365_Exchange_Admin_Reports_Cloud_Collector-search.png

  5. Click Open in Search.

  6. Expand the search time range to determine when ingestion stopped and the time of last successful log ingestion by clicking the calendar icon in the Search query box.

  7. To filter the search results to view Message Trace logs only, edit the auto-generated query to add the following regex.

    AND RGX("dproc=MessageTrace")

  8. Record the timestamp of the last successfully ingested log. Use this value while updating the Ingest From setting of the cloud collector.

Create a New Collector with MessageTraceV2 Data Source

To fetch missing data, create a new Collector with the MessageTraceV2 data source and set IngestFrom to the time when the MessageTrace data source stopped receiving logs. Use New Scale Search to record the timestamp of the last ingested log for the MessageTrace data source. If the original collector already has the MessageTraceV2 data source enabled, the new collector will duplicate MessageTraceV2 data. After backfilling is complete, stop the new collector (only if the original collector already uses MessageTraceV2) to prevent further duplication.

Note

  • After a Service Principal is created, provisioning on the Entra ID (Azure AD) side may take several hours to complete. You can create a new service principal using Graph Explorer or Microsoft Graph PowerShell SDK. For more information, see Graph-based message trace API onboarding guide in the Microsoft documentation.

  • If you encounter a 401 error during or after the Test Connection and update, this typically indicates that Entra ID provisioning has not yet completed. It is recommended to wait for a few hours.