Skip to main content

Responses are generated using AI and may contain mistakes.

CollectorsCloud Collectors Administration Guide

Table of Contents

Prerequisites to Configure Azure Blob Storage Logs Cloud Collector

Before you configure the Azure Blob Storage Cloud Collector you must complete the following prerequisites.

  • Configure Azure Storage account – Configure a queue and endpoint details, set permissions for the SAS token, and obtain storage account properties.

  • Obtain tenant ID, client ID, and client secret.

  • Assign the required roles to the Azure Storage account.

Configure Azure Storage Account

The Azure Storage Cloud Collector consumes data stored in Azure Storage Accounts to retrieve various Azure data sources.

An Azure data source stores data in:

  • Azure Storage Account's Blob store – The store must be StorageV2 to alert its queue on new blobs.

  • Clear-text files with one event per line. For example, .log or.txt format. You can also use JSON-line format with each line as a valid event in JSON format.

Use the following steps to configure a queue and endpoint details, set permissions for the SAS token, and obtain storage account properties.

  1. Go to the the storage account where that data that you want to consume is located.

    1. Go to https://portal.azure.com.

    2. Navigate to Storage Accounts and then select your storage account.

    3. Verify the type of storage is StorageV2 in the storage accounts table, or in the Overview > Account Kind.

  2. Configure a Queue in that storage account to be notified when new Blobs are added to its blob store.

    1. Create a queue dedicated for the connector.

      1. Navigate to Queues.

      2. Click + Queue, name it something unique, for example sk4queue, and then click OK.

      3. Record the queue name for later use.

    2. Navigate to Events.

    3. Click + Event Subscription.

    4. Enter a descriptive name like notify-sk4queue-on-new-blobs.

    5. Verify that Events Schema are set to the default Event Grid Schema.

    6. Under Event Types > Filter to Event Types make sure only the Blob Created is selected.

    7. Configure the Endpoint Details:

      1. In Endpoint Type, select Storage Queues.

      2. In Endpoint, click Select an endpoint.

      3. In the panel that opened, select the subscription where the current Storage Account is located.

      4. Select the storage account.

      5. Select the queue that you created in the first step, for example sk4queue.

      6. Confirm Selection.

    8. Click Create.

      While the queue and the blob can be created in different storage accounts, for the sake of simplicity, they're configured in the same storage account.

  3. Create minimal-permissions SAS tokens.

    1. Navigate to Security + Networking > Shared access signature.

    2. Create a SAS token for the blob storage.

      1. Select the Blob from the Allowed services section.

      2. Select the Object from the Allowed resource types.

      3. Select the Read from the Allowed permissions.

      4. Set the End time to a time far in the future, for example 10 years in the future, so the connector doesn't stop working because of SAS Token invalidation.

      5. Click Generate SAS and connection string.

      6. Copy the value from the SAS Token field.

        For example, the token can look like ?sv=2018-03-28&ss=q&srt=o&sp=p&se=2019-09-23T17:52:57Z&st=2019-09-23T09:52:57Z&spr=https&sig=aASkYB8%2BGf21fMXl3Bnf0Mod7n81Dq8E%2FeggjEQr%2BD8%3D.

      Note

      Similarly, create a SAS Token for Queue with the following settings.

      • Allowed services: Queue

      • Allowed services resource types: Object

      • Allowed permissions: Update and Process

  4. Obtain other Storage Account properties.

    1. Navigate to Properties.

    2. Record the value from the Primary Queue Service Endpoint field, under the Queue service section.

  5. Proceed to Configure the Azure Storage Logs Cloud Collector.

    While configuring the cloud collector, select Azure Storage - SAS Tokens as the authentication method and configure the Sync Strategy.

Obtain Tenant ID, Client ID, and Client Secret

To obtain the Tenant ID, Client ID, and Client Secret, create a Microsoft Entra ID application (formerly called Azure Active Directory) in the Microsoft Azure portal using the following steps.

Create and Register a Microsoft Entra ID Application

  1. Log into your Azure account by accessing the Azure portal.

  2. From the available Azure Services, select Microsoft Entra ID (formerly called Azure Active Directory).

    ms-entra-id.png

  3. From the left navigation pane, select App Registrations, then click New Registration.

  4. On the Register an application page, enter the following information:

    • Name – Specify a name for the new application. For example, Exabeam MS Entra ID App.

    • Supported account types – Select the account type Accounts in this organizational directory only.

    • Redirect URI – (Optional) Select the Web platform and specify a URI.

  5. Click Register. When registration is complete, the Overview tab of the new application is displayed.

    azure-ad-app-overview.png

  6. Copy and make a note of the values for both the Application (client) ID and the Directory (tenant) ID. You will need these values when configuring a Microsoft cloud collector.

  7. In your new Microsoft Entra ID application, click Certificates & secrets in the left navigation pane.

  8. On the Client Secrets tab, click New client secret.

  9. Enter a description and an expiration period for the secret.

  10. Click Add.

  11. Copy and make a note of the value of the new client secret that the Client secrets tab displays.

Assign the Required Roles to the Azure Storage Account

To assign the Storage Blob Data Contributor and the Storage Queue Data Contributor role to the Azure storage account, use the following steps:

  1. On the Azure portal, navigate to Storage Accounts and select your storage account.

  2. In the left pane, click Access control (IAM).

  3. Click Add and then click Add role-assignment.

  4. In the Add role assignment section, under Role, search for and select Storage Blob Data Contributor, and Storage Queue Data Contributor roles.

  5. Click Next.

  6. In the Members section, click Select Members.

  7. Search for and select your Microsoft Entra ID application.

  8. Click Next and then click Review + assign.

    After the role is assigned to your storage account, the storage account has permission to read, write, and delete blob data in Azure Storage containers and read and write messages to Azure Storage queues.