Skip to main content

CollectorsCloud Collectors Administration Guide

Prerequisites to Configure GCP Cloud Logging Cloud Collector

Before you configure the Google Cloud Platform (GCP) Cloud Logging cloud collector you must complete the following prerequisites.

  • Ensure that the https://*.cloud.google.com service is open for communication with the New-Scale Security Operations Platform.

  • Obtain the service credentials JSON file.

  • Ensure that you have the administrative rights to configure GCP.

  • Add Accounts for Google Cloud Collectors.

  • Obtain a Subscription ID – Set up a subscription for the Pub/Sub topic to which the log data is exported. Note the Subscription ID that you set while creating a subscription in GCP Pub/Sub and subscription. Subscription ID is associated with your Pub/Sub topic. For more information, see Set up Pub/Sub topic and subscriptions in the Google documentation.

  • Obtain a Project ID – Note the Project ID that you set while creating the Google service account. See Create a Service Account and Obtain a Project ID.

  • Assign the required permissions to the service account – Assign the following permissions to the service account to support auto-scaling on your Google Cloud Platform:

    • dataflow.metrics.get

    • monitoring.metricDescriptors.get

    • monitoring.metricDescriptors.list

    • monitoring.monitoredResourceDescriptors.list

    • monitoring.timeSeries.list

  • Assign the required subscription permissions – Assign pubsub.subscriptions.consume permission to the service account.

Obtain the Service Credentials JSON File

The GCP APIs are authenticated via a service account. You must obtain the service credentials JSON with the API key that you can use while configuring the GCP connector.

To obtain the service credentials JSON file:

  1. Enable the stack driver API via:     https://console.cloud.google.com/apis/api/logging.googleapis.com/overview?project=<GCP project-to-monitor>

    Note

    The project where the service account is set up must grant permissions to other projects that contain Sinks with the data.

  2. To create the service account key, in the Google cloud console, navigate to the APIs & Services > Credentials.

  3. In the Credentials section, click Create Credentials then click Service account.

  4. Specify the account name and description.

  5. Select the roles Logs Viewer and Private Logs Viewer.

  6. Click DONE.

  7. On the Service Accounts list page, click the name of the account that you created.

  8. Navigate to Keys > Add Key > Create New Key.

  9. In the Key type section, select JSON.

  10. Click Create. A JSON file is downloaded. For more information see the Google Cloud documentation.

  11. Save the JSON file at a secure location to use while configuring the GCP connector. Copy the JSON file contents to se it while creating a sharable Google account.

Create a Service Account and Obtain a Project ID

  1. To create a service account in the project in which the Pub/Sub subscription has been created, in the Google Cloud console, navigate to IAM & Admin > Service Accounts > CREATE SERVICE ACCOUNT page.

  2. Select a Cloud project if you already created a project. If not, create a Cloud project.

  3. Enter the project name and edit the automatically generated project ID. Note the Project ID that you set while creating the Google service account. You require to use the project ID while configuring the GCP Pub/Sub Cloud Collector.

  4. For more information and relevant steps, see Creating and managing service accounts in the Google documentation.

Obtain a Subscription ID

To obtain a subscription ID, first create a sink and GCP Pub/Sub topic and a subscription. Open the Log Router page in your Google logging platform, then select the project containing the logs you want to route. Click Create sink, provide a name and an optional description, and choose the destination where the logs should be sent. Ensure that in the Sink destination section, you select Cloud Pub/Sub topic as a sink service and create or select a pub/sub topic to which you want to send the data to.

While you select or create a particular pub/sub topic to receive the routed log entries, you can add a subscription. You must note this subscription ID that you need to provide while configuring the GCP Cloud Logging cloud collector.

Optionally, set an inclusion filter to specify which logs to route, and an exclusion filter to exclude certain logs. Finally, click Create sink to complete the setup.

For more information, see Create a sink in Google documentation.