Skip to main content

Cloud ConnectorsSetup

Table of Contents

Docker Management

Log in to the Docker Repository

During the installation process, images are pulled from the Exabeam Cloud Connectors docker repository. To pull these images, you need to be logged in. If you are not logged in to the docker repository, you will receive the following error:

Error response from daemon: pull access denied for image, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Failed to login to docker registry. error code [1]. Exiting.

If you are not logged in, you can validate the license key and pull images from the docker repository:

  1. Run the following command to download and run the docker-login script.

    curl -L https://download.skyformation.net/v2/download/installer/edge/docker-login.sh | bash -s -- -l=<license_key>
  2. If you use a proxy server, you can add it as a parameter when you download and run the script: -x=<proxy>

Create a Docker-Compose Override File

The Compose file is a YAML file that defines services, networks, and volumes. Exabeam Cloud Connectors provides a docker-compose.override.yml which contains the base configuration. You can use this file to add configuration overrides for existing services - for example, preferred ports and volumes. This topic will show where you can find the default Exabeam Cloud Connectors docker-compose.override.yml file, and how to create a docker-compose.override.yml file.

  1. Find the docker-compose.yml file.

    The docker-compose.yml file is stored in /opt/exabeam/data/sk4 on the Exabeam type of deployment or in /opt/sk4 in standalone deployments. You can find its location by looking at the Exabeam Cloud Connector’s service file /etc/systemd/system/sk4compose.service.

  2. Create a docker-compose.override.yml file.

    In that directory, you may create a docker-compose.override.yml file and add the configurations you want. Make sure you add the service name and the configurations you would like to override or add.

    For example, we can change the ports of the sk4tomcat container. The docker-compose.override.yml file will look like:

    version: '3'
    services:
      sk4tomcat:
        ports:
        - "8443:8443"

    And the docker-compose.override.yml will look like:

    version: '3'
    services:
      sk4tomcat:
        ports:
        - "8080:8080"
  3. Update the Exabeam cloud connector’s service file docker-compose execute command

    In the etc/systemd/system/sk4compose.service file add a -f flag with the docker-compose.override.yml file:

    ExecStart=/usr/local/bin/docker-compose -f <path to docker-compose.yml> -f <path to docker-compose.yml.override.yml> up
    ExecStop=/usr/local/bin/docker-compose -f <path to docker-compose.yml> -f <path to docker-compose.override.yml> down

    *When the path to the docker-compose.yml and docker-compose.override.yml files is in /opt/exabeam/data/sk4 on an Exabeam type deployment or in /opt/sk4 in standalone deployments.

Change the Docker IP Address Space

Change the Default Docker Network (bridge0) Subnet

  1. Create a docker config file if one doesn't exist in /etc/docker/daemon.json.

  2. Add an entry to the daemon.json with the subnet for the docker bridge0 to run in, under the "bip" entry. For example:

    { 
        "bip": "192.168.1.5/24"
    }
  3. Restart the entire docker service to take effect: sudo systemctl restart docker.

    Note

    Take care to not use the loopback address in the subnet such as IP addresses that end with 0 like 192.168.1.0/24. The bridge0 subnet should have enough addresses for all the containers on the machine that uses the default network. Exabeam Cloud Connectors does not use the default network.

  4. Verify the docker service took effect by running ifconfig docker0 | grep -Po '(?<=inet )[\d.]+' . It should print out the IP address specified in "bip" example of daemon.json full file with only the bip configured in it.

    Source: https://docs.docker.com/v17.09/engine/userguide/networking/default_network/custom-docker0/

Change the Subnet Network on Which the Exabeam Cloud Connectors Docker Containers Run

  1. Create a docker-compose-overrides.yml if one doesn't exist in /opt/sk4 OR /opt/exabeam/data/sk4.

    For Exabeam deployments. if the file did not exist previously, it should be referenced in the systemd unit of Exabeam Cloud Connectors in /etc/systemd/system/sk4compose.service, like so:

    • old: ExecStart=/usr/local/bin/docker-compose -f /opt/sk4/docker-compose.yml up

    • new: ExecStart=/usr/local/bin/docker-compose -f /opt/sk4/docker-compose.yml -f /opt/sk4/docker-compose-overrides.yml up

    Make sure to adjust the path of the overrides file to your deployment.

  2. Add an entry to the overrides file to specify the desired subnet.

    Make sure the subnet does not collide with the bridge0 subnet.

    For an example of full docker-compose-overrides.yml, see:

    version: '3'
    networks:
    isolated_nw:
    driver: bridge
    ipam:
    config:
    - subnet: 192.167.0.0/16

    This file is Indention sensitive. Copy and paste the above and modify the values to avoid indention issues.

  3. If you changed the systemd unit, reload the systemd daemon: sudo systemctl daemon-reload

  4. Restart the Exabeam Cloud Connectors service to take effect: sudo systemctl restart sk4compose

  5. Verify it worked by running:

    sudo docker inspect $(sudo docker container ls -q) | grep \"IPAddress\"

    All the IP addresses should be in the subnet.

    Note

    If running of a host with other docker services, the IP addresses of the other docker services should not be affected.

    ipam source: https://docs.docker.com/compose/compose-file/#ipam

Set or Change Mounted Volumes Directory

Use this workflow to mount the docker volumes used by the Exabeam Cloud Connectors server to a directory of your choice. This procedure is intended for modifying a pre-installed server. If the server hasn’t been installed yet please provide a mounted directory during the install process.

This procedure relates to 2.4.x versions only.

Determine Your Mounted Volumes Directory

There are two different scenarios:

  1. To determine which option you need, inspect one of the Exabeam Cloud Connectors volumes (here we arbitrarily choose sk4_conf) by running:

    sudo docker volume inspect sk4_conf

    If, in the JSON object printed to console, Options is null then the volume is not mounted (the second option), the volume data is stored under the docker root directory and the full path can be found, in the JSON, under Mountpoint.

    [
        {
            "CreatedAt": "2019-12-30T17:16:36Z",
            "Driver": "local",
            "Labels": {
                "com.docker.compose.project": "sk4",
                "com.docker.compose.volume": "conf"
            },
            "Mountpoint": "/var/lib/docker/volumes/sk4_conf/_data",
            "Name": "sk4_conf",
            "Options": null,
            "Scope": "local"
        }
    ]
    

    Note

    Docker root directory, in the above example, is /var/lib/docker and the volumes root directory (SRC_VOLUMES_DIR) is /var/lib/docker/volumes.

    If, on the other hand, Options is not null then the mounted directory is specified under the Options.device.

    [
        {
            "CreatedAt": "2019-12-30T15:08:19Z",
            "Driver": "local",
            "Labels": {
                "com.docker.compose.project": "sk4",
                "com.docker.compose.volume": "conf"
            },
            "Mountpoint": "/var/lib/docker/volumes/sk4_conf/_data",
            "Name": "sk4_conf",
            "Options": {
                "device": "/opt/exabeam/data/sk4/conf",
                "o": "bind",
                "type": "none"
            },
            "Scope": "local"
        }
    ]
    

    Note

    Docker root directory, in the above example, is /var/lib/docker and the volumes mounted root directory (SRC_VOLUMES_DIR) is /opt/exabeam/data/sk4. Although docker allows you to set, for each volume, a different configuration, Exabeam Cloud Connectors does not support it in its install and upgrade procedures.

  2. Proceed to the relevant procedure to set or change your mounted volumes directory.

Move a Local Directory to Another Local Directory

  1. Stop the sk4compose service.

    systemctl stop sk4compose
    
  2. Copy volumes data from current root directory (SRC_VOLUMES_DIR) to a new existing root directory (DST_VOLUMES_DIR).

    rsync -aP SRC_VOLUMES_DIR DST_VOLUMES_DIR
    
  3. Change the directory to the Exabeam Cloud Connectors base directory where the docker-compose.yml is located, by default either /opt/sk4 or /opt/exabeam/data/sk4. You can find its location by examining the Exabeam Cloud Connector’s service file /etc/systemd/system/sk4compose.service.

  4. Back up docker-compose.yml.

    cp docker-compose.yml docker-compose.yml.bak
    
  5. Update volumes mounted directory in the docker-compose.yml (replace DST_VOLUMES_DIR).

    docker run --rm -v `pwd`/docker-compose.yml:/sk4/docker-compose.yml "498895000136.dkr.ecr.us-east-1.amazonaws.com/upgrade-scripts:1.0.0" node map_volumes_to_dir.js -f /sk4/docker-compose.yml -d DST_VOLUMES_DIR
    
  6. Verify that indeed all volumes are now mounted to the DST_VOLUMES_DIR.

    grep device: docker-compose.yml
    
  7. Delete Exabeam Cloud Connectors docker volumes (this operation will not delete the actual data stored in SRC_VOLUMES_DIR)

    docker volume rm $(docker volume ls -q | grep sk4_) -f
    
  8. Restart the Exabeam Cloud Connectors service.

    systemctl start sk4compose
    
  9. To verify Exabeam Cloud Connectors is working properly, log in to the web interface and verify that all the accounts are still visible.

  10. Delete the previously mounted directories from SRC_VOLUMES_DIR.

    Note

    If the SRC_VOLUMES_DIR is the Exabeam Cloud Connectors base directory, do not delete YAML and ENV files. This procedure keeps the Exabeam Cloud Connectors base directory unchanged.

Move a Docker Volume Under the Docker Root Director to a Local Directory

  1. Stop the sk4compose service.

    systemctl stop sk4compose
  2. Copy volumes data from current docker volumes root directory (SRC_VOLUMES_DIR) to the new root directory (DST_VOLUMES_DIR).

    # REPLACE SRC_VOLUMES_DIR with docker's volumes root directory (by default its /var/lib/docker/volumes)
    _volumesDir=SRC_VOLUMES_DIR
    
    # REPLACE DST_VOLUMES_DIR with the mounted directory in which you want all Exabeam Cloud Connectors data to be stored
    _newMountedDir=DST_VOLUMES_DIR
    
    _volumePathSize=${#_volumesDir}
    for _dirName in $_volumesDir/sk4*; do
        _srcDir="${_dirName}/_data/"
        # volume name starts at _volumePathSize + length("/sk4_") + 1 
        _ind=$(($_volumePathSize+6))
        _dstDir=$_newMountedDir/$(echo $_dirName | cut -c$_ind-${#_dirName} | sed 's/sk4_//g')
        mkdir -p $_dstDir
        echo "Copy data from $_srcDir to $_dstDir"
        rsync -aP $_srcDir $_dstDir
    done
  3. Change directory to Exabeam Cloud Connectors base directory where the docker-compose.yml is located in /opt/sk4 (default) or /opt/exabeam/data/sk4. You can find its location by looking at the Exabeam Cloud Connector’s service file /etc/systemd/system/sk4compose.service.

  4. Verify that indeed all volumes data was copied from SRC_VOLUMES_DIR to DST_VOLUMES_DIR. As an extra measure of precaution we recommend to backup the sk4_conf, the sk4_zoo_data and the sk4_zoo_datalog mounted points.

    tar -zcvf conf_volume.tar.gz `docker volume inspect --format='{{.Mountpoint}}' sk4_conf`
    tar -zcvf zoo_data_volume.tar.gz `docker volume inspect --format='{{.Mountpoint}}' sk4_zoo_data`
    tar -zcvf zoo_datalog_volume.tar.gz `docker volume inspect --format='{{.Mountpoint}}' sk4_zoo_datalog`
  5. Back up docker-compose.yml.

    cp docker-compose.yml docker-compose.yml.bak
  6. Update volumes mounted directory in the docker-compose.yml (replace DST_VOLUMES_DIR).

    docker run --rm -v `pwd`/docker-compose.yml:/sk4/docker-compose.yml "498895000136.dkr.ecr.us-east-1.amazonaws.com/upgrade-scripts:1.0.0" node map_volumes_to_dir.js -f /sk4/docker-compose.yml -d DST_VOLUMES_DIR
  7. Verify that indeed all volumes are now mounted to the DST_VOLUMES_DIR.

    grep device: docker-compose.yml
    
  8. Delete Exabeam Cloud Connectors docker volumes.

    Warning

    Because this operation WILL DELETE the actual data stored in docker volumes root directory, backing up the data, as described in step 4, is highly recommended.

    docker volume rm $(docker volume ls -q | grep sk4_) -f
    
  9. Restart the Exabeam Cloud Connectors service.

    systemctl start sk4compose
    
  10. To verify Exabeam Cloud Connectors is properly working, log in to the web UI and verify that all the accounts are still visible.

  11. You can now delete backup archives created in step 4.