Skip to main content

Cloud ConnectorsSetup

Table of Contents

High Availability

High Availability - Active Passive Deployment

To reduce downtime in the event of a server crash, you can deploy Cloud Connectors in high availability (HA) - Active Passive mode. With Active-Passive HA deployments, only a single server runs at any given moment (the active server) and another server operates in standby (the passive server). For this type of deployment, you install and upgrade two Cloud Connector servers in an active-passive deployment by having a shared mounted directory. In the event that the active server crashes, a procedure is executed to promote the passive server to active and vice versa.

To set up HA - Active Passive, set up two Linux machines using the (referred to as sk4-ha-active and sk4-ha-passive). After you set up the servers, refer to the following workflows for continued management as needed:

Prepare your Servers for High Availability for Active Passive Mode

  1. Create a shared folder of size 512GB and mount it in both sk4-ha-active and sk4-ha-passive.

    Without loss of generality, we assume the shared folder is mapped to /active/sk4-shared and /passive/sk4-shared in sk4-ha-active and sk4-ha-passive respectively

  2. In both sk4-ha-active and sk4-ha-passive create the root directory /opt/exabeam/data/sk4.

    mkdir -p /opt/exabeam/data/sk4

  3. Use the mkdir command to create the following directories in the shared folder:

    • conf

    • kafka_conf

    • kafka_data

    • pg_conf

    • pg_data

    • zoo_data

    • zoo_datalog

    From sk4-ha-active it can be done by running the following bash commands:

    mkdir /active/sk4-shared/conf
mkdir /active/sk4-shared/kafka_conf
mkdir /active/sk4-shared/kafka_data
mkdir /active/sk4-shared/pg_conf
mkdir /active/sk4-shared/pg_data
mkdir /active/sk4-shared/zoo_data
mkdir /active/sk4-shared/zoo_datalog

  4. Create soft links in sk4-ha-active for use by the Cloud Connector docker volumes:

    ln -s /active/sk4-shared/conf /opt/exabeam/data/sk4/conf
ln -s /active/sk4-shared/kafka_conf /opt/exabeam/data/sk4/kafka_conf
ln -s /active/sk4-shared/kafka_data /opt/exabeam/data/sk4/kafka_data
ln -s /active/sk4-shared/pg_conf /opt/exabeam/data/sk4/pg_conf
ln -s /active/sk4-shared/pg_data /opt/exabeam/data/sk4/pg_data
ln -s /active/sk4-shared/zoo_data /opt/exabeam/data/sk4/zoo_data
ln -s /active/sk4-shared/zoo_datalog /opt/exabeam/data/sk4/zoo_datalog

  5. Create soft links in sk4-ha-passive for use by the Cloud Connector docker volumes:

    ln -s /passive/sk4-shared/conf /opt/exabeam/data/sk4/conf
ln -s /passive/sk4-shared/kafka_conf /opt/exabeam/data/sk4/kafka_conf
ln -s /passive/sk4-shared/kafka_data /opt/exabeam/data/sk4/kafka_data
ln -s /passive/sk4-shared/pg_conf /opt/exabeam/data/sk4/pg_conf
ln -s /passive/sk4-shared/pg_data /opt/exabeam/data/sk4/pg_data
ln -s /passive/sk4-shared/zoo_data /opt/exabeam/data/sk4/zoo_data
ln -s /passive/sk4-shared/zoo_datalog /opt/exabeam/data/sk4/zoo_datalog

Install the Servers

  1. Install the active server:

    1. Deploy the Exabeam Cloud Connector Platform in sk4-ha-active with the Exabeam deployment flag (-d=exabeam).

    2. Stop the Cloud Connector server in sk4-ha-active.

      sudo systemctl stop sk4compose

    3. Copy the encryption key generated during the installation. Key can be retrieved by running the following bash command:

      grep SKYFORMATION_ENC_KEY /opt/exabeam/data/sk4/docker-compose.yml | head -1 | sed 's/.*SKYFORMATION_ENC_KEY=//'

  2. Install the passive server:

    1. Deploy the Exabeam Cloud Connector Platform in sk4-ha-passive with the Exabeam deployment flag (-d=exabeam).

    2. Stop the Cloud Connectors server in sk4-ha-passive.

      sudo systemctl stop sk4compose

    3. Disable the Cloud Connectors service (to prevent it from running following a machine reboot).

      sudo systemctl disable sk4compose

    4. Copy the encryption key generated during the installation. Key can be retrieved by running the following bash command:

      grep SKYFORMATION_ENC_KEY /opt/exabeam/data/sk4/docker-compose.yml | head -1 | sed 's/.*SKYFORMATION_ENC_KEY=//'

Use Same Encryption Key in Both Active and Passive Servers

  1. Override the encryption key in sk4-ha-passive with the encryption key copied from sk4-ha-active by running, in sk4-ha-passive, the following bash command (after replacing SKYFORMATION_ENC_KEY_FROM_PASSIVE and SKYFORMATION_ENC_KEY_FROM_ACTIVE).

    sed -i 's/SKYFORMATION_ENC_KEY_FROM_PASSIVE/SKYFORMATION_ENC_KEY_FROM_ACTIVE/g' /opt/exabeam/data/sk4/docker-compose.yml

  2. Start the Cloud Connectors server in sk4-ha-active:

    sudo systemctl start sk4compose

Upgrade Servers in HA

  1. Upgrade the Exabeam Cloud Connectors Platform in sk4-ha-active.

    Note

    Do not run the upgrade procedure on the passive server.

  2. Upgrade the passive server.

    In sk4-ha-passive run the following bash commands:

    sudo su -
curl -L https://download.skyformation.net/v2/download/installer/edge/compose-commons.sh > compose-commons.sh
source compose-commons.sh
docker_login REPLACE_WITH_YOUR_LICENSE_KEY
docker-compose -f /opt/exabeam/data/sk4/docker-compose.yml pull
download_docker_images

Switch Between Active and Passive

To manually swap the active and passive servers:

  1. Run the following commands to stop and disable the Cloud Connectors service on the active server.

    sudo systemctl stop sk4compose
sudo systemctl disable sk4compose

  2. Run the following commands to start and enable the Cloud Connectors service on the passive server.

    sudo systemctl start sk4compose
sudo systemctl enable sk4compose