Cloud ConnectorsSetup

Table of Contents

Deployment

Deploy the Exabeam Cloud Connector Platform

Prerequisites to Install Exabeam Cloud Connectors

Before you install the Exabeam Cloud Connectors you must complete the following prerequisites:

  • Ensure that you have the required hardware and software components or libraries to run the Exabeam Cloud Connectors on a dedicated virtual machine (VM). For example, installation of the required Operating System (OS) and Java Rum-time.

  • Enable network security and connectivity. For example, HTTPS from the required machine to update.skyformation.net.

  • Obtain the required information such as credentials, authorization settings, and licenses of the cloud application, to add the Exabeam Cloud Connectors.

The following table displays the details of prerequisites to deploy Exabeam Cloud Connectors.

Category

Type

Details

Hardware

CPU

2 Core minimum (4 recommended)

Software

RAM

16GB minimum

Disk space

512GB minimum

Mounted to : /opt/exabeam/data/sk4 in case cloud connectors is installed on an Exabeam server.

/var/lib/docker/volumes in other cases

In both cases, make sure that /var/lib/docker has at least 20GB mounted to it for miscellaneous docker data.

Operating systems

Ubuntu 16.04+, RHEL/ CentOS/ 7.x

Network security rules

SSL (port 443) Outbound from Exabeam cloud connectors machine to the Exabeam Cloud Connectors cloud services:

For troubleshooting capabilities, open:

Exabeam cloud connectors service allows the downloading of the software, deployment scripts, updates, and more.

Network security software is used for the Exabeam Cloud Connectors license activation. 

SSL (port 443 and 8443) Outbound from Exabeam cloud connectors machine to any internet address for the duration of the install only

This is needed for the Exabeam Cloud Connectors discover and cloud apps connectors modules.

CentOS only

HTTP (port 80) Outbound from the Exabeam cloud connectors application machine to the Yum package manager URL

If HTTP outbound is not permitted a reverse proxy could be used to proxy the Yum package manager requests over SSL. 

Please see:

Configure CentOS package manager Yum to use reverse proxy

Note

The Exabeam Cloud Connectors reverse proxy is only available if coordinated with the support team in advance.

Exabeam Cloud Connectors auto-install uses the Linux package manager to download the needed Linux packages for it to run. Once done this rule can be removed. 

For installation or upgrade:

HTTP and HTTPS outbound from the Exabeam cloud connector machine to the Docker registry

Exabeam Cloud Connectors installation/upgrade process uses these domains to retrieve and install the docker app on the machine (if outdated or missing) and the Exabeam cloud connectors application docker images from its registry. 

An alternative is to have a docker and a docker-compose installed on the host.

Applicative

Maximum number of open files must be 1000000.

See Increasing the maximum number of open files

Install Exabeam Cloud Connectors

To install the Exabeam Cloud Connectors app:

  1. SSH into the Linux system with root or an account that has sudo-level privileges.

  2. Install the latest version or a specific version of the Exabeam cloud connectors application with or without a proxy server.

    Note

    The command for installation uses ‘screen’ to run the installation script in a separate process in such a way that if the session gets disconnected, the installation progress does not get interrupted. You must install screen on the host system. If you cannot install screen, delete the word screen from the commands that you run. Any disruption to the installation session results in corrupt installation. You must uninstall the application before reattempting the installation process.

  3. If you want to install the Exabeam cloud connectors application without a proxy sever:

    1. To install the latest version of the Exabeam cloud connectors application with no proxy server run:

      sudo screen bash -c "$(curl -L
      https://download.skyformation.net/v2/download/installer/edge/pre-install.sh)"
    2. To install a specific version of the Exabeam cloud connectors application without a proxy server, use -v=*requested version* parameter to run the installation with a specific software version when you run the following command:

      curl -L https://download.skyformation.net/v2/download/installer/edge/pre-install.sh > preinstall.sh && chmod +x preinstall.sh && sudo screen ./preinstall.sh -v=*version* && rm preinstall.sh
  4. If you want to install the Exabeam cloud connectors application with a proxy sever:

    1. To install the latest version of the Exabeam cloud connectors application with a proxy server run:

      sudo screen bash -c "$(curl -x https://proxyserver:8080 -L https://download.skyformation.net/v2/download/installer/edge/pre-install.sh)"
    2. To install a specific version of the Exabeam Cloud Connectors application with a proxy server, use the -v=*requested version* parameter to run the installation with a specific software version in the following command:

      curl -x https://proxyserver:8080 -L https://download.skyformation.net/v2/download/installer/edge/pre-install.sh > preinstall.sh && chmod +x preinstall.sh && sudo screen ./preinstall.sh -v=*version* && rm preinstall.sh

      Note

      If you have an authenticated proxy server, and special characters in your username or password, you must send the username or password in URL encoded format to the curl -x parameter and to the install script. Use the following command line on your terminal to see the encoded format of your user and password.

      curl -s -o /dev/null -w %{url_effective} --get --data-urlencode 'ab$#^%cd' "" | cut -c 3-

      In this example, the password is ab$#^%cd which can be replaced by your password. The result SSH into the Linux system with root or an account that has sudo-level privileges. provides you with an encoded password which in this case is: ab%24%23%5E%25cd

Set up an Integration User

You can set up an integration user for Exabeam Cloud Connectors application open-API usage during the installation process.

To set up an integration user:

  1. Use the following optional parameters:

    • -u=inetgration-user-name

    • -p=integration-user-password

    • -r=integration-user-role

      Note

      The only valid value is ‘integration-admin’. If you are setting a regular, non-admin integration user, do not use this parameter.

  2. Run the following installation commands based on your requirement.

    1. If you want to install the Exabeam Cloud Connectors application without a proxy sever run:

      curl -L https://download.skyformation.net/v2/download/installer/edge/pre-install.sh > preinstall.sh && chmod +x preinstall.sh && sudo ./preinstall.sh -u=myintegrationuser -p=myintegerationuserpassword -r=integration-admin && rm preinstall.sh
    2. If you want to install the Exabeam Cloud Connectors application with a proxy sever run:

      curl -L https://download.skyformation.net/v2/download/installer/edge/pre-install.sh > preinstall.sh && chmod +x preinstall.sh && sudo ./preinstall.sh -u=myintegrationuser -p=myintegerationuserpassword -r=integration-admin && rm preinstall.sh

      Note

      Apply encoding for the special characters in the user name and password.

  3. Insert the following parameters when prompted:

    image2.png

    Existing Text

    1. Proxy server [] :

      • If you have no proxy The default is no proxy in use. Press enter if no proxy is in use. 

      • If you are running behind a proxy: insert the proxy DNS and port (e.g. proxyserver:8080). If the proxy requires authentication, also enter your credentials i.e. https://myuser:mypwd@myproxy.corp.com:8080)

        Again, if you have special characters in the user or the password or both, you have to send the encoded strings here.

    2. License key: Insert your Exabeam Cloud Connectors app license key.

    3. Application port [8443]: Port used by the Exabeam Cloud Connectors web application. Enter for default 8443.

    4. You should see the two lines (as shown in the figure):

      >> Testing license key
      >> License key is valid
      image3.png

Migrate an Existing Deployment to Another Server

When migrating from a source system to a target system, both systems must be on the same software version of Exabeam Cloud Connectors.

Versions 2.4.x of the Exabeam Cloud Connectors platform use zookeeper for configuration management while versions 2.5.x use etcd. It is highly recommended that you upgrade your existing systems to the latest Exabeam Cloud Connectors platform version (2.5.x) using etcd before proceeding with the migration. If you need to remain on your current version, you may do so, however you will need to contact support to get the installation command to your specific version.

If you wish to upgrade your source system before continuing to do the migration, see Upgrade the Exabeam Cloud Connectors Platform.

There are two ways to migrate the Exabeam Cloud Connectors platform: manually or automatic. In both of the sections, there are steps to migrate configurations from one Exabeam Cloud Connectors platform instance (the source) to another (the target). The migration must be done between instances with exactly the same code version. Choose the preferred method.

Automate the Migration of Exabeam Cloud Connectors Platform to a Target System

  1. In the source system, please run the following command:

     sudo bash -c "$(curl -L https://download.skyformation.net/v2/download/installer/edge/migration_export.sh)" $@ 2>&1 | tee "migration-export-at-`date --iso-8601=ns`.log"
  2. When prompted, enter a path where your configurations will be saved. Please make sure it is a non-existing directory.

    When the process is finished, you should see the message: ==> Done export! The backup dir is in:<Your backup directory>.

  3. Copy the directory created in step 1, from the source system to the target system (using scp or rsync for example)

  4. In the target system, please run the following command:

    sudo bash -c "$(curl -L https://download.skyformation.net/v2/download/installer/edge/migration_import.sh)" $@ 2>&1 | tee "migration-import-at-`date --iso-8601=ns`.log"
  5. When prompted, enter the path to the directory that was copied in step 2.

    When the process is finished, you should see the messages: Services are running successfully and then ==> Done import!

Manually Migrate the Exabeam Cloud Connectors Platform to a Target System

Please apply instructions that matches your migration path.

  1. On the source system, go to the Cloud Connectors UI , navigate to Settings > Accounts and stop all the active accounts one by one. Then, wait at least 5 minutes to let each account stop gracefully. Note of the accounts that were active.

  2. Remove any old utilities image from the system. On the source system terminal, run the following command:

    sudo docker image rm -f skyformation/troubleshoot-utils 

    If the image is not present you will get the following error :

    Error: No such image: skyformation/troubleshoot-utils

    This is okay. You can ignore this error.

  3. Export source settings:

    On the source host, run the following based on your migration path:

    • If your systems (source and target) run Exabeam Cloud Connectors platform 2.5.x, apply the following command to send configurations into a sk4etcdkbackup- folder (the folder will be created in the same directory where you ran the command):

       sudo docker run --rm --network=sk4_isolated_nw -v "$PWD/sk4etcdbackup-$(date +%Y-%m-%d-%H-%M-%S)":/local skyformation/troubleshoot-utils:latest etcd-export /local 
    • To continue using Exabeam Cloud Connectors platform 2.4.x, run the following command to send configurations into a sk4zookbackup-<date> folder (the folder will be created in the same directory where you ran the command):

       sudo docker run --rm --network=sk4_isolated_nw -v "$PWD/sk4zookbackup-$(date +%Y-%m-%d-%H-%M-%S)":/local skyformation/troubleshoot-utils:latest zookeeper-export /local
  4. (Version 2.5.x only) Install Exabeam Cloud Connectors platform on the target host with the same Exabeam Cloud Connectors platform version as the source. To confirm the source version, in the Exabeam Cloud Connectors platform UI go to Settings > General. To install the Exabeam Cloud Connectors platform with a specific software version, see Install Exabeam Cloud Connectors. To install version 2.4 on your target host, please contact support to get the command to install old version.

  5. Stop the SK4 service on both the source and the target:

    sudo systemctl stop sk4compose
  6. Override the encryption key in the target with the one from the source:

    1. Find the location of the docker-compose.yml file on the source and target systems and if an override file exists (source only)

      1. On the terminal of each system, type:

        cat /etc/systemd/system/sk4compose.service | grep ExecStart 
      2. In the output line, the compose file path shows after the -f. If more than one file appears as a parameter -f, the first one is the compose file and the second one is the override file.

        An example result:

        ExecStart=/usr/local/bin/docker-compose -f /opt/sk4/docker-compose.yml up
        
        The docker-compose.yml file full path is /opt/sk4/docker-compose.yml

        Note

        The location of the docker-compose.yml file may be different between the source and the target. This is okay. You should not change the location of the file on any system.

    2. Find the SKYFORMATION_ENC_KEY string from the source and target machines. Run the following:

      grep SKYFORMATION_ENC_KEY <compose file full path> | uniq | cut -f2- -d=
    3. Replace the SKYFORMATION_ENC_KEY env variable in the target machine (using the key from the source machine). On the target machine run the following:

      sudo sed -i 's/<CURRENT_KEY_IN_TRG>/<OLD_KEY_FROM_SOURCE>/g' <target system compose file full path>

      Repeat the command from to get the SKYFORMATION_ENC_KEY on the target to make sure it was successfully replaced.

    4. If an override file exists on the source machine, copy it to the directory where the compose resides on the target system and edit the file /etc/systemd/system/sk4compose.service to add the additional -f flag with the override file, similar to what was in the source system.

  7. Copy the contents of the three docker volumes (sk4_conf, sk4_pg_data, sk4_pg_conf) from the source system to the target system:

    1. First, find out the location of these docker volumes on the source and target systems – it is not necessarily in the same path! modify only the content, do not change the path/location of the volume on the target system.

      To find the path, type (source and target systems):

      sudo docker volume inspect sk4_conf

      The path is listed under the Mountpoint entry. The three volumes are all located under the same root path.

    2. For each of the three volumes, copy the *contents* of the directory from the source system into their corresponding locations on the target system (using scp or rsync for example).

    3. Check file permissions on YML and SH files in the sk4_conf directory on the target:

      • For the SH file, the execute flag should be enabled. If not, run chmod +x *.sh to configure file permissions.

      • For the YML files, read+write access is required for the user account that runs the Exabeam Cloud Connectors platform.

  8. On the target system, start the configuration service based on your installation version.

    • If the target is running the Exabeam Cloud Connectors platform 2.5.x (or later) run:

      sudo /usr/local/bin/docker-compose -f <docker compose yml full path> up -d sk4etcd 
    • If the target is running the Exabeam Cloud Connectors platform 2.4.x :

      sudo /usr/local/bin/docker-compose -f <docker compose yml full path>  up -d zookeeper 
  9. Import configurations into the target, based on your installation:

    1. Copy the sk4etcdkbackup-<date> or sk4zookbackup-<date> folder generated in Step 3 from the source system to the local home folder ($PWD) on the target.

    2. Run the command for the your migration path:

      • If the target is on Exabeam Cloud Connectors platform 2.5.x, import the etcd contents using the sk4etcdkbackup-<date> folder:

        sudo docker run --rm --network=sk4_isolated_nw -v "$PWD/sk4etcdbackup-<date that was created during the import>":/local skyformation/troubleshoot-utils:latest etcd-import /local 

        Run the following command to ensure data has been imported successfully. You should see entries (more than one) for the imported data, specifically there should be an entry for each account that was imported:

         sudo docker container exec -it sk4etcd etcdctl get "" --prefix=true --keys-only
      • If the target is on Exabeam Cloud Connectors platform 2.4.x, import the zookeeper contents using the sk4etcdkbackup- folder:

        sudo docker run --rm --network=sk4_isolated_nw -v "$PWD/sk4zookbackup-<date> ":/local skyformation/troubleshoot-utils:latest zookeeper-import /local
  10. Start the Exabeam Cloud Connectors platform on the target:

    sudo systemctl start sk4compose
  11. On the Exabeam Cloud Connectors UI on the target, start all the accounts that were active on the source.

  12. After verifying data is flowing on the target, stop and disable Exabeam Cloud Connectors on the source:

    sudo systemctl stop sk4compose
    sudo systemctl disable sk4compose 

Upgrade the Exabeam Cloud Connectors Platform

This workflow applies to Exabeam Cloud Connectors deployments on Linux systems.

Before starting the upgrade process, a backup for your configurations will occur automatically.

You will be asked to enter a full path to a directory that does not already exist where the configurations will be saved.

If the upgrade fails at one of the stages, it will automatically revert your system to its initial state, using the backup directory. In this case, you will receive the following message at the end of the rollback: The system has been reverted to its original state. Please open a support call and attach the update log saved in sk4-update-at-<current-date>

You can use the following flags when running the upgrade:

  • -b or --backup_dir ­­– Backup directory for the configurations before the upgrade.

  • --nobackup – Skip the backup configuration process.

    Note

    If you use this flag and the upgrade fails, roll back to the original state will not be possible automatically nor manually.

To upgrade the Exabeam Cloud Connectors app:

  1. Before you begin, you must have previously installed the Exabeam Cloud Connectors platform on your Linux system. See Install Exabeam Cloud Connectors.

  2. Run software updates.

    Note

    The command uses screen to run the script in a separate process such that if the session disconnects, it will not disrupt the script's progress. If screen in not installed on the host, it is recommended to install it before proceeding further. If it is not possible to install screen, remove the word screen from the command; keep in mind that any disruption to the session will result in corrupt installation, and a manual restoration to a working state will be required.

    1. SSH into the Linux system with root or an account that has sudo-level privileges.

      Without proxy

      sudo screen bash -c "$(curl -L https://download.skyformation.net/v2/download/installer/edge/pre-update.sh)

      With a proxy

      sudo screen bash -c "$(curl -x https://proxyserver:8080 -L https://download.skyformation.net/v2/download/installer/edge/pre-update.sh)

      Note

      If you have an authenticated proxy and you have special characters in your username or password, you need to send them the curl -x parameter in url-encoded format.

      curl -s -o /dev/null -w %{url_effective} --get --data-urlencode'ab$#^%cd'""| cut -c 3

      The result on the screen is the encoded password, in this case: ab%24%23%5E%25cd.

      In case you're having trouble logging into our docker repository, see Log in to the Docker Repository.

    2. Configure access to verify the web app is accessible.

      Open your Chrome browser and enter the URL for your web app using the following format: https://<exabeamserver>:8443

    3. Approve the certificate for the web app.

      skyformation_webapp_access_untrusted_cert_approval.png

      Select the Advanced link and then in the expanded section, select the Proceed to link.

    4. Enter your user name and password on the login page of the Exabeam Cloud Connectors platform.

      skyformation_login_page.png

Note

If the upgrade fails or you need to revert the system to its original state, you can revert the Exabeam Cloud Connector platform to the previous state.

Uninstall the Exabeam Cloud Connectors Platform

Use this workflow to uninstall Exabeam Cloud Connectors 2.1.21 or a later release. If you are using an older version of Exabeam Cloud Connectors, please upgrade to the most recent version.

Warning

The uninstall process removes both the Exabeam software and the Exabeam data from your machine. This is an irreversible process.

The Docker and Docker compose are not uninstalled by this process.

The command uses screen to run the script in a separate process, such that if the session disconnects it will not disrupt the script's progress. If screen in not installed on the host, it is recommended that it is installed beforehand. If it is not possible to install screen, remove the word screen from the command; keep in mind that any disruption to the session will result in corrupt installation, and an uninstall will be required before reattempting installation.

  1. SSH into the Linux machine that hosts the Exabeam Cloud Connectors app with an account that has root-level permissions.

  2. Run the command that is relevant to your deployment.

    Without proxy

    sudo screen bash -c "$(curl -L https://download.skyformation.net/v2/download/installer/edge/pre-uninstall.sh)"

    With a proxy

    sudo screen bash -c "$(curl -x https://proxyserver:8080 -L https://download.skyformation.net/v2/download/installer/edge/pre-uninstall.sh)"

    Note

    If you use an authenticated proxy, and you have special characters in your username or password, you need to send them the curl -x parameter in url-encoded format. If needed, use the following command line on your terminal to view the encoded format of your user and password (in this example the password is ab$#^%cd, replace it with yours):

    curl -s -o /dev/null -w %{url_effective} --get --data-urlencode 'ab$#^%cd' "" | cut -c 3-

    In this example, the encoded password that would be displayed on screen is ab%24%23%5E%25cd.