Installation and setup for Kubernetes

Before getting started, Delphix recommends engaging your Kubernetes Admin for deployment guidance on available Kubernetes deployment platforms (AKS, EKS, etc.). Configurations like node sizing and persistent volume settings will need to be determined.

Hardware requirements

The hardware requirements for Data Control Tower (DCT) on Kubernetes are listed below. In addition to these requirements, inbound port 443 must be open for API clients, and outbound port 443 to engines. This is the minimum total resource request for the Kubernetes deployment of DCT. Individual service-level resource requests are contained in the values.yaml file and can be overridden during deployment.

• CPU: 4-Core

• CPU architecture: x86_64

• Memory: 16GB

• Storage: 50GB

• Port: 443

The recommended minimum 50 GB of storage is shared across the Kubernetes cluster (i.e. hosts). All pods and/or services use this storage for mounted volumes and other utilities, including image storage.

In a single node cluster, if shared volumes are not externalized, the host requires the full 50 GB of storage. If the persistent volume is mounted externally, the host requires 37 GB of storage, since the default storage required by the database (10 GB), gateway (1 GB), masking (1 GB), and virtualization-app (1 GB) draws from the external storage. The default storage configuration for the database, gateway, masking, and virtualization-app can be modified in values.yaml.

Many users may have default container settings as part of their Kubernetes or OpenShift infrastructure management. It is paramount to compare those default settings with the recommended minimum performance specifications (please engage your container infrastructure team to verify). If those default setting are lower, please update them to the minimum or higher.

For users who need to have limits set, you can start with the following sample configuration.

dataBookmarks:
  resources:
    requests:
      memory: "256Mi"      cpu: "100m"    limits:
      memory: "512Mi"      cpu: "300m"dataLibrary:
  resources:
    requests:
      memory: "256Mi"      cpu: "100m"    limits:
      memory: "1024Mi"      cpu: "800m"database:
  resources:
    requests:
      memory: "256Mi"      cpu: "200m"    limits:
      memory: "4096Mi"      cpu: "1100m"gateway:
  resources:
    requests:
      memory: "512Mi"      cpu: "400m"    limits:
      memory: "2048Mi"      cpu: "400m"graphql:
  resources:
    requests:
      memory: "256Mi"      cpu: "100m"    limits:
      memory: "1024Mi"      cpu: "200m"ui:
  resources:
    requests:
      memory: "128Mi"      cpu: "100m"    limits:
      memory: "256Mi"      cpu: "200m"jobs:
  resources:
    requests:
      memory: "256Mi"      cpu: "200m"    limits:
      memory: "1024Mi"      cpu: "400m"jobsCleaner:
  resources:
    requests:
      memory: "256Mi"      cpu: "200m"    limits:
      memory: "512Mi"      cpu: "400m"masking:
  resources:
    requests:
      memory: "512Mi"      cpu: "500m"    limits:
      memory: "2304Mi"      cpu: "700m"virtualizationApp:
  resources:
    requests:
      memory: "512Mi"      cpu: "600m"    limits:
      memory: "2560Mi"      cpu: "700m"virtualization:
  resources:
    requests:
      memory: "256Mi"      cpu: "100m"    limits:
      memory: "512Mi"      cpu: "300m"

Overview

Data Control Tower can be deployed in a matter of minutes, once a Kubernetes cluster has been identified and deployment details have been aligned with your Kubernetes administrator. The installation consists of three components:

• Kubernetes cluster: The identified infrastructure to which DCT will be deployed.

• HELM: This deploys DCT as a Kubernetes application by referencing HELM charts (.yaml files) that make up the DCT install, either by an external Helm repository (https://dlpx-helm-dct.s3.amazonaws.com, this is the quickest path to installing DCT, as it largely automated) or via local install (this is accomplished by downloading the Helm charts directly via the DCT .tar file on download.delphix.com).

• kubectl: Is a command line tool that enables administrative communication with the deployed pods (most useful post-deployment or after an upgrade).

Installation requirements

DCT requires a running Kubernetes cluster; This could be an on-premises cluster, Azure AKS or AWS EKS cluster. DCT also requires a kubectl command line tool to interact with Kubernetes cluster and HELM for deployment on to the cluster.

Installing DCT from external HELM repository

Network requirements

The machine on which them helm commands will be executed must be able to connect to https://dlpx-helm-dct.s3.amazonaws.com (TCP on port 443). Otherwise, see the section for Offline mode below.

Process

1. Add the DCT HELM repo with the following, which will link the DCT HELM repo to the local client HELM repo:

   Copy

   helm repo add dct-services https://dlpx-helm-dct.s3.amazonaws.com

2. Update the added repos with the following, which will update the local HELM charts metadata (index.yaml).

   Copy

   helm repo update

3. Pull the HELM charts with the following command, which is used to download the .tgz file.

   Copy

   helm pull dct-services/delphix-dct --version x.0.0

4. The downloaded file is then extracted using the following command (where x.0.0 should be changed to the version of DCT being installed):

   Copy

   tar -xvf delphix-dct-x.0.0.tgz

5. Update the following properties in values.yaml which is present in the extracted folder delphix-dct.

1. To generate the bootstrap APIKey, set apiKeyCreate: true.

2. Provide image credentials to pull images from docker registry.

1. username: <retrieved from download.delphix.com>

2. password: <retrieved from download.delphix.com>

To get the image credentials, visit the Delphix DCT Download page and login with your customer login credentials. Once logged in, select the DCT Helm Repository link and accept the Terms and Conditions. Use password from the popup screen, as shown below.

Offline mode

Offline mode is supported for the following scenarios:

• Clusters that are air-gapped and cannot access the Delphix Docker registry.

• Instances where Delphix Docker images must be pushed to an organization's internal registry for security scans or to comply with other security policies.

In which case, a jumpbox machine with access to the Delphix Docker registry (or the download site and the organization's internal registry) is required.

1. Get the Docker images one of two ways.

1a. Download and load the images from a tarball.

• This requires a jumpbox machine with access to https://dct.download.delphix.com and https://prod-us-west-2-starport-layer-bucket.s3.us-west-2.amazonaws.com (TCP on port 443). If a firewall is blocking access to either address then refer to option a.

  • Login to the jumpbox machine.

  • Download the HELM chart and Docker image packages as a delphix-dct-x.0.0.tar.gz file, which is available on the download site.

  • Extract the downloaded tar.gz file to get the images.

  • Load the extracted Docker images with:
    sudo docker load --input $image; done

  • Re-tag all above Docker images and all remaining DCT images, as an example for one image:
    $ docker tag registry.delphix.com/delphix-dct:nginx-x.0.0 <local registry url>:nginx-x.0.0
    

  • Push re-tagged images into the internal registry., as an example for one image: $ docker push <local registry url>:nginx-x.0.0

1b. Retrieve the images from the Delphix Docker registry.

• This requires a jumpbox machine with access to https://dct.download.delphix.com and https://prod-us-west-2-starport-layer-bucket.s3.us-west-2.amazonaws.com (TCP on port 443). If a firewall is blocking access to either address then refer to option 1a.

  • Login to the jumpbox machine.

  • Pull the Docker images from the Delphix Docker registry.
    $ docker pull dct.download.delphix.com/delphix-dct:nginx-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:app-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:data-bookmarks-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:delphix-data-library-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:graphql-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:ui-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:jobs-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:postgres-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:virtualization-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:masking-x.0.0

    $ docker pull dct.download.delphix.com/delphix-dct:virtualization-app-x.0.0

  • Re-tag all above Docker images and all remaining DCT images, as an example for one image: $ docker tag dct.download.delphix.com/delphix-dct:nginx-x.0.0 <local registry url>:nginx-x.0.0

  • Push re-tagged images into the internal registry., as an example for one image: $ docker push <local registry url>:nginx-x.0.0

2. Deploy

• Update the values.yaml file with the registry name and image credentials:

  • Registry: <local registry url>

  • Username: <local registry username>

  • Password: <local registry password>

• Install using the instructions in the following sections.

Deployment status

Once deployment is complete, check the status of the deployment using the following command (where X.0.0 should be changed to the version of DCT being installed):

helm list
NAME              NAMESPACE    REVISION    UPDATED                                 STATUS      CHART                    APP VERSION
dct-services      default      1           2023-01-10 19:33:41.713202 -0900        deployed    delphix-dct-x.0.0        x.0.0

Assuming an ingress controller configuration on the Kubernetes cluster is present, when accessing DCT after the deployment, the ingress controller rule needs to be added for proxy service, along with port 443 (if SSL is enabled) and port 80 (if SSL is disabled).