Kubernetes setup

Prerequisites

Before you begin, ensure the following requirements are met:

• Download the Snowflake Profiler
  Download the latest version of the Snowflake Profiler from the Delphix download page.
  

• Prepare the host environment
  Install and configure Helm, kubectl, and Docker on the host running the profiler. Ensure the host is configured to communicate with your Kubernetes cluster using either MicroK8s or Amazon EKS.

• Configure the Hyperscale Snowflake connector
  Set up the Hyperscale Snowflake connector and add the required ConnectorInfo details.

• Set up Snowflake key-pair authentication
  For enhanced security, Snowflake warehouse access requires a service account user configured with key-pair authentication instead of basic authentication. To configure key-pair authentication, refer to Snowflake’s configuring key-pair authentication.

  • Generate base64 encoded values - Linux users
    Use the following commands to generate base64 encoded values for the Snowflake private key, passphrase, and Controller API key. If you are a Windows user skip to the next section, PowerShell commands for Base64 encoding.

    • Private key: echo -n `cat /home/delphix/keys/MKK_TEST_SFHSC_ADMIN_key.p8 | base64

    • Passphrase: echo -n <passphrase> | base64 -w 0

    • Controller API key: echo –n “<controller_api_key>” | base64 –w 0

  • PowerShell commands for Base64 encoding - Windows users
    To generate Base64 encoded values for the privatekey and passphrase, open PowerShell and run the following commands:

    • To encode a plain-text value using UTF-8 encoding: [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes("<value>")
      Example: [Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes("SamplePassword@123"))

    • To encode the contents of a file such as private key, certificate, or .p8 file:
      [Convert]::ToBase64String([System.IO.File]::ReadAllBytes("C:\path\to\file"))
      Example: [Convert]::ToBase64String([System.IO.File]::ReadAllBytes("C:\keys\ADMIN_key.p8"))

Procedure

1. Untar the profiler downloaded from Delphix download page. It should contain the Docker images for the profiler and the snowflake-profiler-compose.tar

tar -xf snowflake-profler.tar.gz

2. Load the delphix-snowflake-profiler.tar Docker image:

cd snowflake-profiler
docker load --input delphix-snowflake-profiler.tar

3. Push the image to a registry which can be accessed by the Kubernetes cluster.

docker tag delphix-snowflake-profiler-service-app:1.0.0 image.registry.com/delphix-snowflake-profiler-service-app:1.0.0
docker push image.registry.com/delphix-snowflake-profiler-service-app:1.0.0
(‘docker login <registry>' may be required)

4. Untar the helm repository and change the directory to snowflake-profiler-helm

tar -xf snowflake-profiler-helm.tar.gz
cd snowflake-profiler-helm

5. Edit the values.yaml file

1. Configure the controllerURL with the controller URL, IP or hostname.

controllerURL: controller.delphix.com

2. If required, configure the registry credentials.
   

imageCredentials:
  username: <registry-username>
  password: <registry-password>
  email: <registry-user-email>

3. Configure the image details.

image:
  repository: image.registry.com/delphix-snowflake-profiler-service-app
  tag: 1.0.0
  pullPolicy: Always

4. Configuring the credentials required to connect to Snowflake instance, similar to how it is configured for the Hyperscale Snowflake connector. If the Kubernetes secret is already configured in the same namespace (.Values.namespace), you can use the same.

secret:
  storedSnowflakeSecretName: stored-secret-name

Or you can configure the required values for snowflakePrivateKey, snowflakePassphrase and controllerApiKey

secret:
  snowflakePrivateKey: user_encrypted_private_key_base64_encoded
  snowflakePassphrase: passhrase_base64_encoded
  controllerApiKey: controller_api_key_base64_encoded

5. Optionally, see additional configurations for further details.

6. Start the snowflake profiler service.

helm install snowflake-profiler-helm <path_to_snowflake_profiler_helm_chart> -f values.yaml

7. Create the Kubernetes Ingress to allow connection to the profiler.

Microk8s

For MicroK8s, create an ingress.yaml with the following values ensuring the namespace matches with profiler namespace:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hyperscale-snowflake-profiler-ingress
  namespace: hyperscale-snowflake-profiler-service
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
spec:
  ingressClassName: nginx
  rules:
http:
        paths:
path: /profiler
          pathType: Prefix
          backend:
            service:
              name: snowflake-profiler-k8s-service
              port:
                 number: 8080

Amazon AWS EKS

For Amazon AWS EKS, ensure the namespace matches with profiler namespace:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hyperscale-snowflake-profiler-ingress
  namespace: hyperscale-snowflake-profiler-service
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":8080}]'
    alb.ingress.kubernetes.io/backend-protocol: HTTP
spec:
  rules:
http:
       paths:
path: /profiler
          pathType: Prefix
          backend:
            service:
              name: snowflake-profiler-k8s-service
              port:
                 number: 8080

8. Apply the Ingress configuration:

kubectl apply –f ingress.yaml

9. Access the profiler Swagger UI at http://<host-ip>/profiler