Kubernetes setup

Prerequisites

1. Download the latest version of the Snowflake Profiler from the Delphix download page.

2. Ensure that the host running the profiler has Helm, kubectl and Docker installed. Also ensure it is set up to communicate to the Kubernetes cluster using either MicroK8s or Amazon’s EKS

3. Set up the Hyperscale Snowflake Connector and add the required ConnectorInfo details.

4. For accessing Snowflake warehouse, we will require a service account user with key pair authentication for enhanced authentication security as an alternative to basic authentication.

1. To configure key-pair authentication for a Snowflake service account, follow the steps provided in the Snowflake’s configuring key-pair authentication.

2. Use the following commands to generate base64 encoded value of encrypted private key and passphrase.

To generate base64 encoded value of encrypted private key:

echo -n `cat /home/delphix/keys/MKK_TEST_SFHSC_ADMIN_key.p8 | base64

To generate base64 encode value of passphrase:

echo -n <passphrase> | base64 -w 0

5. Similarly, the profiler needs the Controller API Key to authenticate to the controller which also needs to be provided in base64 encoded format. To generate base64 encode value of Controller API key:

echo –n “<controller_api_key>” | base64 –w 0

Procedure

1. Untar the profiler downloaded from Delphix download page. It should contain the Docker images for the profiler and the snowflake-profiler-compose.tar

tar -xf snowflake-profler.tar.gz

2. Load the delphix-snowflake-profiler.tar Docker image:

cd snowflake-profiler
docker load --input delphix-snowflake-profiler.tar

3. Push the image to a registry which can be accessed by the Kubernetes cluster.

docker tag delphix-snowflake-profiler-service-app:1.0.0 image.registry.com/delphix-snowflake-profiler-service-app:1.0.0
docker push image.registry.com/delphix-snowflake-profiler-service-app:1.0.0
(‘docker login <registry>' may be required)

4. Untar the helm repository and change the directory to snowflake-profiler-helm

tar -xf snowflake-profiler-helm.tar.gz
cd snowflake-profiler-helm

5. Edit the values.yaml file

1. Configure the controllerURL with the controller URL, IP or hostname.

controllerURL: controller.delphix.com

2. If required, configure the registry credentials.
   

imageCredentials:
  username: <registry-username>
  password: <registry-password>
  email: <registry-user-email>

3. Configure the image details.

image:
  repository: image.registry.com/delphix-snowflake-profiler-service-app
  tag: 1.0.0
  pullPolicy: Always

4. Configuring the credentials required to connect to Snowflake instance, similar to how it is configured for the Hyperscale Snowflake connector. If the Kubernetes secret is already configured in the same namespace (.Values.namespace), you can use the same.

secret:
  storedSnowflakeSecretName: stored-secret-name

Or you can configure the required values for snowflakePrivateKey, snowflakePassphrase and controllerApiKey

secret:
  snowflakePrivateKey: user_encrypted_private_key_base64_encoded
  snowflakePassphrase: passhrase_base64_encoded
  controllerApiKey: controller_api_key_base64_encoded

5. Optionally, see additional configurations for further details.

6. Start the snowflake profiler service.

helm install snowflake-profiler-helm <path_to_snowflake_profiler_helm_chart> -f values.yaml

7. Create the Kubernetes Ingress to allow connection to the profiler.

Microk8s

For MicroK8s, create an ingress.yaml with the following values ensuring the namespace matches with profiler namespace:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hyperscale-snowflake-profiler-ingress
  namespace: hyperscale-snowflake-profiler-service
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
spec:
  ingressClassName: nginx
  rules:
http:
        paths:
path: /profiler
          pathType: Prefix
          backend:
            service:
              name: snowflake-profiler-k8s-service
              port:
                 number: 8080

Amazon AWS EKS

For Amazon AWS EKS, ensure the namespace matches with profiler namespace:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hyperscale-snowflake-profiler-ingress
  namespace: hyperscale-snowflake-profiler-service
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":8080}]'
    alb.ingress.kubernetes.io/backend-protocol: HTTP
spec:
  rules:
http:
       paths:
path: /profiler
          pathType: Prefix
          backend:
            service:
              name: snowflake-profiler-k8s-service
              port:
                 number: 8080

8. Apply the Ingress configuration:

kubectl apply –f ingress.yaml

9. Access the profiler Swagger UI at http://<host-ip>/profiler