Upgrading the Hyperscale Compliance Orchestrator (Docker Compose)

Prerequisite

Before upgrading, ensure you have downloaded the Hyperscale Compliance x.0.0 (where x.0.0 should be changed to the version of Hyperscale being installed) tar bundle from the Delphix Download website.

Limitation

This release does not support using an external PostgreSQL instance as the Hyperscale database.

How to upgrade the Hyperscale Compliance Orchestrator

Perform the following steps to upgrade the Hyperscale Compliance Orchestrator to the x.0.0 version:

1. Run cd /<hyperscale_installation_path>/ and docker-compose down to stop and remove all the running containers.

2. Run the below commands to delete all existing dangling images and hyperscale images:

   Copy

   docker rmi $(docker images -f "dangling=true" -q)
   docker rmi $(docker images "delphix-hyperscale-masking-proxy" -q)
   docker rmi $(docker images "delphix-controller-service-app" -q)
   docker rmi $(docker images "delphix-masking-service-app" -q)
   docker rmi $(docker images "delphix-*load-service-app" -q)

3. Remove all files or folders from the existing installation directory, except docker-compose.yaml and the .env file . Keep their backup outside the installation directory.

4. Untar the new tar in your existing installation path. Replace x.0.0 in the path with the version number of the Hyperscale release you are installing:
   tar -xzvf delphix-hyperscale-masking-x.0.0.tar.gz -C <existing_installation_path>

5. Replace the docker-compose.yaml supplied with the bundle file as per the following:

   • For users upgrading from 3.0.0.x: Use the connector-specific docker-compose-sample.yaml file for example, docker-compose-oracle.yaml or docker-compose-mssql.yaml supplied with the bundle and add the same ‘volumes’ and any other properties, if configured, for each container referencing the backed-up docker-compose.yaml from step 3.

   • For users upgrading from 4.0.0.0 and above: Copy over the differing bits (essentially the volumes and properties for each service) from the backed-up docker-compose.yaml file to the new docker-compose.yaml file supplied in the bundle.

   • Ensure that the volume binding for the staging area path is added to all services except the proxy service. For example:

     Copy

     - /mnt/hyperscale:/etc/hyperscale  

6. Apply the backed up .env file and set the VERSION property as x.0.0 (i.e. VERSION=x.0.0).

7. Run the below commands to load the images(will configure Oracle-based unload/load setup):

   Copy

   docker load --input controller-service.tar
   docker load --input unload-service.tar
   docker load --input masking-service.tar
   docker load --input load-service.tar
   docker load --input proxy.tar
   docker load --input database-service.tar

   • If upgrading from an MSSQL connector setup(supported starting 5.0.0.0 release), instead of running the above commands for load/unload services setup(which are for Oracle), run the below commands(rest remains same for the controller, masking, and proxy services):

     Copy

     docker load --input mssql-unload-service.tar 
     docker load --input mssql-load-service.tar

   • If upgrading from a Delimited/Parquet Files connector setup (supported starting 12.0.0 release), instead of running the above commands for load/unload services setup(which are for Oracle), run the below commands(rest remains same for the controller, masking, and proxy services) after updating new image names in docker-compose.yaml:

     Copy

     docker load --input file-connector-unload-service.tar
     docker load --input file-connector-load-service.tar

   • If upgrading from a MongoDB connector setup (supported starting 13.0.0 release), instead of running the above commands for load/unload services setup (which are for Oracle), run the below commands (rest remains same for the controller, masking, and proxy services):

     Copy

     docker load --input mongo-unload-service.tar
     docker load --input mongo-load-service.tar

     • If upgrading from a Snowflake connector setup (supported starting 28.0.0 release), instead of running the above commands for load/unload services setup (which are for Oracle), run the below commands (rest remains same for the controller, masking, and proxy services):

     Copy

     docker load --input snowflake-unload-service.tar
     docker load --input snowflake-load-service.tar

8. Make sure to have the below ports configured under proxy service:

   Copy

   ports:
       - "443:8443"
       - "80:8080"

    

9. Ensure your mount are configured and accessible, before running a job.
   If upgrading to version 24.0 (and onwards) , ensure that the location mounted on the Hyperscale host is the same as the one mapped to /etc/hyperscale in your docker-compose.yaml. If a previous mount exists on another location, unmount it and re-mount to the right directory e.g. if a mount point exists with name staging_area at path /mnt/provision/staging_area , execute the following commands and restart the containers.

   1. If NFS file server is used as staging server execute this command:

      Copy

      sudo umount /mnt/provision/staging_area
      sudo mount -t nfs4 <source_nfs_endpoint>:<source_nfs_location> /mnt/provision

   2. If the NFS Server installation is a Delphix Continuous Data Engine empty VDB you can either:

      1. Append the staging_area path to the volume binding for all the services in docker-compose.yaml. For example:

         Copy

         volumes:
               - /mnt/provision/staging_area:/etc/hyperscale

      2. Alternatively, you can update mount path of Environment on Continuous Data Engine:

         1. Disable the empty VDB(Data Set).

         2. Update path on Environment → Databases page. For example: change path from /mnt/provision/staging_area to mnt/provision/

         3. Enable the empty VDB(Data Set).

         4. Restart Hyperscale containers.

After re-mounting, recheck the permissions of staging area on Hyperscale host. Refer to instruction number 3 on this page: Installation and Setup for required staging area permissions.

10. Run docker-compose up -d to create containers.