Configuring Azure Blob containers as staging area

The Hyperscale Compliance Orchestrator requires a staging area to read from the source files and write to the target files. In Kubernetes deployment of Hyperscale Compliance, you can now use Azure Blob Containers as a staging area to access the Hyperscale Compliance Orchestrator and the Continuous Compliance engine’s source and target files.

This feature is supported on MicroK8s and Microsoft’s Azure Kubernetes Service (AKS) distributions and it is only available for Continuous Compliance engine version 28.0.0.0 onwards.

Prerequisites:

For Hyperscale Compliance containers to access Blob objects through a file system interface, a CSI driver (Container Storage Interface) must be configured on the Kubernetes cluster.

To configure the mount point through the CSI driver, a Kubernetes cluster administrator or an Azure account administrator will need to:

  1. Create an Azure storage account
    In your Azure portal, create a new storage account under the correct subscription and resource group.

  2. Create a Blob container
    Create a Blob container in the storage account using either the Azure Portal (UI) or the Azure CLI (az). This Blob container will act as the Hyperscale staging area.

  3. Configure access permissions
    Assign the appropriate Azure built‑in IAM roles (for example, Storage Blob Data Contributor) to the identity used by the Hyperscale cluster to access the Blob Storage account. Ensure the role is assigned to the correct user, service principal, or managed identity configured for the cluster. For more information, refer to Managed identities for Azure resources.

  4. Kubernetes Cluster
    Ensure a running Kubernetes cluster such as AKS or MicroK8s is available for deployment.

  5. Install the Azure Blob CSI Driver
    Install Azure Blob CSI Driver v1.25.1 or later on the cluster.

    • Installation for AKS:
      For installation instructions, refer to Azure Kubernetes Service. To install AKS you will need to enable CSI driver on a new or existing AKS cluster:

    Copy
    az aks update --enable-blob-driver --name myAKSCluster --resource-group myResourceGroup

    • Installation for locally managed Kubernetes:
      To install a locally managed Kubernetes such as MicroK8s you will need to install Azure Blob CSI driver:

Copy 
helm repo add blob-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/blob-csi-driver/master/charts
helm install blob-csi-driver blob-csi-driver/blob-csi-driver --set node.enableBlobfuseProxy=true --namespace kube-system --version 1.25.1

Setting Up Azure Blob Containers with Managed Identity (AKS)

This configuration is only required for Azure Kubernetes Service (AKS). Locally managed Kubernetes environments, such as MicroK8s, do not require this setup.

The kubelet identity is a user-assigned managed identity that is primarily used by the AKS nodes, unless explicitly overridden, to interact with Azure resources, such as Azure Container Registry (ACR) and Azure Storage.

When creating an AKS cluster, you need to assign the kubelet identity the 'Storage Blob Data Contributor' role on the Azure Blob Container resource. This role is required for the kubelet to interact with Azure Storage for persistent volume mounting.

To enable Managed Identity on AKS cluster, Azure CLI must be installed on the environment before following the below steps. For more information, see Setup managed identity on AKS

 

Retrieve the Kubelet Identity

To use the Kubelet Identity for authentication, retrieve its details using the following:

Copy 
az aks show --resource-group <your-resource-group> --name <your-cluster-name> --query "identityProfile.kubeletidentity"

Example output:
{
  "clientId": "<kubelet-identity-client-id>",
  "objectId": "<kubelet-identity-object-id>",
  "resourceId": "/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<kubelet-identity-name>"
}
 

 

  • clientId: The client ID of the Kubelet Identity

  • objectId: The principal ID of the Kubelet Identity

  • resourceId: The resource ID of the Kubelet Identity

 

Assign Roles to the Kubelet Identity

To allow the Kubelet Identity to access the Azure Blob Container, assign the Storage Blob Data Contributor role to the Kubelet Identity.

To create the respective role for the kubelet-identity, run the following:

Copy 
az role assignment create \
  --assignee <kubelet-identity-object-id> \
  --role "Storage Blob Data Contributor" \
  --scope /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Storage/storageAccounts/<storage-account-name>

Configure the values.yaml

Provide the azureStorageIdentityClientID obtained above in values.yaml

Copy 
azureStorageIdentityClientID=<kubelet-identity-client-id>

 

Troubleshooting

As MicroK8s uses a different path to other standard distributions, when you are installing the Blob CSI driver on MicroK8s using:/var/snap/microk8s/common/var/lib/kubelet you may receive this error:

Could not check if "/var/snap/microk8s/common/var/lib/kubelet/pods/4a5d6dfb-646a-462d-a223-d510c059f8b7/volumes/kubernetes.io~csi/blob-pv/mount" is a mount point, Failed to read /host/proc/mounts: open /host/proc/mounts: invalid argument

If you encounter the above error, you will need to inform the /var/snap/microk8s/common/var/lib/kubelet path to driver while installing it using helm.

This is the helm chart installation command:

Copy 

helm install blob-csi-driver blob-csi-driver/blob-csi-driver --set node.enableBlobfuseProxy=true -–set linux.kubelet="/var/snap/microk8s/common/var/lib/kubelet" --namespace kube-system --version v1.25.1

You may also need to make the mount location shared:

Copy 
/bin/mount --make-shared /var/snap/microk8s/common/var/lib/kubelet

 

Setup

When you have completed the steps in the prerequisites section, you will need to create a PV (persistent volume) for the staging area using manual PV creation or automatic PV creation. Choose only one approach. Manual PV creation should not be used when automatic PV provisioning is enabled.

Option 1: Manual PV creation (Cluster Administration)

With this option, a Kubernetes administrator manually creates the PV using the provided template and shares the PV name.

  • The PV name must be specified in values.yaml using the stagePvName parameter.

  • Hyperscale will use the existing PV during deployment.

 

Option 2: Automatic PV Creation (Hyperscale Managed)

In this option, Hyperscale Orchestrator automatically creates the PV during deployment using the Azure Blob–related properties defined in values.yaml.

  • No manual PV creation is required.

  • The PV lifecycle is managed by Hyperscale.

 

Copy 
apiVersion: v1   
kind: PersistentVolume   
metadata:   
  annotations:   
    pv.kubernetes.io/provisioned-by: blob.csi.azure.com   
  name: blob-pv   
spec:   
  capacity:   
    storage: 10Gi   
  accessModes:   
    - ReadWriteMany   

  mountOptions:   
    - -o allow_other   
    - -o direct_io   
    - --file-cache-timeout-in-seconds=0   
    - --use-attr-cache=false   
    - --invalidate-on-sync=true 
    - --attr-timeout=0 
  csi:   

Copy 
    driver: blob.csi.azure.com   
    volumeHandle: azure-csi-driver-volume-handle # Should be unique for a container   volumeAttributes:   
      storageAccount: <STORAGE_ACCOUNT> 
      containerName: <CONTAINER_NAME> 
      AzureStorageAuthType: MSI # key, MSI   
      protocol: fuse2   

 

 

The following properties shall be specified in values.yaml when PV is created manually or Hyperscale managed.

  • stagingStorageType

  • blobAccountName

  • blobContainerName

  • blobAuthType

  • blobContainerPrefix

  • blobContainerDelimiter

  • blobSecretKey

  • stagingAzureBlobSecretName

For more information on these parameters, see Commonly Used Properties

 

Additional information

  • Set the stagingStorageType as AZURE_BLOB_STORAGE

  • To connect to the Blob Container from the Continuous Compliance engine, you can use either of the following authentication mechanisms:

    • Instance profile-based authentication (AZURE_MANAGED_IDENTITY)

    • Access and secret key based authentication (AZURE_SECRET)

  • If authMechanism is set as AZURE_MANAGED_IDENTITY, a Managed Identity that grants access to the Blob Container must be attached to the Azure instances of either the Hyperscale Compliance engine, Continuous Compliance engine, or the AKS cluster. Refer to the Azure Kubernetes Services (AKS) knowledge guide for further information.

  • If authMechanism is set as AZURE_SECRET, you will need to provide either of the following:

    • stagingAzureBlobSecretName: Name of the Kubernetes secret that holds blobSecretKey information. The stagingAzureBlobSecretName can be created by a cluster administrator using this command line:

      Copy 

      kubectl create secret generic <secret-name> --from-literal=azurestorageaccountname=<storage-account> --from-literal azurestorageaccountkey=<blob-secret--key> --type=Opaque

    • blobSecretKey: Hyperscale Orchestrator will use these values and automatically create a generic secret.