Cloud file connectors

Cloud file connectors are a subcategory of Create a file connector. They enable DCT Compliance to discover and mask sensitive data stored in cloud object stores directly, without requiring a local file system or mount. For FTP and SFTP-based file connectors, see Create a file connector.

Cloud connectors are created and managed entirely in DCT. Compliance nodes are treated as ephemeral compute and receive a copy of the connector only at job execution time. After the job completes, the connector copy is cleaned up automatically.

Supported cloud platforms

DCT supports the following cloud storage platforms:

  • Amazon S3 (AWS_S3) - Amazon S3 object storage.

  • S3-Compatible (S3_COMPATIBLE) - S3-compatible storage services such as MinIO and Ceph that use the S3 API.

  • Azure Blob Storage (AZURE_BLOB_STORAGE) - Microsoft Azure Blob Storage.

Authentication

Each cloud platform supports different authentication methods:

AWS S3

  • AWS_SECRET - Use explicit AWS access key and secret key tied to an IAM user. Credentials are stored in DCT.

  • AWS_ROLE - Use the IAM role attached to the compliance node's EC2 instance (instance profile). No credentials are stored in DCT; the node assumes the role automatically at runtime.

S3-Compatible

  • AWS_SECRET - Use access key and secret key. Same as AWS S3 but with a custom service endpoint.

Azure Blob Storage

  • AZURE_SECRET - Use a storage account access key. Credentials are stored in DCT.

  • AZURE_MANAGED_IDENTITY - Use the managed identity assigned to the compliance node's VM. No credentials are stored in DCT; Azure grants access based on the node identity at runtime. Note: PARQUET file type is not supported with managed identity.

Key concepts

File type is immutable: Choose your file type (DELIMITED, FIXED_WIDTH, XML, JSON, or PARQUET) when creating the connector. File type cannot be changed after creation.

Prefix and delimiter: Use prefix to scope file paths within the bucket or container (equivalent to a virtual directory). Use delimiter to split the storage path into folder-like segments (e.g., prefix = "raw/pii/" with delimiter = "/" restricts access to that folder).

Credentials are write-only: Access keys and secret keys are credential fields and are never returned in GET responses, even by the account that created them.

Ephemeral copies: When a compliance job runs, DCT pushes a temporary copy of the connector to the compliance node. After the job completes, the copy is cleaned up immediately. Failed or canceled jobs retain copies for 3 days (configurable) to allow investigation.

You can perform the following operations: