Working with file formats

Use these tasks to import, edit, duplicate, export, and delete file formats in the Continuous Compliance Engine. File formats define the structure of the data files that the engine reads during masking and profiling operations.

A file format describes the layout of a source file; its fields, delimiters, record types, or copybook definitions so that the engine can correctly parse the file and apply masking assignments. The supported format types are Delimited, Fixed-Width, XML, JSON, Parquet, and Mainframe Copybook. File inventories are derived from the file format they reference, so any change to a format applies to every data file that uses it.

Import file formats

Use this task to import a Delimited, Fixed-Width, XML, JSON, or Parquet file format. For Mainframe copybooks, see Import a Mainframe format.

Steps

  1. From Home, go to Settings > Data Formats, then select the Files tab at the top.

  2. Click + File Format.

  3. In the Format Type step:

    1. Select the format type.

      Formatting Examples are shown in the wizard, based on each format type.

    2. Click Next.

  4. In the Import Format step:

    1. (Optional) Enter a custom name for the format. If left blank, the format is named after the uploaded file.

    2. Click Choose File and select the file from which the fields will be imported. The contents of the imported file vary for delimited, fixed-width, XML, JSON, and parquet file types.

      Do not upload sensitive data when creating a Parquet file format, because the exported file may expose it.

    3. To replace a file selected in error, click the x next to the file name and select another file.

    4. Click Next.

  5. (Delimited and Fixed-Width only) In the Header & Footer step, configure the number of header or footer records, or leave the values empty.

  6. Click Next.

  7. In the Summary step, review the details.

  8. Click Save.

Import a Mainframe format

Use this task to import a Mainframe copybook. Duplicate field names within a copybook file are not supported.

Steps

  1. From Home, go to Settings > Data Formats, then select the Maninframe tab at the top.

  2. Click + Mainframe Format. This will import the copybook directly into the Continuous Compliance Engine.

  3. The Format Type will be preselected with Copybook, click Next to continue.

  4. In the Import Format step:

    1. (Optional) Enter a custom name for the format. If left blank, the format is named after the uploaded file.

    2. Click Choose File and select the file from which the fields will be imported. To replace a file selected in error, click the icon next to the file name and select another file.

    3. Click Next.

  5. In the Summary step, review the details.

  6. Click Save.

Edit a file format

Use this task to modify an existing file format. Editing is supported for all format types, including Mainframe.

Steps

  1. From Home, go to Settings > Data Formats.

  2. Select the Files or Mainframe tab.

  3. In the Name column, click the hyperlinked name of the format.

  4. Make your changes in the edit screen and save.

The file inventory relies on the file format it represents. As a result, any changes to the file inventory affect all data files using that file format. Thus, altering the file inventory impacts all files conforming to that format across the application. This is applicable to all the file types Fixed-Width, Delimited, Mainframe, JSON & XML.

Duplicate a file format

Use this task to create a copy of an existing file format under a new name. Duplication is supported for all format types, including Mainframe.

Steps

  1. From Home, go to Settings > Data Formats.

  2. Select the Files or Mainframe tab.

  3. Locate the format you want to duplicate.

  4. Open the Duplicate File Format dialog from the Actions column.

  5. Enter a new file format name.

  6. (Optional) Select Do not copy masking assignments to exclude file field algorithm assignments and document store type configurations from the copy. By default, masking assignments are copied.

  7. Click Save.

Duplicate and Export operations are not supported for Mainframe file formats on synced engines. These operations require access to the physical file, which is not included in the sync process. To perform these operations, use the source engine.

Export a file format

Use this task to download an uploaded file format. Export is supported for all format types, including Mainframe.

For Fixed-Width and Delimited formats, you can select a specific record type to export, or leave the selection empty to export the default All Records format. For XML and JSON formats, the system generates a file format template with placeholder values based on the extracted path structure. Mainframe Copybook formats are exported directly without a record type selection.

Steps

  1. From Home, go to Settings > Data Formats.

  2. Select the Files or Mainframe tab.

  3. Locate the format you want to export.

  4. Open the export option from the Actions column.

  5. (Fixed-Width and Delimited only) In the Export File Format dialog, select a record type, or leave it empty to export All Records. For Mainframe Copybook formats, the file downloads directly.

  6. For Fixed-Width and Delimited, click Export to download the file.

For exported XML file formats that contain namespace fields, the namespace definitions must be manually copied back into the exported file before it is used to create a new file format.

Delete a file format

Use this task to remove a file format from the engine.

Steps

  1. From Home, go to Settings > Data Formats.

  2. Select the Files or Mainframe tab.

  3. In the Actions column for the format, click the Delete icon.

  4. Confirm the deletion when prompted.