Preview feature

Workflows are not available for use by default, to use them you must enable the workflows feature flag. Contact us if you'd like to be a part of the preview and try out this new functionality.

What are workflows?

A workflow is a list of jobs and steps that are executed in sequence to produce a final result. For example, to create a masked image from a backup. Under the hood, these use the new rgworkflow CLI. This page will detail how to get started with workflows in the TDM Hub.

New UI

With workflows enabled new pages will be visible within the UI. The landing page will be the Workflows page described below instead of Database models. On the UI's left-hand side, above the Treatment section, will be a new section called Delivery. This section contains numerous pages related to managing workflows and what they produce.

• Workflows - The page where workflows are created, displayed, and run
• Images - The page which contains a list of all images that have been created from workflows
• Clones - The page which contains a list of all clones which have been spun up from the above images
• Connections - The page which contains a list of all connections to databases on which we can run workflows

Getting started with Workflows

Connections

The first step in using workflows is to establish a connection to the database you wish to run that workflow against; this can be done from the connections page.

Connections can be made to SQL Server, PostgreSQL, and MySQL databases.

All connections will require you to pass admin-level credentials, which will be stored securely. This should be provided in the form of an ADO/.NET connection string.

You must provide a file-share path where backups can be found to be used as sources. This can be either a local filepath or to a mapped network drive filepath. Note that the database server you are connecting to must have access to this fileshare.

Workflows

Creating a workflow

When you have created a connection, the next step is to create a workflow.

To do so, visit the Workflows page and click Create workflow

The workflow wizard has multiple steps, first you must select a source for your image.

This source will be the base of the created image: Our image will have an identical schema and, depending on treatment options, some shared data. The next step is to select what level of masking and/or subsetting you wish to apply to your final image.

In the above example, the base image we have is a backup of the database TDM, we then specify that we want our image to be a 1GB subset with any sensitive data masked. The final step is to name the workflow, and we can then begin using it to create images

Running your workflow

After creating a workflow, you will be redirected to your workflow list again, where you will see your new workflow.

This page shows the status of the latest run of each workflow in real time under the "Last run" columns, more detailed information and information on previous runs can be found in each workflow's details page accessed using the link in the Name column:

From this page, click the "Run workflow" button to start a new run.

In a dialog, you will be asked what you'd like to name your image, and if you'd like to pick a different source file from what you originally chose.

Once you start the run with the Run workflow button, you will see the run has been added to the Runs tab.

This tab will be frequently updated with the status of all runs of this workflow, including historically completed ones.

Optionally, you can click on the run's ID to see more details on what is happening within the run, including live logs, which may help you diagnose a problem if the run fails.

Images

If your workflow run succeeds, you will have a new image. There are many place images can be viewed:

• Within a workflows Images tab
• Within a runs image tab
• On the images page

The page you view it on will determine how the images are filtered. For a broad view of all images created across workflows, 

On the images page, there are two display methods on this page, cards or the more detailed data-grid, switchable via a mode toggle in the top-right. Here's the datagrid:

Each image is associated with a specific run of a specific workflow, and links are available to navigate to those two entities. Information is visible on what kind of data treatment has been applied. Columns containing more information can be viewed by clicking the Columns button.

Some actions to run on the image are available in the Actions column, including the ability to create a clone from it.

Clones

The clones page is where information on all existing clones is displayed.

Many clones can be created from each image. When creating a clone, you can select the connection it was created on, or possibly another connection if they have a file share in common.

Clones can either be created from any image list, or from the clone page itself via the Create Clone button.

Once you've entered the necessary details and waited for the clone to be created, you can access the clone by copying it's connection string and connecting to it with your tool of choice