This page will explain what connections are and how to manage them in Test Data Manager.

What is a connection?

A connection in TDM is:

• A link to a server, database or TDM Clone instance (all generalized here as an "instance")
• Coordinates and credentials for said instance
• Mapped drives that instance has access to that TDM should scan for backups

Once created, a connection allows you to create images and clones, the key actions of Test Data Manager.

Legacy connections

If you set up your TDM Hub when only TDM Clone was supported as a connection type (<v0.8.15), you will have entered connection details to your singular clone instance via the config.yml file using fields like RgCloneApiToken. This approach is no longer supported, but those details will have been migrated and imported as a new connection called tdm-rgclone.

It is recommended that you delete the RgClone_ parameters in the config.yml, as changing them will no longer work and leaving them in will result in tdm-rgclone being automatically recreated if it is deleted. Should you need to edit the details of this connection, please do so in the UI.

Managing connections

This section will cover how to view, add, edit and remove connections.

The Connections Page

The connections page can be accessed via the Connections link in the navigation drawer.

There, you will be greeted with a page that looks like this

The page features:

• A list of connections, each with an actions button
• A button to add new connections, level with the page heading.

Adding a connection

To add a connection, click the "Add connection" button. This will open the add connection sidebar.

Here, you will be asked what you would like to name the connection, and the connection type.

Test Data Manager Clone

Test data manager clone connections require you to have set up a TDM Clone (also known as Redgate Clone) instance.

TDM Clone is a system that allows users to create whole virtualized servers in Kubernetes.

For more information about Redgate and how to set it up, please see its documentation.

Use cases

TDM Clone's key advantages over database servers are:

• As many clones of databases can be spun up simultaneously without changing the names of the databases to avoid conflicts.
• Whole instances (multi-database) can be restored together
  NOTE: This feature is not yet supported in TDM Hub. This must be achieved via the standalone CLIs for now.

It's disadvantages are:

• Generally slower to restore to than database servers
• Due to some performance limitations, it may not be the best choice for masking and subsetting operations.

To summarize, TDM Clone is mostly recommended for use to deploy clones for with test data to developers, not to perform heavy subsetting and masking operations on production data.

Connection details

In order to securely communicate with the TDM Clone server, it is required to use the Client Credentials Connection Method. The Access Token method is provided for backwards compatability only.

In order for the Service Principal running the GUI to connect to TDM Clone, you will need to configure your identity provider to enable the Client Credentials flow and to assign the Clone.AdminAccess claim. An example of how to set this up has been outlined below for Azure.

Adding the administrator claim

1. In the Azure portal, navigate to the "App Registrations" section.
2. Select the TDM app.
3. From the 'App roles' tab (located under 'Manage'), click the 'Create app role' button in the action bar.
4. In the 'Create app role' form, ensure the new role has 'Allowed member types' set to 'Applications' and that 'Value' is set to Clone.AdminAccess. The 'Display name' and 'Description' values can be set to any value you like.
5. From the 'API Permissions' tab, click the 'Add a permission' button under 'Configured permissions'.
6. Select 'My APIs' under 'Select an API' and find the App registration that you have created the App role for.
7. Select 'Application permissions'
8. Find the App role you have just created and select it, then click 'Add permissions'.
9. You may need an administrator to approve this request.

Creating a Client Secret

1. In the Azure portal, navigate to the "App Registrations" section.
2. Select the TDM app.
3. From the 'Certificates & secrets' tab (located under 'Manage'), click the 'New client secret' button.
4. Give the new secret a description and choose an appropriate expiry based on your IT policies.
5. Remember that you will need to repeat these steps before the secret expires.
6. Click the 'Add' button.
7. You will be returned to the list of Client secrets, with the newly created secret added. The Value of this secret will be available to copy to your clipboard. This will not be available to copy once you leave the page so securely store this value before leaving.

The connection details needed to connect to a TDM Clone instance are:

• Your instance's API endpoint
• The Client Secret created above
• Any additional scopes your Identity Provider requires to configure the returned access token with the appropriate claims. For the example Azure configuration above, this will need to be set to api://{Application Client Id}/.default 

Database servers

Database servers are typically standalone, physical servers owned or managed by your organisation. For example, a bespoke Microsoft SQL Server.

At the moment, cloud servers e.g. Azure databases are not supported.

The currently supported platforms for database servers are:

• Microsoft SQL Server
• Postgres
• MySQL
• Oracle

Use cases

Database servers key advantages over TDM Clone are:

• Greater restore and masking / subsetting speed.
• No need to set up new bespoke infrastructure if you already have a serviceable server.

It's disadvantages are:

• You cannot restore whole instances (multi-db)
• Clone database names are typically randomized to prevent collisions, so they don't always make for very good clones for developers to use.

To summarize, Database servers are best used to preform masking and subsetting operations on production data, which can then be transferred to TDM Clone to host clones for developers.

Database servers are typically the best choice for subsetting and masking operations that can be too heavy for TDM Clone.

Connection details

The only thing you need to connect to a database server is a connection string. Currently, only ADO (.NET) format connection strings are supported.

The credentials provided should typically have sysadmin or superuser privileges. You can check whether you have enough permissions using the test connection feature.

File shares

Both types of connections allow you to specify files shares.

These file shares are where the connection will look for backups to use as sources for image creation.

Currently, TDM Hub only supports one file share per connection.

For instances running on Windows machines, it is worth noting that services cannot always access network drives mapped to drive letters, so instead of using Z:\backups you might need to use the underlying address, like \\mysmbfile share\backups.

Testing the connection

All connection types feature a connection testing feature. This will use the credentials you have provided and attempt to connect to the server and check that you have sufficient permissions to perform the typical operations that must be performed on connections.

The test connection button does not currently check your access to file shares.

Editing connections

To edit an existing connection, find it in the connections list and click the actions (⁝) button, then click Edit.

You will be provided with a simplified version of the add connection sidebar.

Obfuscated fields

Note that some fields will be greyed out and either fully or partially obfuscated.

This is done for security, as we do not want to fetch and expose credentials.

If untouched, these fields will remain unchanged when the save button is pressed.

If you need to update the credentials, you will need to click the edit (pencil) button, which will clear the field and allow you to enter it again.

Deleting a connection

Similarly to editing a connection, deleting is done by finding your connection in the list clicking the actions (⁝) button, then clicking Delete.

Deleting a connection will prevent users from being able to create new images and clones.

It is not possible to delete a connection if any images, and therefore also any clones, exist on the connection.