Overview

This checklist guides new users through setting up and deploying Redgate Test Data Manager for a successful Proof of Concept (PoC).

Our product includes a user-friendly GUI along with powerful CLI tools including anonymize, data generation, subsetting, and cloning.

Important – In all cases, we recommend you use a separate, non-essential environment for the Proof of Concept. We cannot accept responsibility for any issues resulting from setting up the Proof of Concept. Any activities carried out on production or otherwise important systems are not advised and at your own risk.

Before You Start: What you need to know 

• Test Environment: Use a dedicated test environment to keep live data safe.
• System Requirements: Make sure your system meets our minimum requirements here for a smooth setup. And check out some suggested set up tips in our architecture diagrams.
• Skills needed:  Linux skills and Kubernetes/Docker knowledge are recommended.
• Redgate University Courses: For videos and top tips from the experts, take a look at our Test Data Manager courses on Redgate University.

Installation Overview

Getting started with Clone

1. We'd recommend starting with the installation of Clone as this will provide your environment for delivering test data.
2. See our guide on how to get set up.

Getting clone up and running

1. Start empty: Get yourself an empty container up and running
2. Add Sample Databases: Load databases, for example a sample database, or pre-prepared database, into your container for testing.
3. Use Anonymized Data: Incorporate anonymized data from a previous scrubbing project.
4. Scale Up: Increase capacity as needed.

Anonymize, Subsetting and Data Generation Command Line Installation

• Anonymize: For installation and initial setup of the Anonymize tool, refer to this link.
• Subsetting: Installation instructions for the Subsetting tool and requirements.
  • Subsetting requires both a source database, usually a rehydrated copy of production, and a target database. More detail below.
• Data Generation: See our guide for installing and starting with the Data Generation tool.

For some suggested workflows for Test Data Manager, and visual explanations of the subset and anonymize command lines take a look at our architecture pages.

Database Setup

Source Database

• Use a development database for making and capturing changes. Ensure full access (e.g., db_owner for SQL Server, schema owner for Oracle).
• Create a backup of your production database for testing.

Target Database

• For subsetting, provision an empty, schema-only target database.
• For initial PoC, use an isolated database without linked servers or cross-database dependencies. Let us know if your real projects require these features.

Getting Started with Anonymize

1. Prepare Your Environment:

   • Create a backup copy of the latest version of your chosen database.

2. Run Initial Classification:

   • Create a batch or bash file to run the classify command on your database. Classify will analyze the database and identify sensitive information.

   • Review the classification JSON to understand what data has been identified. See default classifications.

   • Pro tip: Use the --output-all-columns flag with the classify command to get a JSON output of all database columns. This helps identify missing or additional columns to include in your options file.

3. Configure Options File:

   • Based on the classification review, create an options file if you need to change how the data is being handled.

   • For example excluding columns, excluding tables, or creating custom classifications.

4. Map and Mask Data:

   • Update your batch or bash file to include the map and mask commands, along with the --options-file parameter if needed to specify your configuration.

   • Run the batch or bash file to perform the data mapping and masking based on your configuration.

   • Pro tip: Before running the mask command, use the --dry-run flag to simulate the masking process and check for potential errors without actually modifying the data.

5. Review and Iterate:

   • Take a look at the masked data in your database, and check whether the output is what you're expecting.

   • If needed, update your options file and re-run the mapping and masking process until you achieve the desired results.

For more details and advanced configuration options, refer to the Anonymize command line reference.

Next Steps

1. Learn More:
   • Dive deeper into Test Data Manager by exploring our Redgate University videos for comprehensive tutorials and demonstrations.
2. Explore Individual Capabilities:
   Discover the full potential of Test Data Manager by visiting our dedicated pages for each key feature:
   
   • Clone: Create lightweight copies of your databases for testing and development.
   • Subset: Intelligently build small subsets of data from large databases.
   • Anonymize: Protect sensitive information by replacing it with realistic, anonymized data.
   • Data Generation: Populate your databases with synthetic data for testing and development.