Data Masker

About Command Line Automation

The command-line is installed alongside the Data Masker application. To see this in action, see our samples.

Usage: DataMaskerCmdLine [options] [commands]

If no command is specified, the run command will be applied to the given PARFILE.

Options:

  • PARFILE=<parfile>, --parfile <parfile> (REQUIRED for run)
    • Path to the parameter file containing the details of the masking job to run, see the parameter file section for more info.
  • --version
    • Show version information.
  • --help, -?, -h
    • Show help and usage information for the command-line.

Commands:

For example: 

> "C:\Program Files\Red Gate\Data Masker for SQL Server 7\DataMaskerCmdLine.exe" PARFILE=C:\MaskingSets\SamplePARFILE.txt
> "C:\Program Files\Red Gate\Data Masker for Oracle 6\DataMaskerCmdLine.exe" PARFILE="C:\Masking Sets\Sample PARFILE.txt"

The column-template build-mapping-file Command

Builds a default mapping file using information from SQL Data Catalog. By default it will mask all columns tagged with 'Static Masking' under the category 'Treatment Intent', using templates defined in the category 'Masking Data Set'. If a tag in this category is not recognised as a template, the default mapping for that datatype is used instead (see table under options).

Usage: DataMaskerCmdLine column-template build-mapping-file [options]

Options:

OptionRequiredDescription
--catalog-uri <catalog-uri>YesURI to connect to SQL DataCatalog e.g. http://[Your SQL Data Catalog Server FQDN]:15156.
--api-key <api-key>NoAPI key to connect to SQL DataCatalog, if not provided Windows authentication for the current user will be used.
--instance <instance>YesThe fully-qualified name of the SQL Server instance. For a named instance, this should take the form 'fully-qualified-host-name\instance-name' (e.g. myserver.mydomain.com\myinstance). For the default instance on a machine, just the fully-qualified name of the machine will suffice (e.g. myserver.mydomain.com).
--database <database>YesDatabase name to build the mapping file for.
--information-type-category <information-type-category>NoCategory in SQL DataCatalog which defines information type, we expect 1-1 mapping between the tag and the column template name. Default value 'Masking Data Set'.
--sensitivity-tag <sensitivity-tag>NoThe tag to identify whether we should mask the column. Default value 'Static Masking'.
--sensitivity-category <sensitivity-category>NoThe category that contains the sensitive tag. Default value 'Treatment Intent'.
--mapping-file <mapping-file>YesLocation to save the generated mapping file.
--log-directory <log-directory>YesThe location to save log files
--help, -?, -hNoShow help and usage information for the column-template build-mapping-file command.


Default templates (see mapping file section for more information on templates) for each datatype (used when a tag in information-type-category is not recognised as a template):

DatatypeTemplate
bigintInteger
dateDate
datetimeDate
datetimeoffsetDate
datetime2Date
decimalDecimal
floatDecimal
intInteger
moneyDecimal
ntextLong string
numericDecimal
realDecimal
smalldatetimeDate
smallintInteger
textLong string
contains "char" , max precision (e.g. varchar(max))Long string
contains "char", precision under 6 (e.g. varchar(5))Random alphanumeric string
contains "char", precision 6 or over (e.g. varchar(20))Short string

The build Command

Prepares an empty masking-set file for the given connection, unless a mapping-file is provided which will populate the masking-set with default rules.

Usage: DataMaskerCmdLine build [authentication] [options]

Authentication:

  • using-windows-auth
    • Use Windows authentication.
  • using-sql-auth
    • Use SQL authentication.

Options:

OptionRequiredDescription
--masking-set-file <masking-set-file>YesThe location to save the new masking set to be interpreted by Data Masker.
--log-directory <log-directory>YesThe location to save log files.
--instance <instance>YesThe SQL Server instance.
--database <database>YesThe database name.
--username <username>SQL auth only

The user name.

--password <password>SQL auth only

The password.

-s, --schema <schema>NoA schema that you would like to mask. To add multiple schemas, specify this option for each schema. If not specified, all schemas that contain tables will be added to the masking set.
-p, --parfile <parfile>NoThe location to save a parameter file to run the generated masking set.
--mapping-file <mapping-file>NoThe column template mapping file in JSON format, see the mapping file section for more info.

--help, -?, -h

NoShow help and usage information for the build command.

The run Command

Run a masking set using the specified PARFILE.

Usage: DataMaskerCmdLine run [options]

Options:

  • PARFILE=<parfile>, --parfile <parfile> (REQUIRED)
    • Path to the parameter file containing the details of the masking job to run, see the parameter file section for more info.
  • --help, -?, -h
    • Show help and usage information for the run command.

The Parameter File

The parameter file can specify the following options:

ParameterRequired?Description
MASKINGSETRequiredThe full path of the masking set to be run.
LOGFILEDIRRequiredThe full path to the directory where log files will be placed.
DATASETSDIRRequired

The full path to the directory containing the datasets. Unless you have created your own directory, the datasets are installed alongside Data Masker in the following locations:

  • SQL Server: 
    • C:\Program Files\Red Gate\Data Masker for SQL Server 7\DataSets 
  • Oracle: 
    • C:\Program Files\Red Gate\Data Masker for Oracle 6\DataSets 
REPORTSDIRRequiredThe full path to the directory where the reports should be exported.
INTERIM_REPORTSOptional

Either true or false. If not specified, the default value will be false.

If true, the reports in the REPORTSDIR will be overwritten about once every minute, should you wish to track progress while the command-line is running.

LOGINSUBOptional

Add a LOGINSUB line to the file for each controller in the masking set that needs to be changed to point to a different database (or with a different user account).  

For more information about login substitution see: 

DMSSUBVALUExxOptional

SQL Server Only. Currently only available in legacy performance mode.  Data Masker will replace any instance of these parameter names with their value in any SQL. The xx is a two-digit numeric value. See substitution values for more details.

DMOSUBVALUExxOptional

Oracle Only. Currently only available in legacy performance mode.  Data Masker will replace any instance of these parameter names with their value in any user-specified SQL. The xx is a two-digit numeric value. See substitution values for more details.

  

DMSSUBVALUExx (SQL Server version) / DMOSUBVALUExx (Oracle version)

Up to twenty replacement strings can be specified in the parameter file. These runtime substitution values are designed to permit generic SQL statements to be dynamically adjusted at runtime for specific targets during batch operations.

Note: The keywords for these replacement strings are DMSSUBVALUE00 to DMSSUBVALUE19. If the text %DMSSUBVALUExx% is used in a Command rule or in the Where Clause of a masking rule, the value associated with that keyword will be replaced at runtime before the rule is executed.

For example, if a Command rule included a statement like:

TRUNCATE TABLE %DMSSUBVALUE05%_TEMP;

and the parameter file contained a line stating DMSSUBVALUE05=TESTUSER then the actual statement that would be executed would be:

TRUNCATE TABLE TESTUSER_TEMP;

The value TESTUSER would be substituted for the %DMSSUBVALUE05% text in the SQL statement immediately before the statement was executed.

Examples

SQL Server PARFILE

MASKINGSET=C:\Program Files\Red Gate\Data Masker for SQL Server 7\Demo.DMSMaskSet 
LOGFILEDIR=C:\TEMP\Data Masker\
DATASETSDIR=C:\Program Files\Red Gate\Data Masker for SQL Server 7\DataSets 
REPORTSDIR=C:\TEMP\Data Masker\
INTERIM_REPORTS=false 
LOGINSUB=placeholder-login@DEV-DB[DataWarehouse]||DataMasker/SecurePassword@STAGING\SQL2019[DataWarehouseMasked]
-- Substitution values
DMSSUBVALUE00=DM_INVOICE_LINE_HISTORY
DMSSUBVALUE05=DM_INVOICE_LINE_HISTORY

Oracle PARFILE

MASKINGSET=C:\Program Files\Red Gate\Data Masker for Oracle 6\Demo.DMSMaskSet 
LOGFILEDIR=C:\TEMP\Data Masker\
DATASETSDIR=C:\Program Files\Red Gate\Data Masker for Oracle 6\DataSets 
REPORTSDIR=C:\TEMP\Data Masker\
INTERIM_REPORTS=false 
--LOGINSUB=DummyLogin@DummyServer[DummyDatabase]||DataMasker/DataMasker@DEMODB[DMTest] 
--LOGINSUB=XXXLogin@XXXServer[XXXDatabase]||DBSSourceUser/DBSSourceUser@N2KBUILD-PC\N2KTEST1[DBSSource]

DMOSUBVALUE00=DM_INVOICE_LINE_HISTORY
DMOSUBVALUE05=DM_INVOICE_LINE_HISTORY

The Mapping File

Maps columns in the database to pre-defined templates that the build command uses to create a masking set.

Currently available templates are:

Template NameDescription
Given nameMale + Female first names.
Family nameSurnames.
Full nameMale + Female full names.
TitleTitles, e.g. Mrs, Mr
Date of birthRandomly offset dates within roughly two months in either direction.
Email addressGenerated email addresses, e.g. sus.syy@massa.com
Street addressGenerated street addresses, e.g. 272 Getchell St.
City

These templates are correlated - entries on the same row will form a coherent address.

Addresses are US addresses.

State
ZIP code
County
CountryCountries.
Full address

A complete address in the form:

123 Zoffer St.

Lake Charles LA 70609

Debit/credit card numberDigits only VISA numbers.
Debit/credit card expiry dateCard expiry date, e.g. 06/20
Bank account numberBank account numbers - digits only, e.g. 304340718
Bank sort codeUK bank sort codes - digits only, e.g. 930184
SWIFT-BICSWIFT-BIC Codes.
Phone numberNorth American telephone numbers - digits only, e.g. 5062699083
GenderGenders.
NationalityNationalities.
Passport numberPassport numbers - digits only, e.g. 234001394
Social Security numberUnique social security numbers
OccupationJob titles.
Organization nameCompany names
URLGenerated URLs, e.g. http://hexagonal.net
IP addressIP addresses.
MAC addressMac addresses.
Password12 character passwords - e.g. a94tgh32lk09
Driving license numberDriving license numbers - digits only, e.g 940021042

Photo

Photos represented in binary
Vehicle registration numberUK vehicle registration numbers.
IntegerRandom integers between 1 and 100.
DecimalRandom decimal numbers roughly in the range of -125 to 570 with 2 decimal places.
Short stringRandom dictionary words.
Long stringParagraphs of gibberish with max chars 1000, min chars 1 and max word length 10.
Date

Masked identically to Date of birth template.


Example mapping file:

{
    "source": "SQL Data Catalog",
    "mappings": [
      {
        "template": "Given name",
        "independentlyMasked": [
          { "schema": "dbo", "table": "People", "column": "F_NAME" },
          { "schema": "dbo", "table": "Employees", "column": "F_NAME" }
        ]
      },
      {
        "template": "Family name",
        "independentlyMasked": [
            { "schema": "dbo", "table": "People", "column": "S_NAME" },
            { "schema": "dbo", "table": "Employees", "column": "S_NAME" }
        ]
      }
    ]
}


Exit Codes

The command-line will return an exit code to indicate success or failure; if the run of the masking rules was successful, then this exit code will always be zero. Any non-zero exit code implies that an error occurred while opening the masking set or while running the masking rules. If a non-zero exit code is returned, the log file should always be examined to determine the error that occurred. The defined error codes are:

Error codeDescription
200 - EXITCODE_RULERUN_ERROR

There was an error executing one or more masking rules. See the log file for more information.

110 - EXITCODE_PARAMFILESUB_ERRORThere was an error processing the LOGINSUB substitution keyword in the parameter file.
109 - EXITCODE_MASKSETOPEN_ERRORThere was an error opening the masking set. See the log file for more information.
108 - EXITCODE_MASKSETMISSING_ERRORThe masking set specified in the parameter file could not be found. See the log file for more information.
107 - EXITCODE_REPORTSDIR_ERRORThe specified report directory could not be found or there were permissions issues. See the log file for more information.
106 - EXITCODE_DATASETBUILD_ERRORThere was an error reading the datasets. See the log file for more information.
105 - EXITCODE_DATASETSDIR_ERRORThe specified dataset directory could not be found or there were permissions issues. See the log file for more information.
104 - EXITCODE_LOGFILEDIR_ERRORThe specified log file directory could not be found or there were permissions issues. See the log file for more information.
103 - EXITCODE_CMDLINEARGS_ERRORThere was an error processing the command line. See the log file for more information.
102 - EXITCODE_LOGCREATE_ERRORThere was an error creating the log file.
101 - EXITCODE_LOGINIT_ERRORThere was an error initializing the logging sub-system.
100 - EXITCODE_UNSETUnspecified error. See the log file for more information.
99 - EXITCODE_UNHANDLED_EXCEPTIONAn exception occurred. See the log file for more information.
0 - EXITCODE_SUCCESSThe run of the masking set completed successfully. The log file and reports will have full details.


If an error occurs there will be more detail in the log that is created in the LOGFILEDIR directory, some information will be written to the console output. 

If running from the standard command-line then  `echo %errorlevel%` can output the exit code. For example:

C:\Program Files\Red Gate\Data Masker for SQL Server 7>DataMaskerCmdLine.exe PARFILE=C:\workspace\data-masking\DMV6\DMS_TestSets\DebugPARFILE.txt
...
C:\Program Files\Red Gate\Data Masker for SQL Server 7>echo %ERRORLEVEL%
200






Didn't find what you were looking for?