Preface
Introduction to Test Data Management
- Test Data Management Overview
- Test Data Management Use Cases
- TDM Architecture
- TDM Process
- TDM Example
- Using Advanced Parameterization in Test Data Manager
Test Data Manager
- Test Data Manager Overview
- Test Data Manager User Interface
- Overview View
- Policies View
- Data Sets View
- Projects View
- Monitor View
- Parameters View
- Administrator View
- Expression Builder
- Logging In to Test Data Manager
Projects
- Projects Overview
- Project Components
- Parameters
  - Rules and Guidelines for Parameters
- Project Logs
- Data Masking Transformation Session Properties
- Project Management
- Data Sources
- Delete a Table
  - Deleting a Table
- Project Permission and Security
  - Project Permissions
  - Updating User and Group Security
Policies
- Policies Overview
- Policies View
- Policies Task Flow
- Rules
- Data Domains
- Policy Packs
- Import and Export
- Linking Business Glossary Terms to Global Objects
  - Linking a Business Term to an Object
  - Deleting a Business Term Link to an Object
- Policy Management
Data Discovery
- Data Discovery Overview
- Data Discovery Sources
  - Rules and Guidelines for Data Discovery Sources
- Discover View
- Column Properties
- Data Discovery Task Flow
- Primary Key Discovery
  - Primary Key Profile Options
- Entity Discovery
  - Entity Profile Options
- Data Domain Discovery
- Column Profile
  - Inferred Rules
  - Column Profile Options
- Profile Management
- Profile Import
  - Importing a Profile
- Apply the Results
- Project Tables
Creating a Data Subset
- Data Subset Overview
- Data Subset Process Flow
- Data Subset Components
  - Entities
    - Entity Views
  - Groups
    - Group Example
- Creating an Entity
- Data Integrity Options in a Data Subset Plan
- Creating a Group
- Applying Criteria to an Element or Attribute
- Editing a Data Subset Component
- Exporting a Data Subset Component
- Importing a Data Subset Component
- Copying a Data Subset Component
- Deleting a Data Subset Component
- Creating a Data Subset
- Example - Data Subset for XSD Data Sources
Performing a Data Masking Operation
- Data Masking Overview
- Data Masking Task Flow
- Data Masking Rules
- Creating and Assigning Data Masking Rules
- Modifying Data Masking Rules and Assignments
- Performing a Data Masking Operation
- Data Masking Components
  - Masking Components in PowerCenter
    - Mapplet Export
Data Masking Techniques and Parameters
- Data Masking Techniques and Parameters Overview
- Data Masking Techniques
- Data Masking Parameters
  - Repeatable Output
    - Seed
  - Exception Handling
- Custom Masking
  - Custom Masking Parameters
- Advanced Masking
  - Advanced Masking Parameters
  - Advanced Masking Example
- Credit Card Masking
  - Credit Card Masking Parameters
- Email Masking
  - Email Masking Parameters
- Encryption Masking
  - Encryption Masking Parameters
- Expression Masking
  - Expression Masking Parameters
  - Rules and Guidelines for Expression Masking
- IP Address Masking
- Key Masking
  - Mask Format
  - Source String Characters
  - Result String Replacement Characters
  - Case Insensitive
  - Delimited String Masking
  - Date Key Masking
  - Numeric Key Masking Parameters
  - String Key Masking Parameters
- Nullification Masking
- Phone Masking
- Random Masking
  - Range Masking
  - Blurring
  - Mask Format
  - Source String Characters
  - Result String Replacement Characters
  - Date Random Masking Parameters
  - Numeric Random Masking Parameters
  - String Random Masking Parameters
- Shuffle Masking
  - Shuffle Masking Parameters
  - Rules and Guidelines for Shuffle Masking
- SIN Masking
- SSN Masking
- Substitution Masking
  - Substitution Masking Parameters
- URL Masking
- Name Substitution Example
  - Add a Dictionary in Test Data Manager
    - Adding a Relational Dictionary
    - Adding a Flat File Dictionary
  - Creating the Substitution Rule
  - Creating the Advanced Masking Rule
- Shuffle Address Example
  - Creating the Shuffle Rule
  - Create the Advanced Masking Rule
Data Generation
- Data Generation Overview
- Data Generation Components
- Rules and Guidelines for Entities
- Data Generation Task Flow
- Data Generation Rule Types
- Default Settings
- Define Default Generation Rules
- Standard Generation Rules
- Custom Generation Rules
  - Creating a Custom Generation Rule
- Ad Hoc Generation Rules
  - Creating an Ad Hoc Generation Rule
  - Editing an Ad Hoc Generation Rule
- Advanced Generation Rules
  - Advanced Generation Rule Example
  - Creating an Advanced Generation Rule
- Conditional Constraints
  - Conditional Constraints and Data Conflicts
- Data Generation Rule Assignments
- Data Generation for XSD Sources
  - Data Generation Task Flow for XSD Sources
  - Applying Data Generation Rules to XML Elements and Attributes
- Data Generation Plans and Workflows
Data Generation Techniques and Parameters
- Data Generation Techniques and Parameters Overview
- Data Generation Techniques
- Data Generation Parameters
  - Exception Test Data
    - Exception Test Data Parameters
- Custom Generation
  - Custom Generation Parameters
- Advanced Generation
  - Advanced Generation Parameters
- Dictionary Generation
  - Dictionary Generation Parameters
- Effective Dates Generation
  - Effective Dates Generation Example
  - Effective Dates Generation Parameters
- Expression Generation
  - Expression Generation Parameters
- Random Generation
- Credit Card Number Generation
  - Issuer Identification Number
  - Credit Card Generation Parameters
- Reference Lookup Generation
  - Reference Lookup Generation Parameters
- Sequence Generation
  - Date Sequence Generation Parameters
  - Numeric Sequence Generation Parameters
- Set of Values Generation
  - Set of Values Generation Parameters
- Conditional Generation
  - Conditional Generation Parameters
Working with Test Data Warehouse
- Test Data Warehouse Overview
- Test Data Warehouse Process
- Data Sets
  - Data Set Tags
- Test Data Management Self-Service Portal
- Creating a Data Set
- Reset a Data Set
- Editing the Metadata of a Data Set
- Publishing a Data Set to the Self-Service Portal
- Deleting a Data Set
- Related Data Sets
- Data Set Permissions
  - Editing Data Set Permission
- Locking and Unlocking a Data Set
- Monitor a Data Set Job
- View and Manage Data in a Data Set
Analyzing Test Data with Data Coverage
- Data Coverage Analysis Overview
- Data Coverage Process
- Creating a Data Coverage Task
- Data Coverage Task Columns
- Data Coverage Analysis Page
- Editing a Data Coverage Task
- Marking a Cell as Invalid
- Updating Data Across Cells
- User Input in Fill Cell Jobs
- Data Coverage Analysis Example
  - Tables in the Data Set
  - Analysis for Data Coverage
Plans and Workflows
- Plans and Workflows Overview
  - Plans and Workflows Task List
- Workflow Connections
- Plan Components
- Pre Workflow and Post Workflow Parameters
- Target Pre and Post SQL Statements
- Persist Mapping
- Plan Settings
- Masking Components
- Subset Components
- Generation Components
- Hadoop Components
- Component Criteria
  - Filtering Data Subset Components
  - Disabling Masking for a Column
- Source Settings
- Using a List File
- Plan Management
- Workflow Generation
- Parameter Files in Test Data Manager
  - Creating a Parameter File
- Executing a Workflow
- Workflow Executions View
  - Workflow Tasks
  - Workflow Properties Panel
    - Workflow Sessions Tab
    - Session Details
Monitor
- Monitor Overview
- Jobs
  - Job Details
- Monitor Tasks
- Logs
  - Severity Levels
  - Viewing the Log Messages
- Sessions
- Monitoring for Hadoop
Reports
- Reports Overview
- Audit Trail Report
  - Running an Audit Trail Report
- Data Masking Report
  - Running the Data Masking Report
- Plan Audit Report
  - Running a Plan Audit Report
- Plan Detail Report
  - Running the Plan Detail Report
- Row Count Report
  - Running the Row Count Report
ilmcmd
- ilmcmd Overview
- Configuring ilmcmd
- Running ilmcmd
- Entering Options and Arguments
- Syntax Notation
- Delete
  - Delete Examples
- Export
  - Export Examples
- Import
  - Import Examples
- Search
  - Search Examples
- Workflow
  - Workflow Examples
- Reset
- ListPlans
- TDWPlanGenerate
- TDWPlanExecute
- TDWPlanGenExe
tdwcmd
- tdwcmd Overview
- Running tdwcmd
- Entering Options and Arguments
- Syntax Notation
- List
  - List Examples
tdwquery
- tdwquery Overview
- Configuring tdwquery
- Running tdwquery
- Select Clause
Appendix A: Data Type Reference
- Data Type Reference Overview
- Oracle
- Microsoft SQL Server
- Microsoft Azure SQL
- Microsoft Azure SQL Data Warehouse
- Amazon Redshift
- DB2 for Linux, UNIX, and Windows
- Sybase ASE
- HDFS
- Hive
- Hadoop HDFS
- MySQL
- Flat File
- Sequential Single Record
- Sequential Multi Record
- VSAM Flat/Single Record
- VSAM Multi Record
- DB2 for z/OS
- DB2 for IOS
- IMS Flat/Single Record
- IMS Multi Record
- Sybase IQ
- Netezza
- Teradata
- Cassandra
- MongoDB
- PostgreSQL
Appendix B: Data Type Reference for Test Data Warehouse
- Data Type Reference for Test Data Warehouse Overview
- Oracle
- Microsoft SQL Server
- Microsoft Azure SQL
- Microsoft Azure SQL Data Warehouse
- Amazon Redshift
- DB2 for Linux, UNIX, and Windows
- DB2 for z/OS
- IMS Flat/Single Record
- IMS Multi Record
- Sequential Single Record
- Sequential Multi Record
- VSAM Flat/Single Record
- VSAM Multi Record
- Sybase ASE
- Teradata
- MongoDB
- Cassandra
- PostgreSQL
Appendix C: Data Type Reference for Hadoop
- Data Type Reference for Hadoop Overview
- Oracle
- Microsoft SQL Server
- DB2 for Linux, UNIX, and Windows
- Sybase ASE
- Flat File
- Hive
- HDFS
- Hadoop HDFS
- JDBC Connection
Appendix D: Glossary
- Glossary of Terms

User Guide

10.5.2
- 10.5.3
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0

Back Next

Creating a Data Set

Run a plan to create a data set. Create and run different plans for data generation operations.

You cannot create a data set if the tables contain columns with user-defined data types. If a table contains columns with user-defined data types, mark the columns as restricted before you include the tables in a plan to create a data set. To mark a column as restricted, open the project and click

Discover

Columns

and select the option

Restricted

for the required column.

Verify whether the tables in the data source contain primary keys. Create logical keys in all transactional tables that do not have any primary keys. A reset operation might fail if a transactional table with null values does not have primary keys.

If a table contains more than two character columns with a precision of 4000 each, edit the precision to 3,999 or less. A reset operation might fail if a data set contains more than two character columns with a precision of 4,000 each and you do not disable constraints during the reset operation.

You cannot perform

self-service portal

tasks that require a staging connection if a table in the

data pack

contains columns with a precision of 4,000 or greater. Edit the precision to 3,999 or less.

Open a project and add the required data sources and the project components.

Optional. Create parameters to use in the plan. You can use project parameters and global parameters.

To create a plan, click

Execute

and then click

Actions

New

Enter a name and optional description for the plan. Click

Add and configure the required components.

Configure the plan settings.

To create a data set, select

Test Data Warehouse

from the list of target connections.

The

Data Set Properties

section appears.

Enter a data set name. TDM enters the plan name by default. You can edit the name.

When you enter a data set name that does not exist in the test data warehouse, you create the first version of a data set. TDM enters the data set version number.

Enter an application name and application version to identify the data set.

Add tags to the data set. Use a comma to separate tags.

To delete a tag that you enter, click the

icon that appears with the tag. To use the keyboard to delete a tag, press

Tab

Shift+Tab

to select the tag and then press

Delete

from the keyboard.

Configure the error and recovery settings and advanced settings as required.

Optionally, configure an override strategy for a data source or a table.

To save the plan, click