Preface
Data Replication Overview
- Product Overview
- Data Replication Usage Scenarios
- Data Replication Sources and Targets
- Data Replication Architecture and Components
- Stages of Replication Processing
- Alternative Deployment Topologies
- Replication Configurations
Understanding Data Replication
- Overview
- Replicating Source Tables That Do Not Have a Primary Key Definition
- Start Point for the Extractor Task
- Apply Processing
- Checkpointing
- Recovery
  - Recovery Tables
- Tcl and SQL Scripts for Advanced InitialSync and Applier Processing of Data
- Database Character Set Conversion
- Replication of Database-Generated Values
- Generated Virtual Indexes
- Datatype Conversion Rules
- DDL Replication
Sources - Preparation and Replication Considerations
- DB2 for Linux, UNIX, and Windows Sources
- Microsoft SQL Server Sources
- MySQL Sources
- Oracle Sources
Targets - Preparation and Replication Considerations
- Amazon Redshift Targets
- Apache Kafka Targets
  - Preparing Apache Kafka Target Systems
  - Replication Considerations for Apache Kafka Targets
- Cloudera and Hortonworks Targets
  - Preparing Cloudera and Hortonworks Target Systems
- DB2 for Linux, UNIX, and Windows Targets
- Greenplum Targets
- MemSQL Targets
- Microsoft SQL Server Targets
- MySQL Targets
- Netezza Targets
- Oracle Targets
- PostgreSQL Targets
- Teradata Targets
- Vertica Targets
Starting the Server Manager
- Overview
- Installing the Server Manager as a Service on Windows
- Starting the Server Manager as a Windows Service
- Starting the Server Manager as a Daemon on Linux or UNIX
- Manually Starting the Server Manager
- Stopping a Server Manager Service or Daemon
- Uninstalling the Server Manager Service on Windows
Getting Started with the Data Replication Console
- Data Replication Console Interface
- Starting the Data Replication Console
Defining and Managing Server Manager Main Servers and Subservers
- Server Manager Main Server and Subservers
- Defining the Main Server and Its Subservers
- Editing Connection Information for a Main Server or Subserver
- Editing Microsoft SQL Server Instance Settings
- Editing Properties for the Main Server or a Subserver
- Viewing Information About the Server Manager System
- Configuring the Server Manager for HTTPS Communication
- Configuring the Server Manager Main Server to Run with NAT
- Associating a Subserver with Another Main Server
- Deleting Subservers
Creating and Managing User Accounts
- User Account Overview
- Users and Privileges
- Server Manager Security Policies
- Creating a User Account
- Changing the Password for Your User Account - Replication User
- Changing the Password for a User Account - idradmin User
- Unlocking a User Account
- Resetting the Password for the idradmin Account
Creating and Managing Connections
- Connections Overview
- Creating a Source or Target Connection from the Server Manager Tab
- Editing a Source or Target Connection
- Assigning a Different Source or Target Connection to a Configuration
Creating Replication Configurations
- Replication Configuration
- Task Flow: Creating a Replication Configuration
- Defining the Source Database
  - Configuring a Connection to an Oracle ASM Instance
  - Configuring Connections to Oracle RAC Sources for High Availability
    - Connecting to an Oracle RAC by Using Custom Connection Strings
    - Connecting to an Oracle RAC by Using a Virtual IP Address or Host Name
- Defining the Target Database
- Generating Target Tables and Audit Log Tables
- Generating Avro Schemas for Apache Kafka Consumers
- Handling Source Tables with Long Table or Column Names
  - Strategies for Handling Long Table Names
    - Editing the Audit Log Table Suffix
    - Manually Editing Target Table Names in an SQL Script
  - Strategies for Handling Long Column Names
- Mapping Source and Target Tables
- Defining Source Table Indexes
- Enabling Replication of DDL Changes at the Schema and Table Levels
- Customizing Apply Settings for Target Tables
- Configuring the Start Points for Extractor and Applier Tasks
- Configuring Conflict Resolution
  - Example of Configuring a MAXIMUM Resolution Strategy for Update Conflicts
  - Example of Configuring Custom Conflict Resolution
- Customizing Column Mappings
  - Filtering Column Data
- Adding Tcl and SQL Expressions
- Specifying the Database Logs from Which to Extract Data
- Configuring Runtime Settings
- Configuring Message Logging
- Saving Replication Configurations to the Main Server Manager
Materializing Targets with InitialSync
- InitialSync Overview
- Source Connectivity for Data Unload Operations
- Target Connectivity for Data Load Operations
- Sync Point Value
- InitialSync Handling of Target Table Constraints
- Considerations for Running InitialSync
- Task Flow: Using InitialSync to Materialize a Target
Scheduling and Running Replication Tasks
- Methods of Running Replication Tasks
- Types of Replication Tasks
- Schedule and Task Statuses
- Conflicting Replication Tasks
- Running Replication Executables Manually from the Data Replication Console
- Scheduling Replication Tasks
Implementing Advanced Replication Topologies
- Advanced Replication Topologies
- Configuring Continuous Replication
- Configuring Data Replication from One Source to Multiple Targets
- Configuring Bidirectional Replication
- Configuring Cascade Replication
- Loopback Avoidance for Replicated Data
Monitoring Data Replication
- Types of Monitoring Information
- Replication Statistics
- Intermediate Files
- Task Execution Logs
- Server Manager Logs
  - Viewing Server Manager Logs
- User Notifications
- Skipped Transaction Records
- Managing Open Transactions
Managing Replication Configurations
- Configuration Management Tasks
- Switching to Read or Edit Mode for a Replication Configuration
- Editing a Replication Configuration
- Changing the Server Manager Associated with a Replication Configuration
- Clearing User Replication Settings
- Managing Database Supplemental Logging
- Deploying Replication Configurations
- Generating a Reverse-Replication Configuration
- Exporting a Configuration File
- Importing a Configuration File
- Cleaning Replication Processing Information for a Configuration
  - Performing a Clean Operation on a Configuration
- Viewing Earlier Revisions of a Replication Configuration
- Viewing a List of Processed Database Logs
Handling Replication Environment Changes and Failures
- Manually Changing the Source Table Structure After Running Data Replication
- Updating an Avro Schema for Kafka Targets After Running Data Replication
- Adding Table Mappings Manually After Running Data Replication
- Resuming Replication After Upgrading a DB2 for Linux, UNIX, and Windows Source Database
- Resuming Replication After Upgrading a Microsoft SQL Server Source Database
- Handling Applier Failures
Troubleshooting
- Collecting Diagnostic Data for Troubleshooting
- Common Replication Problems
Data Replication Files and Subdirectories
- Files and Subdirectories
- Data Replication Script Files
- Executables Called from the Data Replication Console or Scripts
- Default.cfg File
- Other Key Files
- Subdirectories
Data Replication Runtime Parameters
Command Line Parameters for Data Replication Components
- About Command Line Parameters
- Command Line Parameters for InitialSync
- Command Line Parameters for the Extractor
- Command Line Parameters for the Applier
- Command Line Parameters for the Server Manager
Updating Configurations in the Replication Configuration CLI
- Replication Configuration CLI Overview
- Updating Source and Target Metadata for a Replication Configuration in the CLI
  - Updating Source and Target Metadata for a Configuration in Interactive Mode
  - Updating Source and Target Metadata for a Configuration in Non-interactive Mode
- Replication Configuration CLI Commands
DDL Statements for Manually Creating Recovery Tables
Sample Scripts for Enabling or Disabling SQL Server Change Data Capture
- Microsoft SQL Server Enterprise Edition
Glossary
- Applier
- Applier task
- apply cycle
- Audit Apply
- audit log table
- bidirectional replication
- binary log
- calculated columns
- cascade replication
- change data capture
- checkpoint processing
- Command Line Interface
- configuration file
- continuous replication
- Copy File task
- data files
- data warehouse appliance
- Edit mode
- External task
- Extractor
- Extractor task
- Flat File target
- global transaction
- heterogeneous replication
- Informatica Data Replication Console
- initial materialization
- InitialSync
- InitialSync task
- intermediate files
- log coordinates
- loopback avoidance
- Merge Apply
- Microsoft SQL Server Backup task
- primary target
- CDC Publisher
- Read mode
- recovery table
- Replication Configuration Command Line Interface
- replication configuration file
- replication statistics
- replication tasks
- routing
- secondary target
- Send File task
- Server Manager
- Server Manager Command Line Interface
- Server Manager Main server
- Server Manager subserver
- SQL Apply
- SQL Script Engine
- staging table
- Start Point
- subtask threads
- supplemental logging
- Sync Point
- Tcl expression
- Tcl Script Engine
- transaction files
- transactional replication
- virtual column
- virtual index

User Guide

9.8.0

Back Next

Datatype Conversion Rules

The DataReplication_installation\uiconf\DataTypes.xml file defines rules that Data Replication uses to convert source datatypes to compatible target datatypes.

The conversion occurs when Data Replication generates target schema based on source schema or other target schema and when Data Replication replicates CREATE TABLE, ADD COLUMN, and MODIFY COLUMN changes.

The DataReplication_installation\uiconf\DataTypes.xml file contains a <ConversionRule> section for each combination of supported source and target types. The <ConversionRule> sections define the datatype conversion rules. You can edit the rules or add rules if necessary.

To understand how conversion rules are defined, review the following example <ConversionRule> sections in the DataTypes.xml file:

Example 1.
The following <ConversionRule> section defines a rule for converting the Oracle NUMBER datatype, which has a precision greater than 18 and a scale equal to 0, to the Netezza NUMERIC datatype with the same precision and scale:

<ConversionRule source_db="ORACLE" destination_db="NETEZZA">
  <DataTypes>
    <DataType>
      <OriginalDataType>NUMBER</OriginalDataType>
      <Condition><![CDATA[DataPrecision>18 && DataScale==0]]></Condition>
      <DataPrecision>DestinationDataPrecision=DataPrecision</DataPrecision>
      <DataScale>DestinationDataScale=DataScale</DataScale>
      <DestinationDataType>NUMERIC</DestinationDataType>
    </DataType>
  </DataTypes>
</ConversionRule>

Example 2.
The following <ConversionRule> section defines a rule for converting the Oracle VARCHAR datatype to the Teradata VARCHAR datatype:

<ConversionRule source_db="ORACLE" destination_db="TERADATA">
  </DataTypes>
    <DataType>
      <OriginalDataType>VARCHAR</OriginalDataType>
      <DestinationDataType>VARCHAR</DestinationDataType>
      <Calculation>DestinationDataSize=(DataSize+SrcMinCharsetSize-1)/SrcMinCharsetSize*TgtCharsetSize</Calculation>
    </DataType>
  </DataTypes>
</ConversionRule>

If the character sets of the source and target CHAR columns do not match, the target column size in bytes might differ from the source column size. In this case, Data Replication uses the following formula to calculate the target column size:

DestinationDataSize=(DataSize+SrcMinCharsetSize-1)/SrcMinCharsetSize*TgtCharsetSize

SrcMinCharsetSize

is the minimum number of bytes per character for the source character set.

TgtCharsetSize

is the number of bytes per character for the target character set. Data Replication gets the

SrcMinCharsetSize

and

TgtCharsetSize

values from the DataReplication_installation\uiconf\ICU_table.xml file.

Data Replication uses the formula that is defined by the

Calculation

parameter only if the character sets of the source and target columns do not match.

To add a conversion rule, add a <DataType> section under the <ConversionRule> section for the source and target type combination in the DataReplication_installation\uiconf\DataTypes.xml file. The <DataType> section has the following syntax:

<DataType>
  <OriginalDataType>source_column_datatype</OriginalDataType>
  <Condition>rule_condition<Condition>
  <DestinationDataType>target_column_datatype</DestinationDataType>
</DataType>

Replace the source_column_datatype, rule_condition, and target_column_datatype variables with the appropriate values.

Understanding Data Replication

Generating Target Tables and Audit Log Tables

CREATE TABLE and ADD COLUMN Operations

Enabling Replication of DDL Changes at the Schema and Table Levels

Customizing Apply Settings for Target Tables

Download Guide

Watch

Comments

Communities

Knowledge Base

Success Portal