Operations Guide

10.2
- 10.5 HotFix 2
- 10.5 HotFix 1
- 10.5
- 10.2 HotFix 1
- 10.1 HotFix 1
- 10.1
- 10.0 HotFix 1
- 10.0

Back Next

Overview

The Update Synchronizer is a background process that applies updates to MDM Registry Edition Tables and Indexes to keep them synchronized with changes to User Source Data. It can also compare the contents of the IDT/IDX against the User Source Data and report any differences.

IDTs created with the

SYNC

option can be synchronized with User Source Data.

User Source Data

User Source Data is held in an SQL database but does not have to be. It might be loaded into MDM Registry Edition from a sequential file (known as a Flat-File in MDM Registry Edition terminology).

When the User Source Data is held in a database that MDM Registry Edition can directly access using SQL, the data are said to reside in User Source Tables (UST).

Synchronization against a UST includes the following tasks:

reading transactions from an SQL accessible table known as the Transaction Table.

accessing User Source Data with SQL

applying updates to the IDTs and IDXs

Synchronization against a Flat-File source includes the following tasks:

reading transactions from a Flat-File

applying updates to the IDTs and IDXs

Supplying Transactions

Transaction Data might be read from a Transaction Table or from a Flat-File.

Source Access - Transaction Table: A Transaction Table is an SQL accessible table named
IDS_UPD_SYNC_TXN
held in the source database. It holds information about inserts, updates, and deletions from USTs.
The information in the table records the order in which these events occurred, together with primary key values of the affected source rows.
MDM Registry Edition permits the separation of USTs and IDTs on different databases. All updates to USTs are logged in a Transaction Table residing in the source database to prevent distributed database updates when source rows are modified.

Transactions can be added to the table in two ways:

Transactions added by Triggers: By default, database triggers are attached to the USTs by the Table Loader before source extraction. The triggers automatically insert transactions into the Transaction Table when UST updates occur. Triggers are a reliable method of transaction creation because the DBMS ensures that triggers are fired whenever updates occur.

Most databases do not fire triggers when the source table is maintained by using a mass-load utility. This results in a loss of synchronization.

Oracle does not fire triggers under certain circumstances, such as the addition of records to the source tables by SQL*Loader when using the
DIRECT-PATH
facility.

Microsoft SQL Server: If you are using Microsoft SQL Server DTS to bulk load records, clear the
Use Fast Load
option (enabled by default) under the
Options
tab of the Data Transformation Task property sheet. If the operation is performed using bcp’s
BULK INSERT
statement, specify the
FIRE_TRIGGERS
options.
Transactions added Manually: Some OEM developers prefer not to rely on triggers. Instead they want to directly insert transactions into the Transaction Table at suitable points in their application logic. The disabling of trigger creation is achieved by setting the
Txn-Source
clause to a value of
MANUAL
. The creation of valid transactions then becomes the user’s responsibility.

As an aid, the Table Loader still generates trigger code but instead of attaching the triggers to the USTs, it writes their source code to the Table Loader’s log file. The user must perform the equivalent actions as the trigger code when inserting transactions into the Transaction Table. Any deviation from the order of transaction creation or content will result in incorrect synchronization results.

Informatica Corporation reserves the right to change the trigger format / content at any time. Using the manual trigger option exposes you to the possibility that you might have to change your code. Some degree of independence is afforded by not directly inserting transactions into the transaction table. Instead, call the
IDS_UPDATE_SYNC
package to do this (as the automatically generated trigger code does). The trigger code gathers the required data and passes it to the package for formatting and insertion into the Transaction Table.

No Source Access (NSA)

When access to the source database is not possible, the synchronization method is known as No Source Access. In this situation, the transaction data must contain all the information required to add or delete records from the IDT without referring to any source data. In other words, the transactions must contain complete IDT records. They can be read from either an operating system file known as the Flat-File or from a database table (NSA Transaction Table).

Flat-File: A "flat-file" contains records in IDT format so that the Synchronizer can directly add (delete) them to (from) the IDT. Of course, the Synchronizer also updates the IDXs to reflect the changes there as well.

If you plan to synchronize using flat files the UST must be sourced from a flat file as well. See the
sourced_from
clause in the
DESIGN GUIDE
for the appropriate syntax. See the Transaction File section for more details about the Flat-File layout.
NSA Transaction Table: There is an alternative to providing IDT rows in a Flat-File. The Synchronizer can also read transactions from an SQL table known as the NSA Transaction Table. It is similar in content to a Flat-File. However, it has the advantage that it does not need to be "closed" before passing it to the Synchronizer for processing. See the Transaction File section for more details about the NSA Transaction Table.

Synchronizer Process

The Update Synchronizer process updates the IDT database. At startup, it connects to one of the following components:

All source databases used by the specified System

A flat transaction file specified by the -f parameter

The target database (when using an NSA TransactionTable)

It periodically polls for work by reading the transaction table on each source database. This is known as a duty cycle. A duty cycle can begin in one of two ways:

a specified period of time has elapsed since the last duty cycle (

-t

parameter), or

a new duty cycle commences immediately (without sleeping) if the previous cycle processed any transactions.

It processes a maximum of

Rate

transactions for each duty cycle for each source database before committing the results. The default

Rate

of 100 can be changed using the

-m

parameter. This prevents any one source database from monopolizing all of the Synchronizer’s time at the expense of less active source databases.

If the only source is a flat transaction file, the Synchronizer shuts down automatically when it reaches EOF.

Although designed to be a near real-time process, delays in synchronization are possible for multiple reasons:

USTs are updated while the IDT is still being loaded (that is, the MDM-RE-ID Table and Indexes do not exist yet)

the USTs and IDTs are on different databases and the network link is down.

the Synchronizer process is not running while updates occur.

In these situations, any updates to the USTs are logged and reapplied at a later stage (when using a Transaction Table).

Synchronizer Utilities

You can use the following Update Synchronizer utilities:

updmulti: You can use the
updmulti
utility to synchronize with an IDT in the following scenarios:
If the IDT uses triggers as the transaction source
If you apply updates to the IDT by using the Real Time API or the Real Time Web Service

The
updmulti
utility improves the synchronizer performance when it handles many IDT updates.
updsync: The updsync utility is deprecated, and Informatica recommends that you use the updmulti utility to synchronize IDTs.

Rename Saved Search

Table of Contents

Operations Guide

Operations Guide

Overview

Overview

User Source Data

Supplying Transactions

No Source Access (NSA)

Synchronizer Process

Synchronizer Utilities