Communities
A collaborative platform to connect and grow with like-minded Informaticans across the globe
Product Communities
Connect and collaborate with Informatica experts and champions
Discussions
Have a question? Start a Discussion and get immediate answers you are looking for
User Groups
Customer-organized groups that meet online and in-person. Join today to network, share ideas, and get tips on how to get the most out of Informatica
Get Started
Community Guidelines
Knowledge Center
Troubleshooting documents, product guides, how to videos, best practices, and more
Knowledge Base
One-stop self-service portal for solutions, FAQs, Whitepapers, How Tos, Videos, and more
Support TV
Video channel for step-by-step instructions to use our products, best practices, troubleshooting tips, and much more
Documentation
Information library of the latest product documents
Velocity (Best Practices)
Best practices and use cases from the Implementation team
Learn
Rich resources to help you leverage full capabilities of our products
Trainings
Role-based training programs for the best ROI
Certifications
Get certified on Informatica products. Free, Foundation, or Professional
Product Learning Paths
Free and unlimited modules based on your expertise level and journey
Resources
Library of content to help you leverage the best of Informatica products
Tech Tuesdays Webinars
Most popular webinars on product architecture, best practices, and more
Product Availability Matrix
Product Availability Matrix statements of Informatica products
SupportFlash
Monthly support newsletter
Support Documents
Informatica Support Guide and Statements, Quick Start Guides, and Cloud Product Description Schedule
Product Lifecycle
End of Life statements of Informatica products
Ideas
Events
Change Request Tracking
Marketplace
English
  • PowerExchange for CDC and Mainframe 10.5.5
  • All Products

Table of Contents

Search

  1. Preface
  2. Part 1: PowerExchange CDC Introduction
  3. Part 2: PowerExchange CDC Components
  4. Part 3: PowerExchange CDC Data Sources
  5. Part 4: Change Data Extraction
  6. Part 5: Monitoring and Tuning
  7. Appendix A: DTL__CAPXTIMESTAMP Time Stamps

CDC Guide for Linux, UNIX, and Windows

CDC Guide for Linux, UNIX, and Windows

Back Next

Dynamic Dictionary

Dynamic Dictionary

PowerExchange Express CDC for Oracle supports a dynamic dictionary based on a snapshot of the Oracle catalog to accommodate DDL changes to source tables and table partitions for restart processing. With the dynamic dictionary, Express CDC can update dictionary information in memory for the source tables of CDC interest when DDL changes are committed. The changes, along with the SCN and sequence number, are persisted to an external stage table or file that records all changes of interest.
Use a dynamic dictionary in environments where DDL changes to source tables are likely to occur. If you use a static dictionary in this type of environment, data loss or corruption can occur after a CDC restart if tables were reorganized or renamed, tables or partitions were dropped, or table columns were added, dropped, or altered. By using a dynamic dictionary, you can avoid data loss and corruption after a restart.
To use a dynamic dictionary, perform the following high-level implementation steps:
  1. Perform configuration tasks.
  2. Ensure that an initial snapshot of the Oracle catalog populates the state table or file for the dynamic dictionary. The initial snapshot is automatically created when PowerExchange Express CDC first initializes after dynamic dictionary configuration.
  3. Warm start PowerExchange Express CDC or cold start the PowerExchange Logger. PowerExchange Express CDC loads the in-memory dictionary from the external state storage during initialization of the warm start or cold start.
For more information, see the following sections.

Configuration Tasks

To use the dynamic dictionary, you must define both the DICTIONARY and STATESTORAGE statements in the PowerExchange Express CDC configuration file.
In the DICTIONARY statement, set the MODE parameter to DYNAMIC.
In the STATESTORAGE statement, define the storage that persists the dynamic dictionary information for each source table of CDC interest. You can configure Express CDC to store this information in a state table in the source database or in CCATALOG files in a file system directory. You can also specify a database user in the STATESTORAGE statement if the user requires permissions that are different from the user configured for normal access.
If you use file-based storage, the PowerExchange Express CDC user must have the appropriate permissions in the file system to create subdirectories and files. Ensure that the Express CDC user has read, write, and execute permissions on the base directory. The first time PowerExchange Express CDC starts, it creates one or more subdirectories under the base directory.
If you use a table to store dictionary state information, you can create the table in one of the following ways:
  • Let PowerExchange Express CDC create the table. The first time Express CDC initializes after you enable dynamic dictionary use, it attempts to create the state table if the table does not already exist. If the Express CDC user needs permissions different from the user configured for normal access, specify a database user in the STATESTORAGE statement that has database permissions that allow creation of the state table on the Oracle source system.
  • Manually create the table by using the sample DDL statements that are provided in DDL Statements for Manually Creating a State Table for the Dynamic Dictionary. Ensure that the database user has SELECT and UPDATE permissions on the table.

Initial Snapshot

After dynamic dictionary use is configured, the first time PowerExchange Express CDC initializes, it checks for the state table or file directory. If the table or directory does not exist, Express CDC attempts to create it. If the Express CDC user does not have create table permissions, initialization will fail.
After the table or file directory exists, Express CDC takes an in-memory snapshot of the Oracle catalog and externalizes it to populate the state table or file, thereby creating an initial snapshot.

Incremental Snapshot

After the dynamic dictionary is in use, any tables added to the CCT interest list by means of registration or a rename-in operation, trigger an incremental snapshot during Express CDC initialization. Express CDC takes an in-memory snapshot of the current catalog using the new interest list and compares the snapshot to the old interest list. Any tables that do not match in the old interest list are provided to the dynamic dictionary as "futures". Operations in the log for the tables marked "futures" will get filtered out until the commit SCN of the transaction that contains them matches or exceeds the current SCN at the time the tables were added to the dictionary.

Warm Start Processing

For a warm start of CDC processing, PowerExchange Express CDC uses the SCN in the data dictionary to calculate how far back in the logs to start CDC processing so that no DDL changes for tables of interest are missed.
A warm start Instantiates an in-memory catalog from the state table or file instead of from the Oracle catalog. However, Express CDC still references the Oracle catalog to verify the supplemental logging status for all tables of CDC interest and to check if new tables have been added to the interest list. If tables have been added, an incremental snapshot is taken.

Cold Start Processing

When the PowerExchange Logger is cold started and the dynamic dictionary last restart SCN is less than the PowerExchange Logger restart SCN, the PowerExchange Logger uses the dynamic dictionary restart SCN to capture changes so that the dictionary does not contain gaps.
You can modify this behavior by setting SNAPSHOTONCOLDSTART=Y in the DICTIONARY statement. This setting causes a new snapshot of the current Oracle catalog to be taken, replacing the existing information in the dynamic dictionary.

Usage Considerations

  • The structure of each source table of CDC interest must be consistent with its capture registration.
  • If a source table that is no longer of CDC interest is removed from the CCT interest list by deleting its registration or by editing the Status or Condense option in its registration, the dynamic dictionary continues to track changes for it. If the table is then added back to the interest list, PowerExchange Express CDC does not need to take another snapshot of the table.
  • Dynamic dictionary handles the following types of operations, which are not supported by a static dictionary:
    • Online reorganization of tables
      . Express CDC supports online reorg operations on tables of CDC interest without data loss or requiring a restart.
    • Rename in
      . If a table that is not in the dynamic dictionary is renamed to a name that matches the CCT interest list of source objects, Express CDC queries the Oracle catalog to get the table structure and persists that information to state storage. Express CDC then builds a new filter to include the table and compares it to the previous filter. If the table is marked as "unwanted" in the previous filter, CDC shuts down.
    • Rename out
      . If a table in the dynamic dictionary is renamed such that it no longer matches any entry in the CCT interest list, Express CDC stops capturing data for the table but continues processing.
    • Rollup
      . Dynamic dictionary automatically rolls up the base of the dictionary during initialization and once every 24 hours thereafter. All DDL operations for a table are merged into the base snapshot for that table, retaining only the amount of object history allowed by the retention period. During the rollup, information about dropped tables and partitions is deleted from the dictionary. If the retention period is too short, information might be lost for these tables.
    • Table or partition drop
      . A drop of a table or partition of CDC interest is recorded in the storage container with the commit SCN at the time the drop operation occurred. Since a warm start will rebuild the dictionary to the point-in-time of the restart SCN, the dictionary will still be aware of the object, even if the restart SCN is prior to the drop and no data will be lost.
  • PowerExchange Express CDC issues a warning if a table in the CCT interest list has no entry in the Oracle catalog. If the table is subsequently created or renamed in, it is marked as of CDC interest and included in the table filter. For tables that are renamed in, Express CDC queries the Oracle catalog for the table structure and persists that information to state storage. The new filter is compared to the previous filter. If the table is marked as unwanted for CDC in the old filter, CDC shuts down to avoid potential data loss. In this case, you must warm start Express CDC to refresh the log parser filter so that it does not discard data that is now of CDC interest.
  • If the PowerExchange Logger is cold started and the last SCN written to the dictionary is less than the Logger restart SCN, the dictionary SCN is used for the cold start. This behavior ensures that no operations are missing from the dictionary. To modify this behavior, you can set SNAPSHOTONCOLDSTART=Y to force a new snapshot of the current Oracle catalog to be taken, which replaces the existing dictionary contents.
Back to Top
0 COMMENTS
Back Next
We’d like to hear from you!