User Guide

1.0
- 1.3
- 1.2
- 1.1

Back Next

Handling Changes to Source Tables and Extraction Maps

PowerExchange CDC Publisher does not update the Avro schemas if the source table structure and extraction map definition change. The extractions map definition must change if the source columns or fields for which change data is extracted change, for example, columns are added or removed.

Use the following procedure to add or remove source columns or fields and resume PowerExchange CDC Publisher processing.

On the source system, stop change activity (inserts, updates, and deletes) on the source table.

Verify that any change data that was captured based on the old source table definition has been extracted from the PowerExchange Logger for Linux, UNIX, and Windows log files.

Shut down the PwxCDCPublisher process by using the PwxCDCAdmin SHUTDOWN command. For more information, see Shutting Down the CDC Publisher.

You can issue the command from the

pwxcdcpub_root_installation\bin

subdirectory or from another location if you specified the PWXPUB_HOME environment variable. For more information, see Shutting Down the CDC Publisher.

Shut down the PowerExchange Logger for Linux, UNIX, and Windows.

For more information, see the

PowerExchange CDC Guide for Linux, UNIX, and Windows

In the PowerExchange Navigator, open the original capture registration and set its status to

History

For more information, see the

PowerExchange Navigator User Guide

Alter the source table or object to add or remove columns.

In the PowerExchange Navigator, re-create the capture registration and associated extraction map for the source table. Activate the capture registration.

If you set any PowerExchange CDC Publisher properties that refer to the mapname_tablename portion of the extraction map name, update the properties to refer to the mapname_tablename portion of the regenerated extraction map name.

The extraction map name has the format xnninstance.
mapname_tablename
.

For example, perform this step if you defined the Extract.captureConsumerExcludeNameList and Extract.captureConsumerIncludeNameList filter properties.

On the PowerExchange CDC Publisher system, restart the PwxCDCPublisher process using the RESTART=FROM_END parameter. For more information, see Restarting a PowerExchange CDC Publisher Change Data Stream.

If you restart the CDC Publisher using the default restart behavior, the FROM_BEGINNING option, or the FROM_CONFIG option, the PowerExchange CDC Publisher issues an error when it tries to process change records based on the previous schema.

To clear the old Avro schemas from cache, enter the PwxCDCAdmin command with the CLEAR=FORMAT and TABLE parameters at the command prompt.

The TABLE parameter is required. Include the optional INSTANCE parameter if you are using an instance other than the default "instanceA" at

/pwxcdcpub_root installation/instanceA

Use the following syntax:

PwxCDCAdmin CLEAR=FORMAT TABLE={ALL|mapname_tablename} [INSTANCE=override_instance_directory]

In this syntax, mapname_tablename is the map name and table name portion of the new extraction map name, where the extraction map name has the format xnninstance.
mapname_tablename
. For more information about the command syntax, see PwxCDCAdmin Utility - Command and Parameters.

The command produces output such as the following example output:

  INSTANCE = instanceA. Connecting to host:port USHOST1:26987
  ------------------------------------------------------------------------------
COMMAND       RESULT  RESULT CODE  DESCRIPTION
CLEAR_FORMAT          10           The number of Avro Schema definitions cleared

To retrieve extraction map information for all source tables from PowerExchange again, including the regenerated extraction map for the updated table, enter the PwxCDCAdmin command with the RESET=FORMAT parameter at the command prompt. Include the optional INSTANCE parameter if you are using an instance other than the default "instanceA" at

/pwxcdcpub_root installation/instanceA

Use the following syntax:

PwxCDCAdmin RESET=FORMAT [INSTANCE=override_instance_directory]

Alternatively, you can wait until the CDC Publisher first receives change data for the updated table. At that point, the CDC Publisher automatically caches the extraction map and generates the Avro schema.

If you specified the RESET=FORMAT command and want to force the generation of the updated Avro schema so that you can verify that it properly reflects the changed table structure, enter the PwxCDCAdmin command with the REPORT=FORMAT and TABLE parameters.

The TABLE parameter is required. Include the optional INSTANCE parameter if you are using an instance other than the default "instanceA" at

/pwxcdcpub_root installation/instanceA

Use the following syntax:

PwxCDCAdmin [REPORT=FORMAT TABLE={ALL|mapname_tablename}][INSTANCE=override_instance_directory]

In this syntax, mapname_tablename is the map name and table name portion of the new extraction map name.

This command generates new Avro schema in the

\instance\reports

subdirectory. The schema is generated in a legible format. The generated schema can be used for verification purposes and by consumer applications.

Rename Saved Search

Table of Contents

User Guide

User Guide

Handling Changes to Source Tables and Extraction Maps

Handling Changes to Source Tables and Extraction Maps