Table of Contents

Search

  1. Preface
  2. Change Data Capture Introduction
  3. PowerExchange Listener
  4. PowerExchange Condense
  5. DB2 for i5/OS Change Data Capture
  6. Remote Logging of Data
  7. Introduction to Change Data Extraction
  8. Extracting Change Data
  9. Monitoring CDC Sessions
  10. Managing Change Data Extractions
  11. Tuning CDC Sessions
  12. Appendix A: DTL__CAPXTIMESTAMP Time Stamps

CDC Guide for i5/OS

CDC Guide for i5/OS

Commitment Control Attributes

Commitment Control Attributes

PWXPC, in conjunction with PowerExchange and the PowerCenter Integration Service, controls the timing of commit processing for CDC sessions based on commitment control attributes on PWX CDC connections.
Commit processing is not controlled by a single commitment control attribute. When setting these attributes, try to balance performance and resource consumption with latency requirements.
The
Maximum Rows Per commit
,
Real-Time Flush Latency in milli-seconds
, and
UOW Count
attributes control the timing of real-time flushes of change data to the targets. The
Minimum Rows Per commit
attribute controls if a commit can occur.
Set one or more of the following commitment control attributes on PWX CDC connections:
Maximum Rows Per commit
Maximum number of change records in a source UOW that PWXPC processes before flushing the data buffer to commit the change data to the targets.
Use this attribute to have PWXPC commit change data to the targets without waiting for the UOW boundary, or end-UOW, to be met. This type of commit is called a
subpacket commit
. By using subpacket commits for large UOWs, you can minimize use of storage on the PowerCenter Integration Service machine and locking contention on the target databases.
Because PWXPC can commit the change data to the targets between UOW boundaries, relational integrity (RI) might be compromised. Do not use this connection attribute if you have targets in the CDC session with RI constraints.
After the maximum rows limit is met, PWXPC flushes the change data from the buffer on the PowerCenter Integration Service machine and commits the data to the targets. PWXPC also writes message PWXPC_12128 to the session log. After commit processing completes, the RDBMS releases locks on the target databases and PowerCenter Integration Service can reuse the buffer space for additional change records.
The maximum rows limit is cumulative across all sources in the CDC session. PWXPC issues a real-time flush when the limit is met, regardless of the number of sources with changes.
PWXPC resets the maximum rows limit when a real-time flush occurs. The flush can occur because of the maximum rows limit, UOW count limit, or real-time flush latency timer.
If PWXPC reaches a UOW boundary and the maximum row limit has not been met, PWXPC continues to process change records across UOW boundaries.
Use a maximum rows limit if you have extremely large UOWs in the change stream that might cause the following problems:
  • Locking issues on the target database
  • Resource issues on the PowerCenter Integration Service node
For example, you have a large UOW with 10,000 updates for a single source, and you set the
Maximum Rows per Commit
attribute to 1000. In this case, PWXPC issues a subpacket commit after each 1,000 change records.
Or, you might have a UOW that contains updates for more than one source. For example, the UOW contains 900 updates for source 1, 100 updates for source 2, and then 500 more updates for source 1. If you set the
Maximum Rows per Commit
attribute to 1000, PWXPC issues a subpacket commit after reading 1,000 change records, or after processing the updates for source 2.
Default is 0, which causes PWXPC to not use this maximum rows limit. If you specify 0 or do not enter a value for the maximum rows limit, commits occur only on UOW boundaries.
If you specify a low maximum rows limit, the CDC session uses more system resources on the PowerCenter Integration Service machine and target systems. This increased resource use occurs because PWXPC flushes data to the targets more frequently.
The
Maximum Rows Per commit
attribute is a count of the records within a UOW. The
UOW Count
attribute is a count of complete UOWs.
Minimum Rows Per commit
Minimum number of change records that PowerExchange must pass to PWXPC before passing a commit record. Until the minimum rows limit is met, PowerExchange discards any commit records that it reads from the change stream and passes only change records to PWXPC. After the minimum rows limit is met, PowerExchange passes the next commit record it encounters to PWXPC and then resets the minimum rows counter.
If the change stream has many small UOWs, you can set the
Minimum Rows Per commit
attribute to create larger UOWs of a more uniform size. Online transactions that run in transaction control systems such as CICS and IMS often commit after only a few changes, which results in many, small UOWs in the change stream. PowerExchange and PWXPC process a few large UOWs more efficiently than many small UOWs. By using the minimum rows limit to increase the size of UOWs, you can improve CDC processing efficiency.
The minimum rows limit does not impact the relational integrity of the change data because PowerExchange does not create additional commit points in the change data. PowerExchange skips some of the original commit records in the change stream.
Default is 0, which causes PowerExchange to not use this minimum rows limit.
If you enter a minimum rows limit, PowerExchange changes the number of change records in a UOW to match or exceed this limit.
PWXPC does not commit change data to the targets based on the minimum rows limit. PWXPC commits change data to the targets based on the
Maximum Rows Per commit
,
Real-Time Flush Latency in milli-seconds
, and
UOW Count
attributes.
Real-Time Flush Latency in milli-seconds
For real-time or continuous extraction mode, the number of milliseconds that must elapse before PWXPC flushes the data buffer to commit change data to the targets. After the flush latency interval expires and PWXPC reaches a UOW boundary, PWXPC issues a real-time flush to commit change data and restart tokens to the targets. PWXPC also writes message PWXPC_10082 in the session log.
PWXPC resets the flush latency interval when a real-time flush occurs. The flush can occur because of the maximum rows limit, UOW count limit, or real-time flush latency timer.
Valid values for the real-time flush latency are:
  • -1. Disables data flushes based on time.
  • 0 through 2000. Sets the interval to 2000 milliseconds, or 2 seconds.
  • 2000 through 86400. Sets the interval to the specified number of milliseconds.
Default is 0.
If you set the flush latency interval value is 0 or greater, PWXPC flushes the change data for all complete UOWs after the interval expires and the next UOW boundary occurs. The lower you set the flush latency interval, the faster PWXPC commits change data to the targets. If you require a low latency for applying changes to the targets, enter a low value for the flush latency interval.
However, if you specify a low flush latency interval, the CDC session might consume more system resources on the PowerCenter Integration Service and target systems. This increased consumption occurs because PWXPC commits change data to the targets more frequently.
UOW Count
Number of complete UOWs that PWXPC reads from the change stream before flushing change data to the targets. When PWXPC reads change data from PowerExchange and provides it to the source qualifier in the CDC session, the count of the UOWs begins.
After the UOW count limit is met, PWXPC issues a real-time flush to commit the change data and restart tokens to the targets. PWXPC also writes message PWXPC_10081 in the session log.
PWXPC resets the UOW count after a real-time flush occurs because of the UOW count limit or the real-time flush latency interval.
Valid values for UOW count are:
  • -1 or 0. PWXPC does not use the
    UOW Count
    attribute to control commit processing.
  • 1 through 999999999. PWXPC flushes change data after reading the specified number of UOWs.
Default is 1.
The lower you set the UOW count value, the faster the PowerCenter Integration Service commits change data to the target. If you require the lowest possible latency, enter a UOW count of 1. However, a low latency might result in the session using more system resources on the PowerCenter Integration Service and the target systems.
In the session properties, verify that the
Commit Type
attribute specifies
Source
and that the
Commit at End of File
attribute is disabled. The
Commit at End of File
attribute is enabled by default. If you accept the default, the PowerCenter Integration Service writes additional data to the targets after the CDC reader has committed the restart tokens and shut down. When you restart the CDC session, the session might write duplicate data to the targets.  

0 COMMENTS

We’d like to hear from you!