Bulk Data Movement Guide

10.4.0 HotFix 1
- 10.5.6
- 10.5.5
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0

Back Next

Tuning Partitioned Sessions

If partitioned bulk data movement sessions do not run as efficiently as expected, you can adjust tuning parameters to improve performance.

Use the following statements in the DBMOVER configuration file to tune partitioned sessions:

APPBUFSIZE statement: Configure the APPBUFSIZE statement in the DBMOVER configuration file on the PowerCenter Integration Service machine. This statement specifies the maximum buffer size, in bytes, that PowerExchange uses to read or write data. When the buffer reaches this maximum size, PowerExchange sends the buffer data across the network to the requesting system before starting to use another buffer. To help improve session performance, increase the buffer size.
If dynamic application buffer sizing is enabled, the APPBUFSIZE statement defines the initial size of the application data buffer for all connections made during a PowerExchange Listener run. PowerExchange resizes the application data buffer dynamically for individual connections as needed.
APPBUFSIZEDYN statement: Enable dynamic application buffer sizing on the PowerCenter Integration Service machine. For each connections to a data source, if dynamic application buffer sizing is enabled, PowerExchange resizes the application buffer if the data source contains records that are too large to fit into the buffer. Dynamic application buffer sizing is enabled by default. You can explicitly enable it by specifying APPBUFSIZEDYN=Y in the DBMOVER configuration file.
MAXTASKS statement: If processing slows or hangs for bulk data movement sessions that use reader or writer partitioning, increase the MAXTASKS value to help improve performance.

Use the following PowerCenter parameters to tune partitioned sessions:

Array Size connection attribute: Enter this attribute on the PWX Batch NRDB application connection in PowerCenter. This attribute specifies the storage array size, in number of records, that is shared across the partitions in the partitioned bulk data movement session. This attribute also pertains to worker threads for multithreading. To help improve session performance, increase the array size. Valid values are from 1 through 5000.
If you set the
Write Mode
connection attribute to
Confirm Write On
and the array size is greater than the APPBUFSIZE, PowerExchange sends the buffer across the network each time the number of records in the storage array is:
records=array size/partition_count
Default buffer block size attribute: Set this session configuration attribute to accommodate the partitioned bulk data movement session. This attribute specifies the default buffer block size that the PowerCenter Integration Service uses to move data and index caches from sources to targets. A low block size can reduce the performance gains from increasing the APPBUFSIZE or
Array Size
value. For example, if the block size accommodates only 10 records, the array size for each partition is limited to 10 records. If you set the
Write Mode
connection attribute to
Confirm Write On
, the APPBUFSIZE is also limited to 10 records.
When setting the block size, use the following criteria:
If you use
Confirm Write On
, enter a block size that can accommodate the number of records calculated as
array_size/partition_count
. Also, set the APPBUFSIZE to hold the same number of records.
If you use
Confirm Write Off
, enter a block size that can accommodate the number of records calculated as
array_size/partition_count
.

Also, if you do not use offload processing and resources on the source or target system are constrained, enable offload processing. This feature moves field-level processing of bulk data from the source or target system to the PowerCenter Integration Service machine.

Rename Saved Search

Table of Contents

Bulk Data Movement Guide

Bulk Data Movement Guide

Tuning Partitioned Sessions

Tuning Partitioned Sessions