Bulk Data Movement Guide

10.4.0
- 10.5.6
- 10.5.5
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.5
- 10.4.1
- 10.4.0 HotFix 1

Back Next

Multithreaded Processing Statistics

If you use bulk data offload processing, you can also use multithreaded processing to attempt to increase throughput on the PowerCenter Integration Service machine where the offloaded processing runs.

To issue these messages, you must specify the SHOW_THREAD_PERF statement in the DBMOVER configuration file on the PowerCenter Integration Service machine:

SHOW_THREAD_PERF=number_of_records

This statement specifies the number of records PowerExchange must process before writing statistics messages about multithreaded extraction processing to the PowerExchange message log file. For more information about this statement, see the

PowerExchange Reference Manual

Also, you must specify 1 or greater for the

Worker Threads

attribute on the application connection to implement multithreaded processing so that statistics can be generated.

PowerExchange writes some or all of the following messages during each statistics interval:

PWX-31255.
Cycle time, which is the total time that PowerExchange on the Integration Service machine spent processing the bulk data before passing it to PWXPC. This message includes the total percentage of time and average, minimum, and maximum times in microseconds.

PWX-31256.
I/O time, which is the time that PowerExchange on the Integration Service machine spent reading bulk data from the PowerExchange Listener on the source system. When reading local flat files, the I/O time is the time that PowerExchange spent reading the file. This message includes the I/O percentage of the total time and average, minimum, and maximum times in microseconds.

PWX-31257.
Parsing time, which is the time that PowerExchange on the Integration Service machine spent in column-level processing for the bulk data records on all threads. This message includes the parsing percentage of the total time and average, minimum, and maximum times in microseconds.

PWX-31258.
External time, which is the time that PowerExchange on the Integration Service machine spent combining the bulk data records from all threads back into a single stream to pass to PWXPC and for PWXPC to flush the data to PowerCenter. This message includes the external percentage of the total time and average, minimum, and maximum times in microseconds.

PWX-31259.
Delay time, which is the time that the PowerExchange on the Integration Service machine waited to receive new bulk data records to process from the PowerExchange Listener on the source system. This message includes the delay percentage of the total time and average, minimum, and maximum times in microseconds.

PWX-31261
. PowerExchange joiner thread I/O time. If you use partitioning, this message is included and shows the amount of time spent by the joiner performing I/O during a multi-threaded session. When not performing I/O, the joiner is waiting for the partitions to complete processing.

PWX-31262
. Thread I/O delay. If you use partitioning, this message is included and shows the average time that each partition spends waiting for the reader to provide data to parse.

For example, if you specify SHOW_THREAD_PERF=10000 in the DBMOVER configuration file, PowerExchange writes the following sample messages after 10,000 bulk data records have been read:

PWX-31254 PowerExchange threading stats for last 10000 rows. Cycle (array) size is 25 rows. 0 out of array occured.
PWX-31255 Cycle  time: 100% (avg:      5709 min:     4741 max:      7996 usecs)
PWX-31256 IO     time:   4% (avg:       235 min:       51 max:      1021 usecs)
PWX-31257 Parse  time:  79% (avg:      4551 min:     4102 max:      5495 usecs)
PWX-31258 Extern time:  20% (avg:      1145 min:      618 max:      3287 usecs)
PWX-31259 Delay  time:   0% (avg:         7 min:        4 max:       165 usecs)
PWX-31254 PowerExchange threading stats for last 100000 rows. Cycle (array) size is 25 rows. 0 out of array occured.
PWX-31255 Cycle  time:  99% (avg:      5706 min:     4735 max:      7790 usecs)
PWX-31256 IO     time:   4% (avg:       234 min:       51 max:       950 usecs)
PWX-31257 Parse  time:  79% (avg:      4549 min:     4108 max:      5425 usecs)
PWX-31258 Extern time:  20% (avg:      1144 min:      616 max:      3242 usecs)
PWX-31259 Delay  time:   0% (avg:         7 min:        4 max:       115 usecs)

If the parsing and external processing times are greater than the I/O time, you might be able to improve throughput by increasing the number of threads for the bulk data movement session.

Rename Saved Search

Table of Contents

Bulk Data Movement Guide

Bulk Data Movement Guide

Multithreaded Processing Statistics

Multithreaded Processing Statistics