Developer Mapping Guide

Back Next

Concurrent Read Partitioning

To preserve row order when multiple threads read from a single file source, configure the

Concurrent Read Partitioning

property for a flat file data object to preserve the order.

Configure the

Concurrent Read Partitioning

property in the

Advanced

properties for the flat file data object. Find the property in the

Runtime: Read

section.

Select one of the following options for the

Concurrent Read Partitioning

property:

Optimize throughput: The Data Integration Service does not preserve row order when multiple partitions read from a single file source. Use this option if the order in which multiple partitions read from a file source is not important.

Default option.
Keep relative order: Preserves the sort order of the input rows read by each partition.

The following table shows an example sort order of a file source with 10 rows read by two partitions:

Partition
Rows Read

Partition #1
1,3,5,8,9

Partition #2
2,4,6,7,10
Keep absolute order: Preserves the sort order of all input rows read by all partitions. In a pass-through mapping with passive transformations, the order of the rows written to the target is in the same order as the input rows.

The following table shows an example sort order of a file source with 10 rows read by two partitions:

Partition
Rows Read

Partition #1
1,2,3,4,5

Partition #2
6,7,8,9,10

Partition	Rows Read
Partition #1	1,3,5,8,9
Partition #2	2,4,6,7,10

Partition	Rows Read
Partition #1	1,2,3,4,5
Partition #2	6,7,8,9,10