To achieve optimal performance when using multiple threads to read from a flat file, configure the flat file data object to optimize throughput instead of preserving row order.
Consider the following solution to reduce bottlenecks for partitioned flat file sources:
Configure concurrent read partitioning for the flat file data object to optimize throughput.
In the flat file data object advanced properties, set the
Concurrent Read Partitioning
property to optimize throughput. When you optimize throughput, the Data Integration Service does not preserve row order because it does not read the rows in the file or file list sequentially.