In pass-through partitioning, the Integration Service processes data without redistributing rows among partitions. Therefore, all rows in a single partition stay in that partition after crossing a pass-through partition point.
When you add a partition point to a pipeline, the master thread creates an additional pipeline stage. Use pass-through partitioning when you want to increase data throughput, but you do not want to increase the number of partitions.
You can specify pass-through partitioning at any valid partition point in a pipeline.
The following figure shows a mapping where pass-through partitioning can increase data throughput:
Reader Thread (First Stage).
Transformation Thread (Second Stage).
Writer Thread (Third Stage).
By default, this mapping contains partition points at the source qualifier and target instance. Since this mapping contains an XML target, you can configure only one partition at any partition point.
In this case, the master thread creates one reader thread to read data from the source, one transformation thread to process the data, and one writer thread to write data to the target. Each pipeline stage processes the rows as follows:
Because the pipeline contains three stages, the Integration Service can process three sets of rows concurrently.
If the Expression transformations are very complicated, processing the second (transformation) stage can take a long time and cause low data throughput. To improve performance, set a partition point at Expression transformation EXP_2 and set the partition type to pass-through. This creates an additional pipeline stage. The master thread creates an additional transformation thread:
Reader Thread (First Stage).
Transformation Thread (Second Stage).
Transformation Thread (Third Stage).
Writer Thread (Fourth Stage).
The Integration Service can now process four sets of rows concurrently as follows:
| FIL_1 & EXP_1 Transformations | EXP_2 & LKP_1 Transformatios | |
By adding an additional partition point at Expression transformation EXP_2, you replace one long running transformation stage with two shorter running transformation stages. Data throughput depends on the longest running stage. So in this case, data throughput increases.