If the session uses one flat file in the master pipeline and multiple flat files in the detail pipeline, use one partition for the master source and
n
partitions for the detail file sources (1:
n
). Add a pass-through partition point at the detail Source Qualifier transformation. Do not add a partition point at the Joiner transformation. The Integration Service maintains the sort order when you create one partition for the master source because it does not redistribute sorted data among partitions.
When you have multiple files in the detail pipeline that have the same structure, use the following guidelines to pass the files to the Joiner transformation:
Configure the mapping with one source and one Source Qualifier transformation in each pipeline.
Specify the path and file name for each flat file in the Properties settings of the Transformations view on the Mapping tab of the session properties.
Each file must use the same file properties as configured in the source definition.
The range of sorted data in the flat files can overlap. You do not need to use a unique range of data for each file.
When you sort file data with 1:
n
partitioning, the Joiner transformation might output unsorted data based on the join type. If you use a full outer or detail outer join, the Integration Service processes unmatched master rows last, which can result in unsorted data.
The following image shows sorted file data joined with 1:n partitioning:
