A pipeline Lookup transformation is enabled for caching by default. You can partition the lookup source to improve performance when the Integration Service builds the lookup cache. The Lookup transformation begins processing rows when the lookup source is cached.
When you configure a pipeline Lookup transformation, the lookup source and source qualifier are in a different pipeline from the Lookup transformation. The pipeline is a partial pipeline because it contains no target. The Integration Service reads the source data in the partial pipeline. You can create multiple partitions in the pipeline to improve processing performance.
The Integration Service passes source data from the partial pipeline to the other pipeline when it builds the cache. When the number of partitions in the partial pipeline is different from the number of partitions for the Lookup transformation, the Integration Service creates a partition point. If the Lookup transformation has a hash auto-keys partition point, the Integration Service creates the same number of partitions in the cache as in the Lookup transformation. Otherwise the cache has one partition.
The following figure shows the partitions for a session that contains a pipeline Lookup transformation and a Source Qualifier lookup source:
The Integration Service processes the Employee rows in three partitions. The pipeline containing the Lookup transformation has four partitions. Since the Lookup transformation has a hash auto-key partition point, the cache is partitioned into four partitions.