Cache partitioning creates a separate cache for each partition that processes an Aggregator, Joiner, Rank, or Sorter transformation. During cache partitioning, each partition stores different data in a separate cache. Each cache contains the rows needed by that partition. Cache partitioning optimizes mapping performance because each thread queries a separate cache in parallel.

If the cache size is smaller than the amount of memory required to run the transformation, transformation threads write to the cache directory to store overflow values in cache files. When multiple threads write to a single directory, the mapping might encounter a bottleneck due to I/O contention. An I/O contention can occur when threads write data to the file system at the same time. When you configure multiple cache directories, the Data Integration Service determines the cache directory for each transformation thread in a round-robin fashion.

In an Aggregator, Joiner, or Rank transformation, configure the cache directories in the

Cache Directory

advanced property. Use the default CacheDir system parameter value if an administrator entered multiple directories separated by semicolons for the

Cache Directory

property for the Data Integration Service in the Administrator tool. Or, you can enter a different value to configure multiple cache directories specific to the transformation.

In a Sorter transformation, configure the cache directories in the

Work Directory

advanced property. Use the default TempDir system parameter value if an administrator entered multiple directories separated by semicolons for the

Temporary Directories

property for the Data Integration Service in the Administrator tool. Or, you can enter a different value to configure multiple cache directories specific to the transformation.