When you import Hive data objects, the precision of string ports is set to 4000 by default. Aggregator and Rank transformations are automatically converted to sorted Aggregator and sorted Rank transformations. The sorted Aggregator and Rank transformations have a variable length cache which reduces the size of generated cache. The variable length cache eliminates the need to manually tune precision for string ports and reduces the chance of mapping failures due to inadequate disk space.

Performance best practice: String ports with large precision require a larger buffer memory for data processing, which reduces the number of rows that are processed per block. When performance is critical, tune string ports to avoid large precision.

Mismatched port data types and mismatched port precisions increase the computational overhead. Ensure that the port precisions and data types are consistent across sources, transformations, and targets within a mapping.

The following image shows a mismatch between the data type and the precision values in some of the ports in the Read transformation and the Expression transformation:

When processing cache-based transformations, such as Aggregator, Joiner, Sorter, and Lookup transformations, the Data Integration Service stores conditional values and output values in the transformation cache.

If the memory allocated to the cache-based transformation is not sufficient, the cache data is spilled or written to disk. To avoid data spill to disk, set explicit cache sizes for each Lookup, Joiner, Sorter, and Aggregator transformation in a mapping.

In the Developer tool, select a transformation and set numeric values in bytes for the following run-time properties in the

Advanced

tab: