Data Quality Performance Tuning Guide

10.5
- 10.5.4
- 10.5.3
- 10.5.2
- 10.5.1
- 10.4.1

Back Next

Association

If you run the Association transformation on a large data set, the transformation may not be able to store all associated records in memory, and some records will be written to disk. The Cache File Size property on the transformation specifies the amount of memory available.

The following image shows the property:

You can view the Cache File Size property on the Advanced tab in the Association transformation.

A cache size value below 65536 represents megabytes, and any higher value represents bytes.

The Cache File Directory identifies a storage area for the temporary files that the association operations create. Configure the cache directory on the smallest, fastest disk for performance improvements.

B-Tree Considerations

The Association transformation makes extensive use of B-tree file-based storage. Each column that the transformation reads has its own B-tree, and a general B-tree is used to store all input data rows. The Informatica B-tree is space-efficient but not compressed.

Use the following formulas to determine the needs of the transformation:

Association transformation column size:

Total volume of data for each column + 20 bytes for each input row

On-disk runtime cost of the general storage cache:

Size of input data set + 10 bytes for each row

Maximum internal memory map for association IDs and data rows:

Number of rows + 20 bytes

You cannot partition the Association transformation.

Data Quality Transformations

Download Guide

Watch

Comments