Data Quality Performance Tuning Guide

Data Quality Performance Tuning Guide

Performance Guidelines for Address Validation

Performance Guidelines for Address Validation

Consider the following rules and guidelines when you configure your system for address validation:
  • You can store the reference data on a fast hard disk, solid-state disk, or even a flash disk (high-speed USB stick).
  • Where possible, install sufficient memory to allow all databases to fully pre-load into memory.
  • Preload at least the databases of frequently used countries. At a minimum, the available memory should equal the aggregate size of the most often-used country databases plus 256 MB.
  • If you will use reference data from all countries simultaneously, add memory to cover the size of the databases.
  • Use a 64-bit environment to preload more than 3 GB of reference data.
  • Do not set a country code as a No Preload value. Enter a No Preload value of ALL to avoid using a country code.
  • Minimize the access latency (average access time).
  • If you use a solid-state disk, do not preload the databases. Set a LARGE cache size in the Content Management Service instead.
  • Do not use the same drive to store address reference data and source or target files.
  • When enough memory is available, processor speed directly determines the speed of address processing.
  • Try to sort your address records by country or postcode prior to processing. Validation also benefits from internal and operating system caches for sorted addresses as opposed to addresses in random order.
  • The Max Thread Count value must be greater than or equal to the number of partitions.
  • Configure the Execution Instances property on the Address Validator transformation in conjunction with the Max Thread Count value.

0 COMMENTS

We’d like to hear from you!