Table of Contents

Search

  1. Preface
  2. Performance Tuning Overview
  3. Target Optimization
  4. Source Optimization
  5. Transformation Optimization
  6. Mapping Optimization
  7. Partitioned Mapping Optimization
  8. Run-time Optimization
  9. SQL Data Service Optimization
  10. Web Service Optimization
  11. Connections Optimization
  12. Data Transformation Optimization

Performance Tuning Guide

Performance Tuning Guide

Optimize Flat File Targets for Partitioning

Optimize Flat File Targets for Partitioning

To achieve optimal performance when using multiple threads to write to a flat file, configure partitions to write the target output to separate files and configure multiple target directories.
Consider the following solutions to reduce bottlenecks for partitioned flat file targets:
Configure partitions to write the target output to separate files.
In the flat file data object advanced properties, set the
Merge Type
property to
No merge
. The Data Integration Service concurrently writes the target output to a separate file for each partition. If you require merged target data, the concurrent merge type optimizes performance more than the sequential merge type.
Configure multiple target directories.
When multiple threads write to a single directory, the mapping might encounter a bottleneck due to input/output (I/O) contention. An I/O contention can occur when threads write data to the file system at the same time. When you configure multiple directories, the Data Integration Service determines the output directory for each thread in a round-robin fashion.
Configure the output file directories in the advanced properties for the flat file data object. Use the default TargetDir system parameter value if an administrator entered multiple directories separated by semicolons for the
Target Directory
property for the Data Integration Service in the Administrator tool. Or, you can enter a different value to configure multiple output file directories specific to the flat file data object.

0 COMMENTS

We’d like to hear from you!