Table of Contents

Search

  1. Preface
  2. Mappings
  3. Mapplets
  4. Mapping Parameters
  5. Mapping Outputs
  6. Generate a Mapping from an SQL Query
  7. Dynamic Mappings
  8. How to Develop and Run a Dynamic Mapping
  9. Dynamic Mapping Use Cases
  10. Mapping Administration
  11. Export to PowerCenter
  12. Import From PowerCenter
  13. Performance Tuning
  14. Pushdown Optimization
  15. Partitioned Mappings
  16. Developer Tool Naming Conventions

Developer Mapping Guide

Developer Mapping Guide

Optimize Output File Directories for Partitioned File Targets

Optimize Output File Directories for Partitioned File Targets

By default when a flat file data object has a file output type, each thread writes the target output to a separate file. For optimal performance when multiple threads write to a file target, configure multiple output file directories.
When multiple threads write to a single directory, the mapping might encounter a bottleneck due to input/output (I/O) contention. An I/O contention can occur when threads write data to the file system at the same time.
When you configure multiple directories, the Data Integration Service determines the output directory for each thread in a round-robin fashion. For example, you configure a flat file data object to use directoryA and directoryB as target directories. If the Data Integration Service uses four threads to write to the file target, the first and third writer threads write target files to directoryA. The second and fourth writer threads write target files to directoryB.
If the Data Integration Service does not use multiple threads to write to the target, the service writes the output file to the first listed directory.
Configure the output file directories in the
Advanced
properties for the flat file data object. Find the
Output File Directory
property in the
Runtime: Write
section. By default, the property is configured to use the system parameter value defined for the Data Integration Service. Use the default TargetDir system parameter value if an administrator entered multiple directories separated by semicolons for the
Target Directory
property for the Data Integration Service.
You can enter a different value to configure multiple output file directories specific to the flat file data object. Enter multiple directories separated by semicolons for the property or for the user-defined parameter assigned to the property.

0 COMMENTS

We’d like to hear from you!