Table of Contents

Search

  1. Preface
  2. Mappings
  3. Mapplets
  4. Mapping Parameters
  5. Mapping Outputs
  6. Generate a Mapping from an SQL Query
  7. Dynamic Mappings
  8. How to Develop and Run a Dynamic Mapping
  9. Dynamic Mapping Use Cases
  10. Mapping Administration
  11. Export to PowerCenter
  12. Import From PowerCenter
  13. Performance Tuning
  14. Pushdown Optimization
  15. Partitioned Mappings
  16. Developer Tool Naming Conventions

Developer Mapping Guide

Developer Mapping Guide

Commands for Partitioned File Targets

Commands for Partitioned File Targets

When a flat file data object has a command output type, you can use a command to process target data for a single partition or to process merge data for all target partitions in a mapping. The Data Integration Service sends the data to a command or to a merge command instead of a flat file or a merge file.
Configure the command that processes data for partitions in the
Advanced
properties for the flat file data object. Find the command properties in the
Runtime: Write
section.
The following image shows a flat file data object configured to use a command to process target data for a single partition:
The output type is set to command, and the merge type is set to no merge. The following command is entered for the Command property: cat >> C:\data\MyOutput.txt
  1. Command output type
  2. Command options
On UNIX, use any valid UNIX command or shell script. On Windows, use any valid DOS or batch file.
You can use a command to process the following types of target data:
Data for a single partition
Enter a command that each writer thread runs separately. Each thread sends the target data to the command when the mapping runs. Each thread runs the same command with a different set of data.
When you enter the command, you must consider the operating system on which the mapping runs. For example, if you enter the command
cat > /log/data/myoutput.txt
, multiple threads write to the same file which could cause an operating system error. If you enter the command
cat >> /log/data/myoutput.txt
, multiple threads append data to the same file which is less likely to cause an operating system error.
To send the target data for a single partition to a command, select command for the
Output Type
property, and select no merge for the
Merge Type
property. Enter a command for the
Command
property.
Merge data for all partitions
Enter a merge command that processes the merge data for all writer threads. The Data Integration Service must use a concurrent merge type for a command to process merge data. Each thread concurrently sends target data to the merge command when the mapping runs. The merge command runs once with all of the data. The command might not maintain the order of the target data.
To send merge data for all partitions to a merge command, select command for the
Output Type
property, and select concurrent for the
Merge Type
property. Enter a command for the
Merge Command
property.

0 COMMENTS

We’d like to hear from you!