When a flat file data object has a command output type, you can use a command to process target data for a single partition or to process merge data for all target partitions in a mapping. The Data Integration Service sends the data to a command or to a merge command instead of a flat file or a merge file.
Configure the command that processes data for partitions in the
Advanced
properties for the flat file data object. Find the command properties in the
Runtime: Write
section.
The following image shows a flat file data object configured to use a command to process target data for a single partition:
Command output type
Command options
On UNIX, use any valid UNIX command or shell script. On Windows, use any valid DOS or batch file.
You can use a command to process the following types of target data:
Data for a single partition
Enter a command that each writer thread runs separately. Each thread sends the target data to the command when the mapping runs. Each thread runs the same command with a different set of data.
When you enter the command, you must consider the operating system on which the mapping runs. For example, if you enter the command
cat > /log/data/myoutput.txt
, multiple threads write to the same file which could cause an operating system error. If you enter the command
cat >> /log/data/myoutput.txt
, multiple threads append data to the same file which is less likely to cause an operating system error.
To send the target data for a single partition to a command, select command for the
Output Type
property, and select no merge for the
Merge Type
property. Enter a command for the
Command
property.
Merge data for all partitions
Enter a merge command that processes the merge data for all writer threads. The Data Integration Service must use a concurrent merge type for a command to process merge data. Each thread concurrently sends target data to the merge command when the mapping runs. The merge command runs once with all of the data. The command might not maintain the order of the target data.
To send merge data for all partitions to a merge command, select command for the