PowerExchange for Hadoop User Guide for PowerCenter

10.5
- 10.5.7
- 10.5.6
- 10.5.4
- 10.4.0

Back Next

Hadoop Target Partitioning

When you configure a session to load data to a Hadoop target, you can write the target output to a separate file for each partition or to a merge file that contains the target output for all partitions.

You can select the following merge types for Hadoop target partitioning:

No Merge: The Integration Service generates one target file for each partition. If you stage the files, the Integration Service transfers the target files to the remote location at the end of the session. If you do not stage the files, the Integration Service generates the target files at the remote location.
Sequential Merge: The Integration Service creates one output file for each partition. At the end of the session, the Integration Service merges the individual output files into a merge file, deletes the individual output files, and transfers the merge file to the remote location.; If you set the merge type to sequential, you need to define the merge file path and the output file path in the session properties. The merge file path determines the final Hadoop target location where the Integration Service creates the merge file. The Integration Service creates the merge file from the intermediate merge file output in the location defined for the output file path.

Rename Saved Search

Table of Contents

PowerExchange for Hadoop User Guide for PowerCenter

PowerExchange for Hadoop User Guide for PowerCenter

Hadoop Target Partitioning

Hadoop Target Partitioning