PowerExchange for Hadoop User Guide for PowerCenter

PowerExchange for Hadoop User Guide for PowerCenter

Session Properties for a Hadoop Source

Session Properties for a Hadoop Source

You can configure a session for a Hadoop source to set staging and partitioning properties.
The following table describes the properties that you can configure for a Hadoop source:
Session Property
Description
Is Staged
Before reading the source file, the Integration Service stages the remote file or files locally.
Staged File Name
The local file name directory where the Integration Service stages files. If you use direct staging, the Integration Service stages the source file using this file name. If you use indirect staging, the Integration Service uses the file name to create the indirect file.
Concurrent read partitioning
Order in which multiple partitions read input rows from a source file. You can choose one of the following options:
  • Optimize throughput
    . The Integration Service does not preserve row order when multiple partitions read from a single file source. Use this option if the order in which multiple partitions read from a file source is not important.
  • Keep relative input row order
    . The Integration Service preserves the input row order for the rows read by each partition. Use this option to preserve the sort order of the input rows read by each partition.
  • Keep absolute input row order
    . The Integration Service preserves the input row order for all rows read by all partitions. Use this option to preserve the sort order of the input rows each time the session runs. In a pass-through mapping with passive transformations, the order of the rows written to the target will be in the same order as the input rows.
File Path
The Hadoop directory path of the flat file source. The path can be relative or absolute. If relative, it is relative to the home directory of the Hadoop user.
For example, you might specify the following value in the
File Path
property:
\home\foo\

0 COMMENTS

We’d like to hear from you!