Table of Contents

Search

  1. Preface
  2. Introduction to PowerExchange for HDFS
  3. PowerExchange for HDFS Configuration
  4. HDFS Connections
  5. HDFS Data Objects
  6. HDFS Data Extraction
  7. HDFS Data Load
  8. HDFS Mappings
  9. Appendix A: Data Type Reference

PowerExchange for HDFS User Guide

PowerExchange for HDFS User Guide

Custom Formats Configuration

Custom Formats Configuration

Before you use custom formats, you must complete configuration tasks in the Informatica environment.
To use custom formats in the native environment, copy the .jar files that implement the custom formats to the following directory:
<Informatica installation directory>/services/shared/hadoop/<hadoop distribution name>/infaLib
To use custom formats in the Hadoop environment, see the Hadoop documentation for information about the prerequisite tasks.
If the custom compression includes native libraries, depending on the run-time environment, add the path of the native libraries to the environment variable $LD_LIBRARY_PATH or to the Hadoop connection. If you use the native environment, add the path of the native libraries to the environment variable $LD_LIBRARY_PATH. If you use the Hadoop environment, add the path of the native libraries to the Hadoop connection.
Perform the following steps to add the path of the native libraries to the Hadoop connection:
  1. Click the
    Common Attributes
    tab in the Hadoop connection.
  2. Under the
    Common Properties
    section, click
    Edit
    next to the
    Advanced Properties
    field.
  3. Add the infapdo.java.opts property and set its value to the path of the native libraries.
    For example, the following property specifies a native library path for a Cloudera distribution:
    infapdo.java.opts=-Djava.library.path=$HADOOP_NODE_INFA_HOME/services/shared/bin:$HADOOP_NODE_INFA_HOME/services/shared/hadoop/CDH_5.13/lib/native
    If you use Hortonworks or MapR distributions, change the native library path based on the distribution.

0 COMMENTS

We’d like to hear from you!