Install Python for the Python transformation on Hadoop

Back Next

Install Jep for streaming mappings

Streaming mappings require Jep as part of the Python installation. If you run the Python transformation in a streaming mapping, complete the following tasks to install Jep:

Set JAVA_HOME

Jep has a dependency on JDK. Before you install Jep, you must set the environment variable

$JAVA_HOME

. The Informatica domain contains an installation of Java, so you can use the JDK that is packaged in the Informatica domain.

Run the following command to set

$JAVA_HOME

to the packaged JDK:

export JAVA_HOME=$INFA_HOME/java

Install Jep

Run the following command to change directories to

$PYTHONHOME

cd $PYTHONHOME

Run the following command to install Jep:

bin/pip3 install jep

Copy Python to the Informatica domain

Run the following command to remove all files from the following directory in the Informatica domain:

rm -rf $INFA_HOME/services/shared/spark/python/*

Run the following command to copy the Python installation from

$PYTHONHOME

to the Informatica domain:

cp -r $PYTHONHOME/* $INFA_HOME/services/shared/spark/python/

Integrate Jep with the Hadoop connection

To integrate Jep with the Hadoop connection, complete the following tasks in the Administrator tool:

Navigate to

Manage > Connections

Select the Hadoop connection that connects to the Hadoop cluster.

Edit the

Spark configuration

Edit the

Advanced Properties

Configure the following advanced properties:

infaspark.pythontx.executorEnv.LD_PRELOAD: Location of the Python shared library. For example, set the property to the following value:

$INFA_HOME/services/shared/spark/python/lib/libpython3.6m.so
infaspark.pythontx.executorEnv.PYTHONHOME: Location of the Python installation directory. Keep the following default value:

$INFA_HOME/services/shared/spark/python/
infaspark.pythontx.submit.lib.JEP_HOME: Location of the Jep package. For example, set the property to the following value:

$INFA_HOME/services/shared/spark/python/lib/python3.6/site-packages/jep/

Rename Saved Search

Table of Contents

Install Python for the Python transformation on Hadoop

Install Python for the Python transformation on Hadoop

Install Jep for streaming mappings

Install Jep for streaming mappings

Set JAVA_HOME

Install Jep

Copy Python to the Informatica domain

Integrate Jep with the Hadoop connection