Hadoop Integration Guide

Back Next

Create Blaze Engine Directories

Create a blaze user account and directories required by the Blaze engine.

Complete the following tasks to prepare the Hadoop cluster for the Blaze engine:

Create a home directory for the blaze user.: If you created a blaze user, create home directory for the blaze user. For example,
hdfs hadoop fs -mkdir /user/blaze hdfs hadoop fs -chown blaze:blaze /user/blaze
If you did not create a blaze user, the Hadoop impersonation user is the default user.
Optionally, create a local services log directory.: By default, the Blaze engine writes the service logs to the YARN distributed cache. For example, run the following command:
mkdir -p /opt/informatica/blazeLogs; The Blaze configuration advanced properties in the Hadoop connection contains the following custom property for the services log directory:
infagrid.node.local.root.log.dir=$HADOOP_NODE_INFA_HOME/blazeLogs; $HADOOP_NODE_INFA_HOME gets set to the YARN distributed cache. If you create a directory, you must update the value of the custom property in the Hadoop connection.

Create an aggregated HDFS log directory.: Create a log directory on HDFS to contain aggregated logs for local services. For example:
hadoop fs -mkdir -p /var/log/hadoop-yarn/apps/informatica; The Blaze configuration advanced properties in the Hadoop connection contains the following custom property for the HDFS log directory:
infacal.hadoop.logs.directory=/var/log/hadoop-yarn/apps/informatica; Ensure that value of the custom property in the Hadoop connection matches the directory that you created.
Optionally, create a Blaze staging directory.: You can write the logs to the Informatica Hadoop staging directory, or you can create a Blaze staging directory. If you do not want to use the default location, create a staging directory on the HDFS. For example:
hadoop fs -mkdir -p /blaze/workdir; If you do not create a staging directory, clear the Blaze staging directory property value in the Hadoop connection and the Data Integration Service uses the HDFS directory
/tmp/blaze_<user name>
.
Grant permissions on the local services log directory, aggregated HDFS log directory, and the staging directory.: Grant permission to the following users:
Blaze user
Hadoop impersonation user
Mapping impersonation users
If the blaze user does not have permission, the Blaze engine uses a different user, based on the cluster security and the mapping impersonation configuration.