Table of Contents

Search

  1. Preface
  2. Part 1: Hadoop Integration
  3. Part 2: Databricks Integration
  4. Appendix A: Connections

Create Blaze Engine Directories

Create Blaze Engine Directories

Create a blaze user account and directories required by the Blaze engine.
Complete the following tasks to prepare the Hadoop cluster for the Blaze engine:
Create a home directory for the blaze user.
If you created a blaze user, create home directory for the blaze user. For example,
hdfs hadoop fs -mkdir /user/blaze hdfs hadoop fs -chown blaze:blaze /user/blaze
If you did not create a blaze user, the Hadoop impersonation user is the default user.
Optionally, create a local services log directory.
By default, the Blaze engine writes the service logs to the YARN distributed cache. For example, run the following command:
mkdir -p /opt/informatica/blazeLogs
$HADOOP_NODE_INFA_HOME gets set to the YARN distributed cache. If you create a directory, you must update the value of the advanced property in the Hadoop connection.
Create an aggregated HDFS log directory.
Create a log directory on HDFS to contain aggregated logs for local services. For example:
hadoop fs -mkdir -p /var/log/hadoop-yarn/apps/informatica
Ensure that value of the advanced property in the Hadoop connection matches the directory that you created.
Optionally, create a Blaze staging directory.
You can write the logs to the Informatica Hadoop staging directory, or you can create a Blaze staging directory. If you do not want to use the default location, create a staging directory on the HDFS. For example:
hadoop fs -mkdir -p /blaze/workdir
If you do not create a staging directory, clear the Blaze staging directory property value in the Hadoop connection and the Data Integration Service uses the HDFS directory
/tmp/blaze_<user name>
.
Grant permissions on the local services log directory, aggregated HDFS log directory, and the staging directory.
Grant permission to the following users:
  • Blaze user
  • Hadoop impersonation user
  • Mapping impersonation users
If the blaze user does not have permission, the Blaze engine uses a different user, based on the cluster security and the mapping impersonation configuration. Blaze users must also have write permissions on
/tmp
.