Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Big Data Management
  3. Connections
  4. Mappings in the Hadoop Environment
  5. Mapping Objects in the Hadoop Environment
  6. Monitoring Mappings in the Hadoop Environment
  7. Mappings in the Native Environment
  8. Profiles
  9. Native Environment Optimization
  10. Data Type Reference
  11. Function Reference
  12. Parameter Reference
  13. Multiple Blaze Instances on a Cluster

Blaze Engine Logs

Blaze Engine Logs

The mapping run log appears in the LDTM log on the domain and in the tasklet logs on the Hadoop cluster.
You can find information about the mapping run on the Blaze engine in the following log files:
LDTM log
The LDTM logs the results of the mapping run on the Blaze engine. You can view the LDTM log from the Developer tool or the Monitoring tool for a mapping job.
You can configure the Data Integration Service to log details about the mapping execution to the session log. To enable logging of LDTM mapping execution details, set the log tracing level to verbose initialization or verbose data. Mapping execution details include the following information:
  • Start time, end time, and state of each task
  • Blaze Job Monitor URL
  • Number of total, succeeded, and failed/cancelled tasklets
  • Number of processed and rejected rows for sources and targets
  • Data errors, if any, for transformations in each executed segment
Blaze component and tasklet logs
The Blaze engine stores tasklet and Blaze component log events in temporary and permanent directories on the Hadoop cluster. The log file directories are specified by properties in the hadoopEnv.properties file located in the following location for each Hadoop distribution:
<Informatica Installation directory>/services/shared/hadoop/<distribution directory>/infaConf
The temporary directory is specified by the following property in the hadoopEnv.properties file:
infagrid.node.local.root.log.dir
. An administrator must create a directory with read, write, and execute permissions on all nodes on the Hadoop cluster.
For example, configure the following path for the property:
infagrid.node.local.root.log.dir=$HADOOP_NODE_INFA_HOME/dtmLogs
After the mapping completes, the Data Integration Service moves the tasklet log events from the temporary directory to a permanent directory on HDFS. The permanent directory is specified by the following property in the hadoopEnv.properties file:
infacal.hadoop.logs.directory
.
For example, configure the following path for the property:
infacal.hadoop.logs.directory=/var/log/hadoop-yarn/apps/informatica
If you want to retain the tasklet logs in the temporary directory, set the value of the following property in the hadoopEnv.properties file to false:
infagrid.delete.local.log
If you do not configure the temporary or permanent directories, the tasklet log events appear in the directory configured for the DTM Process. You can get the directory for the DTM Process from the value for the
yarn.nodemanager.local-dirs
property in yarn-site.xml on the cluster node.
The following sample code describes the
yarn.nodemanager.local-dirs
property:
<property> <name>yarn.nodemanager.local-dirs</name> <value>/var/lib/hadoop-yarn/cache/${user.name}/nm-local-dir</value> <description>List of directories to store local files.</description> </property>


Updated July 03, 2018