Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Big Data Management
  3. Mappings in the Hadoop Environment
  4. Mapping Sources in the Hadoop Environment
  5. Mapping Targets in the Hadoop Environment
  6. Mapping Transformations in the Hadoop Environment
  7. Processing Hierarchical Data on the Spark Engine
  8. Configuring Transformations to Process Hierarchical Data
  9. Processing Unstructured and Semi-structured Data with an Intelligent Structure Model
  10. Stateful Computing on the Spark Engine
  11. Monitoring Mappings in the Hadoop Environment
  12. Mappings in the Native Environment
  13. Profiles
  14. Native Environment Optimization
  15. Cluster Workflows
  16. Connections
  17. Data Type Reference
  18. Function Reference
  19. Parameter Reference

Big Data Management User Guide

Big Data Management User Guide

Blaze Engine Monitoring

Blaze Engine Monitoring

You can monitor statistics and view log events for a Blaze engine mapping job in the Monitor tab of the Administrator tool. You can also monitor mapping jobs for the Blaze engine in the Blaze Job Monitor web application.
The following image shows the Monitor tab in the Administrator tool:
The Monitor tab is selected in the Administrator tool. The Execution Statistics view is selected, and the navigator shows Ad Hoc Jobs selected on the left. A list of jobs appears in the contents panel.
The Monitor tab has the following views:

Summary Statistics

Use the
Summary Statistics
view to view graphical summaries of object states and distribution across the Data Integration Services. You can also view graphs of the memory and CPU that the Data Integration Services used to run the objects.

Execution Statistics

Use the
Execution Statistics
view to monitor properties, run-time statistics, and run-time reports. In the Navigator, you can expand a Data Integration Service to monitor
Ad Hoc Jobs
or expand an application to monitor deployed mapping jobs or workflows
When you select
Ad Hoc Jobs
, deployed mapping jobs, or workflows from an application in the Navigator of the
Execution Statistics
view, a list of jobs appears in the contents panel. The contents panel displays jobs that are in the queued, running, completed, failed, aborted, and cancelled state. The Data Integration Service submits jobs in the queued state to the cluster when enough resources are available.
The contents panel groups related jobs based on the job type. You can expand a job type to view the related jobs under it.
Access the following views in the
Execution Statistics
view:
Properties
The
Properties
view shows the general properties about the selected job such as name, job type, user who started the job, and start time of the job. You can also monitor jobs on the Hadoop cluster from the Monitoring URL that appears for the mapping in the general properties. The Monitoring URL opens the Blaze Job Monitor in a web page. The Blaze Job Monitor displays detailed monitoring statistics for a mapping such as the number of grid tasks, grid segments, or tasklets, and recovery attempts for each tasklet.
Blaze Execution Plan
The Blaze execution plan displays the Blaze engine script that the Data Integration Service generates based on the mapping logic. The execution plan includes the tasks that the script depends on. Each script has a unique identifier.
Summary Statistics
The
Summary Statistics
view appears in the details panel when you select a mapping job in the contents panel. The
Summary Statistics
view displays throughput and resource usage statistics for the job.
You can view the following throughput statistics for the job:
  • Source. The name of the mapping source file.
  • Target name. The name of the target file.
  • Rows. The number of rows read for source and target. If the target is Hive, this is the only summary statistic available.
  • Average Rows/Sec. Average number of rows read per second for source and target.
  • Bytes. Number of bytes read for source and target.
  • Average Bytes/Sec. Average number of bytes read per second for source and target.
  • First Row Accessed. The date and time when the Data Integration Service started reading the first row in the source file.
  • Dropped rows. Number of source rows that the Data Integration Service did not read.
Detailed Statistics
The
Detailed Statistics
view appears in the details panel when you select a mapping job in the contents panel. The
Detailed Statistics
view displays graphs of the throughput and resource usage statistics for the job run.

0 COMMENTS

We’d like to hear from you!