English
Cloud Platform All Products

Table of Contents

Search

  1. Preface
  2. Monitoring jobs
  3. Monitoring Data Integration jobs
  4. Data Integration job log files
  5. Monitoring Mass Ingestion jobs
  6. Monitoring Data Accelerator for Azure jobs
  7. Monitoring Data Profiling jobs
  8. Monitoring imports and exports
  9. Monitoring file transfer jobs
  10. Monitoring elastic clusters
  11. Monitoring source control logs

Monitor

Monitor

Back Next

Viewing log files for elastic jobs

Viewing log files for
elastic jobs

When you monitor an
elastic job
, you can view the execution plan, the session log, the agent job log, and the Spark driver and Spark executor logs. If an initialization script runs on the cluster, you can also view the init script logs and cloud-init logs.
To access a specific log file for an
elastic job
, you might have to download the log from Monitor or from your cloud platform.
The following table indicates whether you can download each log from Monitor, your cloud platform, or both:
Log
Download location
Execution plan
Monitor
Session log
Monitor
Agent job log
Monitor, cloud platform¹
Spark driver log
Monitor, cloud platform¹
Spark executor log
Monitor², cloud platform¹
Initialization script log
Cloud platform
Cloud-init log
Cloud platform
Spark event log
Cloud platform
1
If you use a serverless runtime environment, downloading from your cloud platform is not available.
2
If you do not use a serverless runtime environment, downloading from Monitor is not available. Use the displayed log location to download the log from your cloud platform.

Downloading the execution plan and session log

You can download the execution plan and the session log from the job results in Monitor.

Downloading advanced logs

You can download the agent job log and the Spark driver log in the Spark task details in Monitor. You can also copy the advanced log location and navigate to the location on your cloud platform. You must navigate to the advanced log location to download Spark executor logs.
If you use a serverless runtime environment, you can download the advanced logs from Monitor. The
Download
 option becomes available when the job completes.

Downloading init script and cloud-init logs

To download an initialization script log or a cloud-init log, navigate to the following location on your cloud platform: 
<Log location>/ClusterLogs/<cluster ID>/<timestamp>/<cluster node instance ID>/
The log location is the location for log files that is specified in the
elastic configuration
.
You can download cloud-init logs only in an AWS environment.

Downloading Spark event logs

To download the Spark event log, navigate to the following location: 
<Log location>/eventLogs/atscaleagent
The log location is the location for log files that is specified in the
elastic configuration
.
If the job fails to start, you see the following sample text in the event log file: 
{"Event":"SparkListenerLogStart","Spark Version":"3.0.0"}
For more information about the job failure, see the Spark driver log.

Rules and guidelines

Refer to the following rules and guidelines when you view log files:
  • If you download the Spark driver and agent job logs while the job is running, the files reflect the current state of the job.
  • If you view a log file for an
    elastic mapping
     with a mapping name that begins with a number or a space, the mapping name is prefixed with an underscore in the log file.
    For example, when a log file references a mapping named
    123Mapping
    , the log files use the name
    _123Mapping
    .

Updated January 22, 2021

Download Guide
Send Feedback
Explore Informatica Network
Communities
Knowledge Base
Success Portal