Table of Contents


  1. Preface
  2. Monitoring jobs
  3. Monitoring Data Integration jobs
  4. Data Integration job log files
  5. Monitoring Mass Ingestion jobs
  6. Monitoring Data Accelerator for Azure jobs
  7. Monitoring Data Profiling jobs
  8. Monitoring imports and exports
  9. Monitoring file transfer jobs
  10. Monitoring elastic clusters
  11. Monitoring source control logs



Data Integration job log files

Data Integration
job log files

Data Integration
generates log files to help you monitor running, failed, and completed jobs. You can access some of the log files from the
All Jobs
Running Jobs
, and
My Jobs
pages, and from the job details.
Data Integration
generates the following types of log files:
Error rows file
Data Integration
generates error rows files for
task and
task instances. An error rows file shows the rows that failed and the reason why each row failed. The error rows file includes the first 50 fields of a source error row.
For example, the following error appears in the error rows file when the task tries to insert two records with the same external ID into a Salesforce target:
Error loading into target [HouseholdProduct__c] : Error received from Fields [ExternalId__c]. Status code [DUPLICATE_VALUE]. Message [Duplicate external id specified: 1.0].
Session log file
Data Integration
generates a session log file for each job. If a job fails, download the log file to help you troubleshoot the job.
Reject file
Data Integration
creates a reject file for each flat file and Oracle target in a mapping or mapping task that contains error rows. The reject file contains information about each rejected target row and the reason that the row was rejected.
Data Integration
saves the reject file to the following default folder:
$PMBadFileDir/<task federated ID>
Execution plan
An execution plan shows the runtime Scala code that the
Serverless Spark engine
uses to run the data logic in an
elastic mapping
. You can use the Scala code to debug issues in the mapping.
Agent job log
An agent job log shows the logic that the Secure Agent uses to push the Spark execution workflow for an
elastic job
to an
elastic cluster
for processing.
Spark driver and Spark executor logs
Spark driver and Spark executor logs show the logic that the
Serverless Spark engine
uses to run an
elastic job
Initialization script log
If an initialization script runs on the cluster, the init script log shows the script output.
Cloud-init log
If an initialization script runs on the cluster, the cloud-init log contains information about how cluster nodes were initialized and bootstrapped. You can use the cloud-init log to check if any init scripts failed to run.
You can view the cloud-init log only in an AWS environment.
Spark event log
The Spark event log streams runtime events for an
elastic job

Updated February 12, 2021