Data Engineering Administrator Guide

Back Next

Scenarios Where Recovery is Possible

The Data Integration Service can recover queued or running jobs upon node failure in the following scenarios:

The job has been taken from the queue but is not yet submitted to the cluster.: Depending on the timing of node failure, the status might be Queued or Running. The Data Integration Service fails the job over to the restored node or another node and continues processing it from the point of interruption.
The job is partially submitted to the cluster.: The status of tasks sent to the cluster is Running, and the status of tasks in the queue remains Queued. The Data Integration Service identifies unsent tasks and sends them to the restored node or another node, which processes the unsent tasks and sends them to the cluster for processing.
The job is fully submitted to the cluster.: The Data Integration Service has completed its job processing, and node failure at this point has no effect on the queue or on Hadoop processing. The Data Integration Service fails over to a restored node or another node to continue communication with Hadoop and to monitor job status.

If the Data Integration Service runs on a single node, the recovery process begins when the node is running again.