Table of Contents

Search

  1. Preface
  2. Monitoring jobs
  3. Monitoring Data Integration jobs
  4. Data Integration job log files
  5. Monitoring Mass Ingestion jobs
  6. Monitoring Data Accelerator for Azure jobs
  7. Monitoring Data Profiling jobs
  8. Monitoring imports and exports
  9. Monitoring file transfer jobs
  10. Monitoring elastic clusters
  11. Monitoring source control logs

Monitor

Monitor

Cluster statuses

Cluster statuses

For each cluster that you view on the
Elastic Clusters
page, you can view the status of the cluster.
A cluster can be in one of the following statuses:
Status
Description
Starting
The cluster is starting. A cluster starts as soon as you run an
elastic job
.
Running
The cluster is running. The
Serverless Spark engine
is processing
elastic jobs
on the cluster.
Stopping
The cluster is stopping. The
elastic jobs
that were running on the cluster have completed and the cluster has reached the idle timeout in the
elastic configuration
, or you recently stopped the cluster in Monitor.
The time that it takes to stop a cluster depends on the cloud platform. If you run an
elastic job
while the cluster is stopping, the cluster does not start and the job fails.
Stopped
The cluster has stopped.
Error
The cluster has an error. During an error, the Secure Agent attempts to recover the cluster.
User action might be necessary, such as when you receive a fail-to-start or fail-to-stop exception.
Unknown
The status of the cluster is unknown.
If the status is unknown, verify that the Secure Agent is running. If the agent is not running, enable the agent and check whether the cluster starts running.
If the cluster does not start running, an administrator can run the command to list clusters. If the command output returns the cluster state as partial or in-use, the administrator can run the command to delete the cluster.
For more information about the commands, see
Administrator
in the Administrator help.
Monitor might not reflect the current cluster status if the following conditions are true:
  • The Secure Agent machine is shut down.
  • You update the
    elastic configuration
    , choosing to disable the
    elastic cluster
    when you save the configuration.
There is also a delay between the time that the cluster status changes and the time that the agent receives information about the cluster status. So the agent might submit a job to the cluster while the cluster is stopping or stopped. The job fails, and you must run the job again to restart the cluster.
For example, if the agent is notified that the cluster is running and the cluster reaches its idle timeout immediately afterwards, the agent submits the job to the cluster and the job fails.


Updated August 03, 2020