Advanced Clusters

Back Next

Running a job

To run a job, the Secure Agent, master node, and worker nodes access sources and targets, as well as the staging, log, and initialization script locations.

The following image shows the process that the Secure Agent and cluster nodes use to run the job:

The following steps describe the process that the Secure Agent and cluster nodes use to run the job:

The worker nodes use the worker role to access source and target data.

The Secure Agent uses the Secure Agent role to store job dependencies in the staging location.

The worker nodes use the worker role to get job dependencies and stage temporary data in the staging location.

The master node uses the master role to orchestrate processes on the cluster.

The master node uses the master role to access and run the initialization script on the master node and to scale up the worker nodes. The added worker node uses the worker role to access the initialization script again to run the script on the worker node.

The worker nodes use the worker role to store logs in the log location.

The Secure Agent uses the Secure Agent role to upload the agent job log to the log location.

If you create master and worker roles and service accounts, the master and worker nodes use their respective roles. Otherwise, the master and worker nodes use the Secure Agent role.

Rename Saved Search

Table of Contents

Advanced Clusters

Advanced Clusters

Running a job

Running a job