Advanced Clusters

Back Next

Create user-defined master and worker roles

Create user-defined master and worker roles to fine-tune permissions for the master and worker nodes in an

advanced cluster

. The nodes use the permissions to run the Spark applications in an

advanced job

. After you complete these tasks, you can specify the master and worker instance profiles in an

advanced configuration

If you're looking for a quick setup, you can use default master and worker roles. For more information, see Use default master and worker roles (alternative) and Master and worker role types reference.

To create user-defined roles, complete the following tasks:

Create the master and worker roles.

Create master policies.

Create worker policies.

Attach the policies to the master and worker roles.

Allow the cluster operator role to assume the worker role.

Allow the cluster operator role to assume the master role.

The master and worker roles, the instance profiles, and the cluster operator role must be defined under the same AWS account.

When the Secure Agent starts the

advanced cluster

, the agent uses the cluster operator role to validate whether the instance profiles exist and whether the master and worker roles have access to required cluster directories, such as staging, log, and initialization script locations. If validation fails, the cluster fails to be created.

Step 7. Create IAM roles

Create the master and worker roles

Create master policies

Create worker policies

Attach the policies to the master and worker roles

Allow the cluster operator role to assume the worker role

Allow the cluster operator role to assume the master role

Use default master and worker roles (alternative)

Download Guide

Watch

Comments

Communities