Advanced Clusters

Back Next

Verify AWS subscriptions

Verify that you have the necessary AWS subscriptions to create an

advanced cluster

in an AWS environment.

You must have the following services on AWS:

Amazon Elastic Block Service (Amazon EBS): Amazon EBS volumes are attached to Amazon EC2 instances as local storage. The local storage is used to store information that the
Serverless Spark engine
needs to run
advanced jobs
. For example, local storage is used to store the content of the Spark image. The
Spark engine
also requires local storage to process data logic and to persist data during processing.
Amazon Elastic Compute Cloud (Amazon EC2): Amazon EC2 instances are launched to host an
advanced cluster
. One Amazon EC2 instance hosts the master node, and additional instances host the worker nodes.
Amazon EC2 Auto Scaling: Amazon EC2 Auto Scaling automatically adds or removes cluster nodes in the
advanced cluster
based on job-processing requirements.
Amazon Elastic Load Balancing (Amazon ELB): A load balancer accepts incoming
advanced jobs
from a Secure Agent and provides an entry point for the jobs to an
advanced cluster
.
Amazon Identity and Access Management (IAM): AWS IAM provides access control that you can use to specify which services and resources an
advanced cluster
can access in your AWS environment.
Amazon Route 53: Nodes in an
advanced cluster
communicate information with other nodes in the same cluster using Route 53.
Amazon Simple Storage Service (Amazon S3): An
advanced cluster
is staged in Amazon S3 buckets. Amazon S3 is also used to store logs that are generated for
advanced jobs
.