Administrator
- Administrator
- All Products
Property | Description |
---|---|
Name | Name of the
advanced configuration .
|
Description | Description of the
advanced configuration .
|
Runtime Environment | Runtime environment to associate with the
advanced configuration . The runtime environment can contain only one Secure Agent. A runtime environment cannot be associated with more than one configuration.
|
Cloud Platform | Cloud platform that hosts the cluster.
Select Google Cloud Platform (GCP). |
Private Cluster | Creates an
advanced cluster in which cluster resources have only private IP addresses.
When you choose to create a private cluster, you must specify the VPC and subnet in the advanced properties.
The Secure Agent must be
in the same VPC network or a VPC network that can connect to the VPC that you
specify in the advanced properties. |
Property | Description |
---|---|
Region | Region in which to create the cluster. Use the drop-down menu to view the regions that you can use.
|
Master Instance Type | Instance type to host the master node. Use the drop-down menu to view the
instance types that you can use. |
Master Service Account | Service account to attach to the master node. |
Worker Instance Type | Instance type to host the worker nodes. Use the drop-down menu to view the instance types that you can use.
|
Number of Worker Nodes | Number of worker nodes in the cluster. Specify the minimum and maximum number of worker nodes.
|
Worker Service Account | Service account to attach to the worker nodes. |
Availability Zones | List of
availability zones where cluster nodes are created. The master node is created
in the first availability zone in the list. If multiple zones are specified,
the cluster nodes are created across the specified zones. The zones
must be unique and be within the specified region. |
Disk
Size | Size of the persistent disk to attach to a
worker node for temporary storage during data processing. The disk size must be
between 50 GB and 16 TB. |
Cluster Shutdown | Cluster shutdown method. You can select one of the following cluster shutdown methods:
|
Mapping Task Timeout
| Amount of time to wait for a
mapping task to complete before it is terminated. By default, a
mapping task does not have a timeout.
If you specify a timeout, a value of at least 10 minutes is recommended. The timeout begins when the
mapping task is submitted to the Secure Agent.
|
Staging Location | Location on Google Cloud Storage for staging data. The location name
must start with gs:// . |
Log
Location | Location on Google Cloud Storage to
store logs that are generated when you run an
advanced job . The location name must start with gs:// .
|
Property | Description |
---|---|
VPC | Google Cloud
Virtual Private Cloud (VPC) in which to create the cluster. If you
choose to not create a private cluster, you do not need to specify a VPC. In
this case, the agent creates a VPC on your Google Cloud account based on the
region and the availability zones that you select. |
Subnet | Subnets in which to create cluster nodes. Use a comma-separated list to specify the subnets.
Required if a VPC is specified. Each subnet must be in a different
availability zone within the specified VPC. If you do not specify a
VPC, you cannot specify subnets. You must provide availability zones instead of
subnets. |
IP
Address Range | CIDR block that specifies the IP address range that the cluster can use.
For example:
10.0.0.0/24
|
Initialization Script Path | Google Cloud Storage file path of the initialization script
to run on each cluster node when the node is created. Use the format:
<bucket name>/<folder name> . The script can
reference other init scripts in the same bucket or in a subdirectory. The script must be a bash script. |
Cluster Labels | Labels to
apply to cluster nodes. Each label has a key and a value. The key can be up to
63 characters long. You can list a maximum of 55 labels. The Secure
Agent also assigns default labels to cloud resources. The default labels do not
contribute to the limit of 55 labels. Labels cannot include UTF-8
characters \u241e and \u241f that correspond to record and unit separators
represented by ASCII control characters 30 and 31. |
Property | Description |
---|---|
Encrypt Data | Indicates whether temporary data on the cluster is encrypted.
|
Runtime Properties | Custom properties to customize the cluster and the jobs that run on the cluster.
|