User Guide

Back Next

Advanced Properties for the Blaze Engine

If you want to use the Blaze engine to run mappings on the cloud platform cluster, you must set cluster configuration properties in the Software Setting property of the Create Cluster task.

Configure the Create Cluster task to set configuration properties in *-site.xml files on the cluster. Hadoop clusters run based on these settings.

Use a text file to specify *-site.xml file settings, and specify the path to the file in the Software Settings property.

The following text shows sample configuration of the Software Settings property file:

[
  {
     "Classification":"yarn-site",
     "Properties":{
     "yarn.scheduler.minimum-allocation-mb":"250",
     "yarn.scheduler.maximum-allocation-mb":"8192",
     "yarn.nodemanager.resource.memory-mb":"16000",
     "yarn.nodemanager.resource.cpu-vcores":"12"
   }
  },
  {
    "Classification":"core-site",
    "Properties":{
    "hadoop.proxyuser.<DIS/OSPUSER>.groups":"<group names>",
    "hadoop.proxyuser.<DIS/OSPUSER>.hosts":"*"
  }
 }
]

yarn-site

yarn.scheduler.minimum-allocation-mb: The minimum RAM available for each container. Required for Blaze engine resource allocation.
yarn.scheduler.maximum-allocation-mb: The maximum RAM available for each container. Required for Blaze engine resource allocation.
yarn.nodemanager.resource.memory-mb: The maximum RAM available for each container. Set the maximum memory on the cluster to increase resource memory available to the Blaze engine.
yarn.nodemanager.resource.cpu-vcores: The number of virtual cores for each container. Required for Blaze engine resource allocation.

core-site

hadoop.proxyuser.<proxy user>.groups: Defines the groups that the proxy user account can impersonate. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.; Set to group names of impersonation users separated by commas. If less security is preferred, use the wildcard " * " to allow impersonation from any group.
hadoop.proxyuser.<user>.hosts: Defines the host machines from which impersonated connections are allowed. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.; Set the property to " * " to allow impersonation from any host. This is required to run a Blaze mapping on a cloud platform cluster.

Rename Saved Search

Table of Contents

User Guide

User Guide

Advanced Properties for the Blaze Engine

Advanced Properties for the Blaze Engine

yarn-site

core-site