Enables server side encryption for S3 buckets. Required for SSE and SSE-KMS encryption.

Set to TRUE.

The ID for the Blaze and Spark engines to connect to the Amazon S3 file system.

Set to your access key.

The password for the Blaze and Spark engines to connect to the Amazon S3 file system

Set to your access ID.

The server-side encryption algorithm for S3. Required for SSE and SSE-KMS encryption. Set to the encryption algorithm used.

Defines the groups that the proxy user account can impersonate. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.

Set to group names of impersonation users separated by commas. If less security is preferred, use the wildcard " * " to allow impersonation from any group.

Defines the host machines that a user account can impersonate. On a secure cluster the <proxy user> is the Service Principal Name that corresponds to the cluster keytab file. On a non-secure cluster, the <proxy user> is the system user that runs the Informatica daemon.

Set to a single host name or IP address, or set to a comma-separated list. If less security is preferred, use the wildcard " * " to allow impersonation from any host.

Comma-separated list of hosts that you want to allow the Hive user to impersonate on a non-secure cluster.

Enables compression on temporary staging tables.

Set to a comma-separated list of compression codec classes on the cluster.

Translates the principal names from the Active Directory and MIT realm into local names within the Hadoop cluster. Based on the Hadoop cluster used, you can set multiple rules.

Set to: RULE:[1:$1@$0](^.*@YOUR.REALM)s/^(.*)@YOUR.REALM\.COM$/$1/g

Set to: RULE:[2:$1@$0](^.*@YOUR.REALM\.$)s/^(.*)@YOUR.REALM\.COM$/$1/g

Enables preemption for the Fair Scheduler. The Blaze engine does not support preemption. If YARN preemption is enabled for the cluster, you need to disable it for the queue allocated to the Blaze engine.

Set to FALSE for the queue allocated to the Blaze engine.

For example:

<queue name="Blaze">
    <weight>1.0</weight>
    <allowPreemptionFrom>false</allowPreemptionFrom>
    <schedulingPolicy>fsp</schedulingPolicy>
    <aclSubmitApps>*</aclSubmitApps>
    <aclAdministerApps>*</aclAdministerApps>
</queue>

Identifies HBase master and region servers.

Set to the relative path to the znode directory of HBase.

The KeyProvider used to interact with encryption keys when reading and writing to an encryption zone. Required if sources or targets reside in the HDFS encrypted zone on Java KeyStore KMS-enabled Cloudera CDH cluster or a Ranger KMS-enabled Hortonworks HDP cluster.

Set to: kmf://http@xx11.xyz.com:16000/kms

The token store implementation. Required for HiveServer2 high availability and load balancing.

Set to: org.apache.hadoop.hive.thrift.DBTokenStore

Enables dynamic partitioned tables for Hive tables. Applicable for Hive versions 0.9 and earlier.

Set to: TRUE

Allows all partitions to be dynamic. Required if you use Sqoop and define a DDL query to create or replace a partitioned Hive target at run time.

Set to: nonstrict

The value of the ZooKeeper namespace in the JDBC connection string. Required for HiveServer2 high availability.

Set to:

jdbc:hive2://<zookeeper_ensemble>/default;serviceDiscoveryMode=zooKeeper;

The run-time framework to run MapReduce jobs. Values can be local, classic, or yarn. Required for Sqoop.

Set to: yarn

Location of the MapReduce JobHistory Server. The default port is 10020. Required for Sqoop.

Set to: <MapReduce JobHistory Server>:<port>

Directory where MapReduce jobs write history files. Required for Sqoop.

Set to: /mr-history/tmp

Directory where the MapReduce JobHistory Server manages history files. Required for Sqoop.

Set to: /mr-history/done

The Service Principal Name for the MapReduce JobHistory Server. Required for Sqoop.

Set to: mapred/_HOST@YOUR-REALM

Web address of the MapReduce JobHistory Server. The default value is 19888. Required for Sqoop.

Set to: <host>:<port>

The HDFS staging directory used while submitting jobs.

Set to the staging directory path.

The sort buffer memory. Required when the output needs to be sorted for Blaze and Spark engines.

Set value to 270 MB.

Required for dynamic resource allocation.

Add spark_shuffle.jar to the class path. The .jar file must contain the class "org.apache.spark.network.yarn.YarnShuffleService."

The maximum RAM available for each container. Set the maximum memory on the cluster to increase resource memory available to the Blaze engine.

Set the value to at least 16GB.

The number of virtual cores for each container. Required for Blaze engine resource allocation.

Set the value to at least 10.

The minimum RAM available for each container. Required for Blaze engine resource allocation.

Set the value to at least 6GB.

Disables virtual memory limits for containers. Required for the Blaze and Spark engines.

Set to: false

Required for dynamic resource allocation for the Spark engine.

Add an entry for "spark_shuffle."

Required for dynamic resource allocation for the Spark engine.

Set to: org.apache.spark.network.yarn.YarnShuffleService

Defines the YARN scheduler that the Data Integration Service uses to assign resources.

Set to: org.apache.hadoop.yarn.server.resourcemanager.scheduler