Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Big Data Management
  3. Connections
  4. Mappings in the Hadoop Environment
  5. Mapping Objects in the Hadoop Environment
  6. Processing Hierarchical Data on the Spark Engine
  7. Stateful Computing on the Spark Engine
  8. Monitoring Mappings in the Hadoop Environment
  9. Mappings in the Native Environment
  10. Profiles
  11. Native Environment Optimization
  12. Data Type Reference
  13. Complex File Data Object Properties
  14. Function Reference
  15. Parameter Reference

Scheduling and Node Labeling Configuration

Scheduling and Node Labeling Configuration

Update the yarn-site.xml file on the domain environment to enable scheduling and node labeling in the Hadoop environment. Configure the following properties:
yarn.resourcemanager.scheduler.class
Defines the YARN scheduler that the Data Integration Service uses to assign resources on the cluster.
<property> <name>yarn.resourcemanager.scheduler.class</name> <value><org.apache.hadoop.yarn.server.resourcemanager.scheduler.[Scheduler Type].[Scheduler Type]Scheduler></value> </property>
For example:
<property> <name>yarn.resourcemanager.scheduler.class</name> <value><org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler></value> </property>
yarn.node-labels.enabled
Enables node labeling.
<property> <name>yarn.node-labels.enabled</name> <value><TRUE></value> </property>
yarn.node-labels.fs-store.root-dir
The HDFS location to update the node label dynamically.
<property> <name>yarn.node-labels.fs-store.root-dir</name> <value><hdfs://[Node name]:[Port]/[Path to store]/[Node labels]/></value> </property>


Updated December 13, 2018