Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Big Data Management
  3. Mappings
  4. Sources
  5. Targets
  6. Transformations
  7. Data Preview
  8. Cluster Workflows
  9. Profiles
  10. Monitoring
  11. Hierarchical Data Processing
  12. Hierarchical Data Processing Configuration
  13. Hierarchical Data Processing with Schema Changes
  14. Intelligent Structure Models
  15. Stateful Computing
  16. Appendix A: Connections
  17. Appendix B: Data Type Reference
  18. Appendix C: Function Reference

Define YARN Queues

Define YARN Queues

You can define multiple YARN queues to redirect jobs to specific queues. You can redirect Blaze, Spark, and Sqoop jobs.
To redirect jobs on the Blaze engine to a specific queue, configure the following Blaze configuration property in the Hadoop connection:
Property
Description
YARN Queue Name
The YARN scheduler queue name used by the Blaze engine that specifies available resources on a cluster.
To redirect jobs on the Spark engine to a specific queue, configure the following Spark configuration property in the Hadoop connection:
Property
Description
YARN Queue Name
The YARN scheduler queue name used by the Spark engine that specifies available resources on a cluster. The name is case sensitive.
To redirect Sqoop jobs to a specific queue, configure the following JDBC connection property:
Property
Description
Sqoop Arguments
The Sqoop connection-level argument to direct a MapReduce job to a specific YARN queue.
Use the following format:
-Dmapred.job.queue.name=<YARN queue name>
If you do not direct the job to a specific queue, the Spark engine uses the default queue.

0 COMMENTS

We’d like to hear from you!