Table of Contents

Search

  1. Preface
  2. Introduction to Informatica Data Engineering Integration
  3. Mappings
  4. Mapping Optimization
  5. Sources
  6. Targets
  7. Transformations
  8. Python Transformation
  9. Data Preview
  10. Cluster Workflows
  11. Profiles
  12. Monitoring
  13. Hierarchical Data Processing
  14. Hierarchical Data Processing Configuration
  15. Hierarchical Data Processing with Schema Changes
  16. Intelligent Structure Models
  17. Blockchain
  18. Stateful Computing
  19. Appendix A: Connections Reference
  20. Appendix B: Data Type Reference
  21. Appendix C: Function Reference

Define YARN Queues

Define YARN Queues

You can define multiple YARN queues to redirect jobs to specific queues. You can redirect Blaze, Spark, and Sqoop jobs.
To redirect jobs on the Blaze engine to a specific queue, configure the following Blaze configuration property in the Hadoop connection:
Property
Description
YARN Queue Name
The YARN scheduler queue name used by the Blaze engine that specifies available resources on a cluster.
To redirect jobs on the Spark engine to a specific queue, configure the following Spark configuration property in the Hadoop connection:
Property
Description
YARN Queue Name
The YARN scheduler queue name used by the Spark engine that specifies available resources on a cluster. The name is case sensitive.
To redirect Sqoop jobs to a specific queue, configure the following JDBC connection property:
Property
Description
Sqoop Arguments
The Sqoop connection-level argument to direct a MapReduce job to a specific YARN queue.
Use the following format:
-Dmapred.job.queue.name=<YARN queue name>
If you do not direct the job to a specific queue, the Spark engine uses the default queue.

0 COMMENTS

We’d like to hear from you!