Big Data Management User Guide

Back Next

Define YARN Queues

You can define multiple YARN queues to redirect jobs to specific queues. You can redirect Blaze, Spark, Hive, and Sqoop jobs.

To redirect jobs on the Blaze engine to a specific queue, configure the following Blaze configuration property in the Hadoop connection:

Property	Description
YARN Queue Name	The YARN scheduler queue name used by the Blaze engine that specifies available resources on a cluster.

To redirect jobs on the Spark engine to a specific queue, configure the following Spark configuration property in the Hadoop connection:

Property	Description
YARN Queue Name	The YARN scheduler queue name used by the Spark engine that specifies available resources on a cluster. The name is case sensitive.

To redirect jobs on the Hive engine to a specific queue, configure the following Hive connection property:

Property	Description
Data Access Connection String	The Hive connection string to specify the queue name for Hive SQL override mappings on the Blaze engine. Use the following format: MapReduce. mapred.job.queue.name=<YARN queue name> Tez. tez.queue.name=<YARN queue name> For example, jdbc:hive2://business.com:10000/default;principal=hive/_HOST@INFAKRB?mapred.job.queue.name-root.test

To redirect Sqoop jobs to a specific queue, configure the following JDBC connection property:

Property	Description
Sqoop Arguments	The Sqoop connection-level argument to direct a MapReduce job to a specific YARN queue. Use the following format: -Dmapred.job.queue.name=<YARN queue name> If you do not direct the job to a specific queue, the Spark engine uses the default queue.