Configuring YARN in Informatica Big Data Management®

Configuring YARN in Informatica Big Data Management®

Overview

Overview

You can use YARN schedulers, YARN queues, and node labels to manage resources on a Hadoop cluster when you run jobs in the Hadoop environment.
Schedulers, queues, and node labels operate in the following ways:
YARN Schedulers
A YARN scheduler allocates resources to YARN queues and determines how the jobs that are submitted to a queue can use the allocated resources. You can use a fair scheduler or a capacity scheduler depending on your organization's policies on sharing resources.
YARN Queues
A YARN queue is an organizing structures for jobs that run on the cluster. You can redirect Blaze, Spark, and Hive jobs to specific queues. The jobs in a queue can access the resources that are allocated to the queue by a YARN scheduler.
Node Labels
Node labels divide a cluster into partitions. Each partition is a collection of nodes that have specific characteristics. You can run jobs in a partition to allow jobs to utilize a node's particular characteristics.

0 COMMENTS

We’d like to hear from you!