Configuring YARN in Informatica Big Data Management®

Configuring YARN in Informatica Big Data Management®

YARN Schedulers

YARN Schedulers

A YARN scheduler determines how resources are allocated on a Hadoop cluster.
When you configure a scheduler, you define the scheduler and the queues that the scheduler manages. The scheduler allocates cluster resources to the queues. The resources that are allocated to a queue determine the resources that are available to jobs that are submitted to the queue.
By integrating queues within schedules, multiple tenants can share a Hadoop cluster. Depending on your organization's policies, you might use a fair scheduler or a capacity scheduler.
For example, the users in an organization might share a single Hadoop cluster. The organization can allocate an equal number of resources to the jobs that users submit. To allocate resources evenly, the organization can employ a fair scheduler.
On the other hand, an association of organizations might collectively fund a Hadoop cluster. Each organization in the association has different computing needs and invests a certain stake in the cluster. Based on an organization's investment, the organization is guaranteed a certain amount of cluster resources. To divide and allocate cluster resources between the organizations in the association, the association can employ a capacity scheduler.

0 COMMENTS

We’d like to hear from you!