- This topic has 2 replies, 1 voice, and was last updated 5 years, 7 months ago by
.
-
Posts
-
-
-
ResourceManager which is one of the core component of YARN – Yet Another Resource Negotiator, is the master that arbitrates all the available cluster resources and thus helps manage the distributed applications running on the YARN system.
The ResourceManager has two main components: Scheduler and ApplicationManager
The Scheduler is responsible for allocating resources to the various running applications subject to constraints of capacities, queues etc. It performs its scheduling function based on the resource requirements of the applications. This is done by a resource Container, which retrieves elements such as memory, cpu, disk, network etc.
The Scheduler is called a pure scheduler in ResourceManager, which means that it does not perform any monitoring or tracking of status for the applications. Also, if there is an application failure or hardware failure, the Scheduler does not guarantee to restart the failed tasks.
Furthermore, there is a pluggable policy in the Scheduler which is responsible for partitioning the cluster resources among various applications. There are two such plug-ins: CapacityScheduler and FairScheduler, which are currently used as Schedulers in ResourceManager.
Follow the link for more detail: ResourceManager in YARN
-
ResourceManager is the core component of YARN. The idea is to have a global Resource Manager and per application Application Master.
The Resource Manager has two main components :
1. Scheduler
2. Application MasterThe Scheduler is responsible for allocating resources to the running various applications subject to constraints of capacities, queue etc. It’s a pure scheduler that it performs no monitoring and tracking of application, also it provides no guarantees to restart a failed job either it’s failed due to application failure or hardware failure.
The Scheduler provides a pluggable policy that is responsible for partitioning cluster resources among the various queues and applications etc.
Capacity Scheduler and Fair Scheduler would be example of plug – ins.
-
- You must be logged in to reply to this topic.
