{"id":418,"date":"2016-06-13T13:07:20","date_gmt":"2016-06-13T13:07:20","guid":{"rendered":"http:\/\/data-flair.training\/blogs\/?p=418"},"modified":"2021-05-19T18:24:25","modified_gmt":"2021-05-19T12:54:25","slug":"hadoop-yarn-node-manager-tutorial","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/hadoop-yarn-node-manager-tutorial\/","title":{"rendered":"Deep Dive into Hadoop YARN Node Manager – A Yarn Tutorial"},"content":{"rendered":"

In this Hadoop Yarn<\/strong> node manager tutorial, we will discuss node manager in Yarn, how it interact with the resource manager, how it allocates containers. We will also cover different Node manager components, how auxiliary services run\u00a0in the node manager?<\/p>\n

Introduction to Hadoop Yarn Node Manager<\/h2>\n

Conceptually, the NodeManag<\/b>er<\/strong> is more of a generic and efficient version of TaskTracker<\/b>\u00a0(of Hadoop1 architecture) which is more flexible than TaskTracker.<\/p>\n

In contrast to fixed number of slots for map and reduce tasks in MRV1, the NodeManager of MRV2 has a number of dynamically created resource containers<\/b>. There is no hard code split available into Map and Reduce slots as in MRV1.<\/p>\n

The container refers to a collection of resources such as memory, CPU, disk and network IO. The number of containers on a node is the product of configuration parameter and the total amount of node resources. Node manager is the slave daemon of Yarn.<\/p>\n

\"Node<\/a>

Hadoop yarn Node Manager<\/p><\/div>\n

The Hadoop Yarn Node Manager is the per-machine\/per-node framework agent who is responsible for containers, monitoring their resource usage and reporting the same to the ResourceManager<\/strong>.<\/p>\n

Overseeing container\u2019s lifecycle management, NodeManager also tracks the health of the node on which it is running, controls auxiliary services which different YARN applications may exploit at any point in time.<\/p>\n

NodeManager can execute any computations that make sense to ApplicationMaster <\/b>just by creating the container for each task.The above architecture diagram gives a detailed view of the NodeManager components.<\/p>\n

The above architecture diagram gives a detailed view of the NodeManager components.<\/p>\n

Yarn NodeManager Components<\/h2>\n

This section of Hadoop Yarn node manager tutorial will provide you a detailed description of yarn node manager components-<\/p>\n

1. NodeStatusUpdater<\/h3>\n

On startup, this component registers with the ResourceManager(RM) and sends information about the resources available to every node. Subsequent NM-RM communication exchange updates on container statuses of every node like containers running on the node completed containers, etc.<\/p>\n

In addition, the RM may signal the NodeStatusUpdater<\/em> to potentially kill already running containers.<\/p>\n

2. Container Manager<\/h3>\n

Being the core component of the NodeManager, shoulder the responsibilities of managing the containers running on each node with its sub-components, each of which performs a subset of the functionality that is needed to manage containers running on the node.<\/p>\n