{"id":16276,"date":"2018-06-13T04:20:22","date_gmt":"2018-06-13T04:20:22","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=16276"},"modified":"2018-06-13T04:20:22","modified_gmt":"2018-06-13T04:20:22","slug":"hbase-architecture","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/hbase-architecture\/","title":{"rendered":"HBase Architecture – Regions, Hmaster, Zookeeper"},"content":{"rendered":"

In this HBase tutorial<\/strong>, we will learn the concept of HBase Architecture.\u00a0Moreover, we will see the 3 major components of HBase, such as HMaster, Region Server, and ZooKeeper. <\/span><\/p>\n

Along with this, we will see the working of HBase Components, HBase Memstore<\/strong>, HBase Compaction in Architecture of HBase.\u00a0This HBase Technology tutorial also includes the advantages and limitations of HBase Architecture to understand it well.<\/span><\/p>\n

So, let’s start HBase Architecture.<\/p>\n

What is HBase Architecture?<\/span><\/h2>\n

Basically, there are 3 types of servers in a master-slave type of HBase Architecture. They are HBase HMaster, Region Server, and ZooKeeper<\/strong>. Let\u2019s start with Region servers, these servers serve data for reads and write purposes. <\/span><\/p>\n

That means clients can directly communicate with HBase Region Servers while accessing data. Further, the HBase Master process handles the region assignment as well as DDL (create, delete tables) operations. And finally, a part of HDFS<\/strong>, Zookeeper, maintains a live cluster state.<\/span><\/p>\n

\"HBase<\/a>

What is HBase Architecture<\/p><\/div>\n

In addition, the data which we manage by Region Server further stores in the Hadoop DataNode. And, all HBase data is stored in HDFS<\/strong> files. <\/span>Then for the data served by the RegionServers, Region Servers are collocated with the HDFS DataNodes, which also enable data locality. <\/span><\/p>\n

Here, data locality refers to putting the data close to where we need. Make sure, when we write HBase data it is local, but while we move a region, it is not local until compaction.<\/span><\/p>\n

Moreover, for all the physical data blocks the NameNode maintains Metadata information that comprise the files.<\/span><\/p>\n

HBase Architecture – Regions<\/span><\/h2>\n

In HBase Architecture, a region consists of all the rows between the start key and the end key which are assigned to that Region. And, those Regions which we assign to the nodes in the HBase Cluster, is what we call \u201cRegion Servers\u201d. <\/span><\/p>\n

Basically, for the purpose of reads and writes these servers serves the data. While talking about numbers, it can serve approximately 1,000 regions. However, we manages rows in each region in HBase in a sorted order.<\/span><\/p>\n

\"HBase<\/a>

HBase Architecture – Regions<\/p><\/div>\n

These Regions of a Region Server are responsible for several things, like handling, managing, executing as well as reads and writes HBase operations<\/strong> on that set of regions. The default size of a region is 256MB, which we can configure as per requirement.<\/span><\/p>\n

HBase Architecture – HMaster<\/span><\/h2>\n

HBase master in the architecture of HBase is responsible for region assignment as well as DDL (create, delete tables) operations.<\/span><\/p>\n

There are two main responsibilities of a master in HBase architecture:<\/span><\/p>\n

\"HBase<\/a>

The Architecture of HBase – HMaster<\/p><\/div>\n

a. Coordinating the region servers<\/strong>
\nBasically, a master assigns Regions on startup. Also for the purpose of recovery or load balancing, it re-assigns regions.<\/span>
\nAlso, a master monitors all RegionServer instances in the HBase Cluster.<\/span><\/p>\n

b. Admin functions<\/strong>
\nMoreover, it acts as an interface for creating, deleting and updating tables in HBase.<\/span><\/p>\n

ZooKeeper in HBase Architecture<\/span><\/h2>\n

However, to maintain server state in the HBase Cluster, HBase uses ZooKeeper as a distributed coordination service. <\/span><\/p>\n

Basically, which servers are alive and available is maintained by Zookeeper, and also it provides server failure notification. Moreover, in order to guarantee common shared state, Zookeeper uses consensus. <\/span><\/p>\n

\"HBase<\/a>

HBase Architecture – Zookeeper<\/p><\/div>\n

How HBase Components Works?<\/span><\/h2>\n

As we know, to coordinate shared state information for members of distributed systems, HBase uses Zookeeper. Further, active HMaster, as well as Region servers, connect with a session to ZooKeeper. Then for active sessions, ZooKeeper maintains ephemeral nodes by using heartbeats. <\/span><\/p>\n

Ephemeral nodes mean znodes which exist as long as the session which created the znode is active and then znode is deleted when the session ends.<\/span><\/p>\n

\"HBase<\/a>

HBase Architecture – working of Components<\/p><\/div>\n

In addition, each Region Server in HBase Architecture produces an ephemeral node. Further, to discover available region servers, the HMaster monitors these nodes. <\/span><\/p>\n

Also for server failures, it monitors these nodes. Moreover, to make sure that only one master is active, Zookeeper determines the first one and uses it. <\/span><\/p>\n

As a process, the active HMaster sends heartbeats to Zookeeper, however, the one which is not active listens for notifications of the active HMaster failure.<\/span><\/p>\n

Although, the session gets expired and the corresponding ephemeral node is also deleted if somehow a region server or the active HMaster fails to send a heartbeat. Then for updates, listeners will be notified of the deleted nodes. <\/span><\/p>\n

Further, the active HMaster will recover region servers, as soon as it listens for region servers on failure. Also, when inactive one listens for the failure of active HMaster, the inactive HMaster becomes active, if an active HMaster fails.<\/span><\/p>\n

HBase Architecture – Read or Write<\/span><\/h2>\n

When the first time a client reads or writes to HBase<\/strong>:<\/span><\/p>\n