{"id":17665,"date":"2018-06-26T04:19:05","date_gmt":"2018-06-26T04:19:05","guid":{"rendered":"https:\/\/data-flair.training\/blogs\/?p=17665"},"modified":"2018-06-26T04:19:05","modified_gmt":"2018-06-26T04:19:05","slug":"hbase-troubleshooting","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/","title":{"rendered":"HBase Troubleshooting – Problem, Cause & Solution"},"content":{"rendered":"

In this HBase tutorial,<\/strong>\u00a0we will learn HBase troubleshooting. It is possible that several kinds of problems may occur while working on HBase. So, in this article, “HBase Troubleshooting” we will discuss different problems along with their solution, to solve any error or failures in Troubleshooting in HBase.<\/span><\/p>\n

So, let’s start HBase Troubleshooting.<\/p>\n

HBase Troubleshooting<\/span><\/h2>\n

i. Problem 1<\/h3>\n

Problem Statement:<\/strong>\u00a0Thrift Server Crashes after Receiving Invalid Data.<\/span>
\nIf a Thrift server receives a large amount of invalid data, due to a buffer overrun, it may crash.<\/span><\/p>\n

Cause:<\/strong>
\nIn order to check the validity of data which Thrift server receives, it allocates memory. So, it may need to allocate more memory than is available if there is<\/span>\u00a0a large amount of invalid data. However, it occurs because of limitation in the Thrift library itself.<\/span><\/p>\n

Solution:<\/strong>
\nBasically, to prevent from crashes, we can use the framed and compact transport protocols. By default, these protocols are disabled. The reason behind this is\u00a0it may\u00a0need changes to <\/span>our client code. There are two options to add to our hbase-site.xml<\/strong> are:<\/span><\/p>\n

    \n
  • hbase.regionserver.thrift.framed,<\/span><\/li>\n
  • hbase.regionserver.thrift.compact<\/span><\/li>\n<\/ul>\n

    Like in the XML below, set each of these to true. Moreover, using the hbase.regionserver.thrift.framed.max_frame_size_in_mb option,\u00a0we can also specify the maximum frame size.<\/p>\n

    <property>\n <name>hbase.regionserver.thrift.framed<\/name>\n <value>true<\/value>\n<\/property>\n<property>\n <name>hbase.regionserver.thrift.framed.max_frame_size_in_mb<\/name>\n <value>2<\/value>\n<\/property>\n<property>\n <name>hbase.regionserver.thrift.compact<\/name>\n <value>true<\/value>\n<\/property><\/pre>\n
    ii. Problem 2<\/h3>\n
    Problem Statement:<\/strong> Master server initializes but region servers not initialize<\/span>
    \nBasically, through their IP addresses, the Communication between Master and region servers occurs. As Master is going to listen that region servers are running or having the IP address of 127.0.0.1.<\/p>\n
    Cause:<\/strong>
    \nBasically, region server continuously informs Master server about their IP addresses are 127.0.0.1, in dual communication between region servers and master.<\/span><\/p>\n
    Solution:<\/strong>
    \nAs a solution, remove master server name node from localhost especially, which is present in the hosts file
    \nHost file location \/etc\/hosts<\/span><\/p>\n
    What to change:<\/strong>
    \nGo to this location, after opening \/etc.\/hosts:<\/span>
    \n127.0.0.1 fully.qualified.regionservernameregionservernamelocalhost.localdomain localhost<\/strong>
    \n: : 1 localhost3.localdomain3 localdomain3<\/strong><\/p>\n
    Also, do modifications in the above configuration:<\/span>
    \n127.0.0.1 localhost.localdomainlocalhost<\/strong>
    \n: : 1 localhost3.localdomain3 localdomain3<\/strong><\/p>\n
    iii. Problem 3<\/h3>\n
    Problem Statement:<\/strong> <\/span>Couldn’t find my address: XYZ in the list of Zookeeper quorum servers<\/span><\/p>\n
    Cause:<\/strong>
    \n1. ZooKeeper server will throw an error like .xyz in the name of the server, and also it was not able to start.<\/span>
    \n2. Also,\u00a0 to start a ZooKeeper server, HBase attempts, on some machine but at the same time machine is not able to find itself the quorum configuration.<\/span><\/p>\n
    Solution:<\/strong>
    \n1. Its solution is to replace the hostname with a hostname that is presented in the error message<\/span>
    \n2. we can set the below configurations in HBase-site.xml, but only if we suppose we are having DNS server.<\/span>
    \n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0– HBase.zookeeper.dns.interface<\/span>
    \n\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0– HBase.zookeeper.dns.nameserver<\/span><\/p>\n
    iv. Problem 4<\/span><\/h3>\n
    Problem Statement:<\/strong> Created Root Directory for HBase through Hadoop DFS<\/span>
    \n1. As we need to run the HBase migrations script.<\/span>
    \n2. The HBase migrations script respond like no files in the root directory, upon running that.<\/span><\/p>\n
    Cause:<\/strong>
    \n– Using Hadoop Distributed file system, create a new directory for HBase.<\/span>Two possibilities are:<\/p>\n
    1) Root directory not to exist<\/span>
    \n2) Before only, HBase previous running instance initialized.<\/span><\/p>\n
    Solution:<\/strong>
    \nEnsure that the HBase root directory does not currently exist or by a previous run of HBase instance, it has been initialized.<\/span>
    \nStep 1: <\/strong>
    \nTo delete the HBase root directory uses Hadoop dfs.<\/span>
    \nStep 2: <\/strong>
    \nBy itself, HBase creates and initializes the directory.<\/span><\/p>\n
    v. Problem 5<\/span><\/h3>\n
    Problem Statement:<\/strong> Zookeeper session expired events<\/span><\/p>\n
    Cause:<\/strong>
    \n1. By throwing Exceptions, HMaster or HRegion servers shutting down.\u00a0<\/span>
    \n2. Moreover, we can find out the actual exceptions that thrown, if we observe logs.<\/span>
    \nBecause of Zookeeper expired event, the following shows the exception thrown:<\/span><\/p>\n
    Log files code:<\/strong><\/p>\n
    WARN org.apache.zookeeper.ClientCnxn: Exception\nclosing session 0x278bd16a96000f to sun.nio.ch.SelectionKeyImpl@355811ec\njava.io.IOException: TIMED OUT\nat org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:906)\nWARN org.apache.hadoop.hbase.util.Sleeper: We slept 79410ms, ten times longer than scheduled: 5000\nINFO org.apache.zookeeper.ClientCnxn: Attempting connection to server hostname\/IP:PORT\nINFO org.apache.zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=\/IP:PORT remote=hostname\/IP:PORT]\nINFO org.apache.zookeeper.ClientCnxn: Server connection successful\nWARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x278bd16a96000d to sun.nio.ch.SelectionKeyImpl@3544d65e\njava.io.IOException: Session Expired\nat org.apache.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java:589)\nat org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:709)\nat org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)\nERROR org.apache.hadoop.hbase.regionserver.HRegionServer: ZooKeeper session expired<\/pre>\n
    Solution:<\/strong><\/p>\n
      \n
    1. \u00a0RAM size is 1 GB,\u00a0by default. So, we have maintained RAM capacity more than 1 GB, for doing long running imports.<\/span><\/li>\n
    2. It is must to increase the session timeout for Zookeeper.<\/span><\/li>\n
    3. Also, we have to modify the following property in “hbase-site.xml” that present in hbase \/conf folder path, for increasing session time out of Zookeeper.<\/span><\/li>\n
    4. Session timeout is 60 seconds, by default. So, we can change it to 120 seconds.<\/span><\/li>\n<\/ol>\n
      <property>\n   <name> zookeeper.session.timeout <\/name>\n   <value>1200000<\/value>\n<\/property>\n<property>\n   <name> hbase.zookeeper.property.tickTime <\/name>\n   <value>6000<\/value>\n<\/property><\/pre>\n
      So, this was all about Troubleshooting in HBase. Hope you like our explanation.<\/p>\n
      Conclusion: HBase Troubleshooting<\/span><\/h2>\n
      Hence, in this HBase Troubleshooting tutorial, we saw solutions to various problems which can often occur during Troubleshooting in HBase. Still, if any doubt regarding HBase Troubleshooting, ask in the comment tab.<\/span>
      \n<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"
      In this HBase tutorial,\u00a0we will learn HBase troubleshooting. It is possible that several kinds of problems may occur while working on HBase. So, in this article, “HBase Troubleshooting” we will discuss different problems along...<\/p>\n","protected":false},"author":7,"featured_media":18575,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[23],"tags":[5443,5490,5492,14962],"class_list":["post-17665","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hbase","tag-hbase-known-issues","tag-hbase-troubleshooting","tag-hbase-tutorials","tag-troubleshooting-in-hbase"],"yoast_head":"\nHBase Troubleshooting - Problem, Cause & Solution - DataFlair<\/title>\n<meta name=\"description\" content=\"HBase Troubleshooting tutorial:problems in HBase troubleshooting,solutions & cause of HBase Known issues,Thrift Server Crashes after Receiving Invalid Data\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"HBase Troubleshooting - Problem, Cause & Solution - DataFlair\" \/>\n<meta property=\"og:description\" content=\"HBase Troubleshooting tutorial:problems in HBase troubleshooting,solutions & cause of HBase Known issues,Thrift Server Crashes after Receiving Invalid Data\" \/>\n<meta property=\"og:url\" content=\"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/\" \/>\n<meta property=\"og:site_name\" content=\"DataFlair\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DataFlairWS\/\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-26T04:19:05+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/06\/HBase-Troubleshooting-01-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"DataFlair Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:site\" content=\"@DataFlairWS\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DataFlair Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"HBase Troubleshooting - Problem, Cause & Solution - DataFlair","description":"HBase Troubleshooting tutorial:problems in HBase troubleshooting,solutions & cause of HBase Known issues,Thrift Server Crashes after Receiving Invalid Data","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/","og_locale":"en_US","og_type":"article","og_title":"HBase Troubleshooting - Problem, Cause & Solution - DataFlair","og_description":"HBase Troubleshooting tutorial:problems in HBase troubleshooting,solutions & cause of HBase Known issues,Thrift Server Crashes after Receiving Invalid Data","og_url":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/","og_site_name":"DataFlair","article_publisher":"https:\/\/www.facebook.com\/DataFlairWS\/","article_published_time":"2018-06-26T04:19:05+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/06\/HBase-Troubleshooting-01-1.jpg","type":"image\/jpeg"}],"author":"DataFlair Team","twitter_card":"summary_large_image","twitter_creator":"@DataFlairWS","twitter_site":"@DataFlairWS","twitter_misc":{"Written by":"DataFlair Team","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#article","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/"},"author":{"name":"DataFlair Team","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/beb0cab24b7aa54423a3b50e669a9dcd"},"headline":"HBase Troubleshooting – Problem, Cause & Solution","datePublished":"2018-06-26T04:19:05+00:00","mainEntityOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/"},"wordCount":725,"commentCount":0,"publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/06\/HBase-Troubleshooting-01-1.jpg","keywords":["HBase Known issues","HBase Troubleshooting","HBase tutorials","troubleshooting in HBase"],"articleSection":["HBase Tutorials"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/","url":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/","name":"HBase Troubleshooting - Problem, Cause & Solution - DataFlair","isPartOf":{"@id":"https:\/\/data-flair.training\/blogs\/#website"},"primaryImageOfPage":{"@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#primaryimage"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#primaryimage"},"thumbnailUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/06\/HBase-Troubleshooting-01-1.jpg","datePublished":"2018-06-26T04:19:05+00:00","description":"HBase Troubleshooting tutorial:problems in HBase troubleshooting,solutions & cause of HBase Known issues,Thrift Server Crashes after Receiving Invalid Data","breadcrumb":{"@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#primaryimage","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/06\/HBase-Troubleshooting-01-1.jpg","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2018\/06\/HBase-Troubleshooting-01-1.jpg","width":1200,"height":628,"caption":"HBase Troubleshooting - Problem, Cause & Solution"},{"@type":"BreadcrumbList","@id":"https:\/\/data-flair.training\/blogs\/hbase-troubleshooting\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog Home","item":"https:\/\/data-flair.training\/blogs\/"},{"@type":"ListItem","position":2,"name":"HBase Tutorials","item":"https:\/\/data-flair.training\/blogs\/category\/hbase\/"},{"@type":"ListItem","position":3,"name":"HBase Troubleshooting – Problem, Cause & Solution"}]},{"@type":"WebSite","@id":"https:\/\/data-flair.training\/blogs\/#website","url":"https:\/\/data-flair.training\/blogs\/","name":"DataFlair","description":"Learn Today. Lead Tomorrow.","publisher":{"@id":"https:\/\/data-flair.training\/blogs\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/data-flair.training\/blogs\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/data-flair.training\/blogs\/#organization","name":"DataFlair","url":"https:\/\/data-flair.training\/blogs\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/","url":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","contentUrl":"https:\/\/data-flair.training\/blogs\/wp-content\/uploads\/sites\/2\/2016\/07\/Data-Flair.png","width":106,"height":48,"caption":"DataFlair"},"image":{"@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DataFlairWS\/","https:\/\/x.com\/DataFlairWS","https:\/\/www.linkedin.com\/company\/dataflair-web-services-pvt-ltd\/","https:\/\/www.youtube.com\/user\/DataFlairWS"]},{"@type":"Person","@id":"https:\/\/data-flair.training\/blogs\/#\/schema\/person\/beb0cab24b7aa54423a3b50e669a9dcd","name":"DataFlair Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/c322416204232f4dd97ef3901b0a499a5d34d7ba7fe333f4bfe53a907873d293?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/c322416204232f4dd97ef3901b0a499a5d34d7ba7fe333f4bfe53a907873d293?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c322416204232f4dd97ef3901b0a499a5d34d7ba7fe333f4bfe53a907873d293?s=96&d=mm&r=g","caption":"DataFlair Team"},"description":"DataFlair Team specializes in creating clear, actionable content on programming, Java, Python, C++, DSA, AI, ML, data Science, Android, Flutter, MERN, Web Development, and technology. Backed by industry expertise, we make learning easy and career-oriented for beginners and pros alike.","url":"https:\/\/data-flair.training\/blogs\/author\/dfteam3\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/17665","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/comments?post=17665"}],"version-history":[{"count":0,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/posts\/17665\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media\/18575"}],"wp:attachment":[{"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/media?parent=17665"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/categories?post=17665"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/data-flair.training\/blogs\/wp-json\/wp\/v2\/tags?post=17665"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}