This topic contains 2 replies, has 1 voice, and was last updated by  dfbdteam3 1 year, 1 month ago.

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • #6338

    dfbdteam3
    Moderator

    What are the modes in which Hadoop run?

    #6340

    dfbdteam3
    Moderator

    1) Local Mode or Standalone Mode

    Standalone mode is the default mode in which Hadoop run. Standalone mode is mainly used for debugging where you don’t really use HDFS.
    You can use input and output both as a local file system in standalone mode.

    You also don’t need to do any custom configuration in the files- mapred-site.xml, core-site.xml, hdfs-site.xml.

    Standalone mode is usually the fastest Hadoop modes as it uses the local file system for all the input and output. Here is the summarized view of the standalone mode-

    • Used for debugging purpose
    • HDFS is not being used
    • Uses local file system for input and output
    • No need to change any configuration files
    • Default Hadoop Modes

    2) Pseudo-distributed Mode

    The pseudo-distribute mode is also known as a single-node cluster where both NameNode and DataNode will reside on the same machine.

    In pseudo-distributed mode, all the Hadoop daemons will be running on a single node. Such configuration is mainly used while testing when we don’t need to think about the resources and other users sharing the resource.

    In this architecture, a separate JVM is spawned for every Hadoop components as they could communicate across network sockets, effectively producing a fully functioning and optimized mini-cluster on a single host.

    Here is the summarized view of pseudo distributed Mode-

    • Single Node Hadoop deployment running on Hadoop is considered as pseudo distributed mode
    • All the master & slave daemons will be running on the same node
    • Mainly used for testing purpose
    • Replication Factor will be ONE for Block
    • Changes in configuration files will be required for all the three files- mapred-site.xml, core-site.xml, hdfs-site.xml

    3) Fully-Distributed Mode (Multi-Node Cluster)

    This is the production mode of Hadoop where multiple nodes will be running. Here data will be distributed across several nodes and processing will be done on each node.

    Master and Slave services will be running on the separate nodes in fully-distributed Hadoop Mode.

    • Production phase of Hadoop
    • Separate nodes for master and slave daemons
    • Data are used and distributed across multiple nodes

    In the Hadoop development, each Hadoop Modes have its own benefits and drawbacks. Definitely fully distributed mode is the one for which Hadoop is mainly known for but again there is no point in engaging the resource while in testing or debugging phase. So standalone and pseudo-distributed Hadoop modes are also having their own significance.

    #6341

    dfbdteam3
    Moderator

    Hadoop can run in 3 different modes.
    1. Standalone(Local) Mode
    By default, Hadoop is configured to run in a no distributed mode. It runs as a single Java process. Instead of HDFS, this mode utilizes the local file system. This mode useful for debugging and there isn’t any need to configure core-site.xml, hdfs-site.xml, mapred-site.xml, masters & slaves. Stand alone mode is usually the fastest mode in Hadoop.

    2. Pseudo Distributed Mode(Single node)
    Hadoop can also run on a single node in a Pseudo Distributed mode. In this mode, each daemon runs on seperate java process. In this mode custom configuration is required( core-site.xml, hdfs-site.xml, mapred-site.xml ). Here HDFS is utilized for input and ouput. This mode of deployment is useful for testing and debugging purposes.

    3. Fully Distributed Mode
    This is the production mode of Hadoop. In this mode typically one machine in the cluster is designated as NameNode and another as Resource Manager exclusively. These are masters. All other nodes act as Data Node and Node Manager. These are the slaves. Configuration parameters and environment need to specified for Hadoop Daemons.
    This mode offers fully distributed computing capability,, reliability , fault tolerance and scalability.

Viewing 3 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.