{"id":2022,"date":"2017-04-15T08:55:38","date_gmt":"2017-04-15T08:55:38","guid":{"rendered":"http:\/\/data-flair.training\/blogs\/?p=2022"},"modified":"2018-11-21T11:46:21","modified_gmt":"2018-11-21T06:16:21","slug":"apache-mesos-tutorial","status":"publish","type":"post","link":"https:\/\/data-flair.training\/blogs\/apache-mesos-tutorial\/","title":{"rendered":"Apache Mesos Tutorial – Architecture and Working"},"content":{"rendered":"

1. Objective – Apache Mesos Tutorial<\/h2>\n

In this Apache Mesos tutorial, we will learn what is Apache Mesos, what is the need of Mesos, Mesos architecture and various components of Apache Mesos. Moreover, we will also see the working of Apache Mesos to get the in-depth knowledge to learn Apache Mesos.<\/p>\n

So, let’s start Apache Mesos Tutorial.<\/p>\n

\"Apache<\/a>

Apache Mesos Tutorial – Architecture and Working<\/p><\/div>\n

2. What is Apache Mesos?<\/h2>\n

Apache Mesos<\/strong> is the first Open Source cluster manager that handles the workload in <\/em>distributed environment through dynamic resource sharing and isolation<\/em>.<\/p>\n

It is good for deployment and management of applications in large-scale cluster environments. Mesos groups together the existing resources of the machines\/nodes in a cluster into a single unit, from this unit a variety of workloads may utilize. This is known as node abstraction in Mesos that reduces an overhead of allocating a specific machine for different workloads.
\nIt is a resource management platform for Hadoop<\/strong><\/a> and Big Data<\/strong><\/a> cluster. Companies such as Twitter, Xogito, and Airbnb utilize Apache Mesos. Mesos two level scheduler discern the platform that allows distributed applications such as Apache Spark<\/a><\/strong>, Apache Kafka, and Apache Cassandra.
\nIn some way, Apache Mesos is the opposite of virtualization because in virtualization one physical resource is divided into multiple virtual resources, while in Mesos multiple physical resources are clubbed into a single virtual resource.<\/p>\n

3. A need of Apache Mesos<\/h2>\n

Many resource manager exists today like Hadoop on Demand, batch scheduler(e.g. Torque), VM Scheduler (e.g. Eucalyptus) but the problem with Hadoop like workload is data locality<\/a>,<\/strong> it \u00a0was compromised because of static partitioning of\u00a0 nodes and since the job holds nodes for full duration of time, utilization<\/strong> of the system was affected. These drawbacks were solved in Mesos with the help of Fine-grained Sharing<\/strong> and Two-level scheduling<\/strong>.
\nMesos shares a resource in a fine-grained manner meaning it allows the framework to achieve data locality by taking turns and reading data stored on each machine.<\/p>\n

4. The architecture of Apache Mesos<\/h2>\n

The below diagram shows the key components of Apache Mesos:<\/p>\n

\"Apache<\/a>

Apache Mesos Tutorial – Architecture<\/p><\/div>\n

i. Mesos Master<\/h3>\n

Mesos master is the heart of the cluster. It guarantees that the cluster will be highly available. It hosts the primary user interface that provides information about the resources available in the cluster. The master is a central source of all running task, it stores in memory all the data related to the task. For the completed task, there is only fixed amount of memory available, thus allowing the master to serve the user interface and data about the task with the minimal latency.<\/p>\n

ii. Mesos Agent<\/h3>\n

The Mesos Agent holds and manages the container that hosts the executor (all things runs inside a container in Mesos). It manages the communication between the local executor and Mesos master, thus agent acts as an intermediate between them. The Mesos agent publishes the information related to the host they are running in, including data about running task and executors, available resources of the host and other metadata. It guarantees the delivery of status update of the tasks to the schedulers.<\/p>\n

iii. Mesos Framework<\/h3>\n

Mesos Framework has two parts: The Scheduler and The Executor. The Scheduler<\/strong> registers itself in the Mesos master, and in turn gets the unique framework id. It is the responsibility of scheduler to launch task when the resource requirement and constraints match with received offer the Mesos master. It is also responsible for handling task failures and errors. The executor<\/strong> executes the task launched by the scheduler and notifies back the status of each task.
\nSome of the Frameworks provided by Mesos are:<\/p>\n