- This topic has 3 replies, 1 voice, and was last updated 5 years, 7 months ago by
.
-
Posts
-
-
-
The Hive architecture consists of the following components:
- Command Line Interface: By default, it is the way to access Hivequeries and commands
- Hive Server: It runs Hive as a server exposing a thrift service, which enables access from a range of clients written in different languages.
- Hive Web Interface: Hive Hive provides web-based GUI for executing Hive queries and commands
- Shell: A shell is the command line interface which allows interactive queries similar to MySQL shell connected to the database. It also supports web and JDBC clients. Driver, compiler and execution engine take the hiveql scripts and run in Hadoop environment.
- Driver: The driver is the component which receives the queries. This component implements the notion of session handles and provides execute and fetch APIs modeled on JDBC/ODBC interfaces.
- Compiler: The compiler parses the query does semantic analysis on the different query blocks and query expressions. It eventually generates an execution plan with the help of the table and partition metadata looked up from the metastoremetastore.
- Execution Engine: This component executes the execution plan created by the compiler. The plan has a DAG of stages. The engine manages the dependencies between these different stages of the plan. It executes these stages on the associated system components.
- Metastore: This component stores all the structure information of the various table and partitions in the warehouse. It includes column and column type information, the serializers and deserializers (also called SerDe) necessary to read and write data and the corresponding hdfs files where the data is stored.
The Compiler is invoked by the driver upon receiving a HiveQL statement. The compiler translates this statement into a plan which consists of a Directed Acyclic Graph of Map Reduce jobs.
This driver submits the individual map-reduce jobs from the DAG to the Execution Engine in an order. Hive currently uses Hadoop as the execution engine.
Learn more about Hive architecture -
Below are the main components of hive architecture:
a) Hive Clients- Different application can interact with hive with help of different hive clients provided by hive.These hive clients are hive thrift client,hive JDBC driver,hive ODBC driver.
b) Hive Web User interface-The user interface provided by hive are Hive Web UI or Hive HD Insight.We can submit hive queries directly to hive server with help of these Web UI.
c) CLI- This is the default shell provided by hive to execute the hive queries or commands directly.
d) Hive Server- This server is based on the thrift language,therefore all the application which support thrift can interact with hive server with different drivers.It allows different clients to submit requests to Hive and retrieve the final result.
e) Hive Driver- The Driver is used to receive the quires from UI .It provides execute and fetch APIs modeled on JDBC/ ODBC interfaces.
f) Compiler- This receives query from the driver and perform the parsing, type checking, and semantic analysis with the help of schema present in the metastore.
g) Executor-The component which executes the execution plan created by the compiler. The plan is a DAG of stages.In each stage of DAG there is either map or reduce task.The execution engine manages the dependencies between these different stages of the plan and executes these stages on the appropriate system components.
h) Metastore – This contains metadata for Hive tables (like their schema and location) and partitions in a relational database.Client can access this information .by metastore service API.
i) Storage Layer- This contains HDFS or HBASE for the storage of data in the tables.
-
Below are the main components of hive architecture:
a) Hive Clients- Different application can interact with hive with help of different hive clients provided by hive.These hive clients are hive thrift client,hive JDBC driver,hive ODBC driver.
b) Hive Web User interface-The user interface provided by hive are Hive Web UI or Hive HD Insight.We can submit hive queries directly to hive server with help of these Web UI.
c) CLI- This is the default shell provided by hive to execute the hive queries or commands directly.
d) Hive Server- This server is based on the thrift language,therefore all the application which support thrift can interact with hive server with different drivers.It allows different clients to submit requests to Hive and retrieve the final result.
e) Hive Driver- The Driver is used to receive the quires from UI .It provides execute and fetch APIs modeled on JDBC/ ODBC interfaces.
f) Compiler- This receives query from the driver and perform the parsing, type checking, and semantic analysis with the help of schema present in the metastore.
g) Executor-The component which executes the execution plan created by the compiler. The plan is a DAG of stages.In each stage of DAG there is either map or reduce task.The execution engine manages the dependencies between these different stages of the plan and executes these stages on the appropriate system components.
h) Metastore – This contains metadata for Hive tables (like their schema and location) and partitions in a relational database.Client can access this information .by metastore service API.
i) Storage Layer- This contains HDFS or HBASE for the storage of data in the tables.
-
- You must be logged in to reply to this topic.
