What is Spark SQL ?

Spark SQL is a Spark interface to work with Structured and Semi-Structured data (data that as defined fields i.e. tables). It provides abstraction layer called DataFrame and DataSet through with we can work with data easily. One can say that DataFrame is like a table in a relational database. Spark SQL can read and write data in a variety of Structured and Semi-Structured formats like Parquets, JSON, Hive. Using SparkSQL inside Spark application is the best way to use it. This empowers us to load data and query it with SQL. we can also combine it with “regular” program code in Python, Java or Scala.

For detailed study on SparkSQL, Refer link: Spark SQL

Spark sql is a module in apache spark for structured and semi structured data processing.The interface provided by spark sql provides spark more information about the structure of data and computation being performed on the data.It integrates relational processing with spark’s functional programming.It offers much tighter integration of relational processing with procedural processing through declarative dataframe API’s. Dataframe api’s and Dataset api’s are the way to interact with spark sql.