Explain API createOrReplaceTempView().

[DataFlair Team]

Explain API createOrReplaceTempView().

[DataFlair Team]

<li style=”list-style-type: none”>

• Its basic Dataset function.
• Its under org.apache.spark.sql

 

• def createOrReplaceTempView(viewName: String): Unit
• Creates a temporary view using the given name.
• The lifetime of this temporary view is tied to the SparkSession that was used to create this Dataset.

 

<br />
scala> val df = spark.read.csv("/home/hdadmin/titanic_data.txt")<br />
df: org.apache.spark.sql.DataFrame = [_c0: string, _c1: string ... 9 more fields]</p>
<p>scala> df.printSchema<br />
root<br />
 |-- _c0: string (nullable = true)<br />
 |-- _c1: string (nullable = true)<br />
 |-- _c2: string (nullable = true)<br />
 |-- _c3: string (nullable = true)<br />
 |-- _c4: string (nullable = true)<br />
 |-- _c5: string (nullable = true)<br />
 |-- _c6: string (nullable = true)<br />
 |-- _c7: string (nullable = true)<br />
 |-- _c8: string (nullable = true)<br />
 |-- _c9: string (nullable = true)<br />
 |-- _c10: string (nullable = true)</p>
<p>scala> df.show<br />
+---+---+---+--------------------+-------+-----------+--------------------+-------+-----------------+-----+------+<br />
|_c0|_c1|_c2|                 _c3|    _c4|        _c5|                 _c6|    _c7|              _c8|  _c9|  _c10|<br />
+---+---+---+--------------------+-------+-----------+--------------------+-------+-----------------+-----+------+<br />
|  1|1st|  1|Allen, Miss Elisa...|29.0000|Southampton|        St Louis, MO|    B-5|       24160 L221|    2|female|<br />
|  2|1st|  0|Allison, Miss Hel...| 2.0000|Southampton|Montreal, PQ / Ch...|    C26|             null| null|female|<br />
|  3|1st|  0|Allison, Mr Hudso...|30.0000|Southampton|Montreal, PQ / Ch...|    C26|             null|(135)|  male|<br />
|  4|1st|  0|Allison, Mrs Huds...|25.0000|Southampton|Montreal, PQ / Ch...|    C26|             null| null|female|<br />
|  5|1st|  1|Allison, Master H...| 0.9167|Southampton|Montreal, PQ / Ch...|    C22|             null|   11|  male|<br />
|  6|1st|  1|  Anderson, Mr Harry|47.0000|Southampton|        New York, NY|   E-12|             null|    3|  male|<br />
|  7|1st|  1|Andrews, Miss Kor...|63.0000|Southampton|          Hudson, NY|    D-7|        13502 L77|   10|female|<br />
|  8|1st|  0|Andrews, Mr Thoma...|39.0000|Southampton|         Belfast, NI|   A-36|             null| null|  male|<br />
|  9|1st|  1|Appleton, Mrs Edw...|58.0000|Southampton| Bayside, Queens, NY|  C-101|             null|    2|female|<br />
| 10|1st|  0|Artagaveytia, Mr ...|71.0000|  Cherbourg| Montevideo, Uruguay|   null|             null| (22)|  male|<br />
| 11|1st|  0|Astor, Colonel Jo...|47.0000|  Cherbourg|        New York, NY|   null|17754 L224 10s 6d|(124)|  male|<br />
| 12|1st|  1|Astor, Mrs John J...|19.0000|  Cherbourg|        New York, NY|   null|17754 L224 10s 6d|    4|female|<br />
| 13|1st|  1|Aubert, Mrs Leont...|     NA|  Cherbourg|       Paris, France|   B-35|     17477 L69 6s|    9|female|<br />
| 14|1st|  1|Barkworth, Mr Alg...|     NA|Southampton|       Hessle, Yorks|   A-23|             null|    B|  male|<br />
| 15|1st|  0| Baumann, Mr John D.|     NA|Southampton|        New York, NY|   null|             null| null|  male|<br />
| 16|1st|  1|Baxter, Mrs James...|50.0000|  Cherbourg|        Montreal, PQ|B-58/60|             null|    6|female|<br />
| 17|1st|  0|Baxter, Mr Quigg ...|24.0000|  Cherbourg|        Montreal, PQ|B-58/60|             null| null|  male|<br />
| 18|1st|  0| Beattie, Mr Thomson|36.0000|  Cherbourg|        Winnipeg, MN|    C-6|             null| null|  male|<br />
| 19|1st|  1|Beckwith, Mr Rich...|37.0000|Southampton|        New York, NY|   D-35|             null|    5|  male|<br />
| 20|1st|  1|Beckwith, Mrs Ric...|47.0000|Southampton|        New York, NY|   D-35|             null|    5|female|<br />
+---+---+---+--------------------+-------+-----------+--------------------+-------+-----------------+-----+------+<br />
only showing top 20 rows</p>
<p>scala> df.createOrReplaceTempView("titanicdata")</p>
<p>

[DataFlair Team]

NOTE : Please note that, this API is metioned as ‘Experimental’ in Spark API documentation

[Author]

Posts