Free Online Certification Courses – Learn Today. Lead Tomorrow. › Forums › Apache Spark › Explain API createOrReplaceTempView().
- This topic has 2 replies, 1 voice, and was last updated 5 years, 6 months ago by DataFlair Team.
Viewing 2 reply threads
-
AuthorPosts
-
-
September 20, 2018 at 2:42 pm #5260DataFlair TeamSpectator
Explain API createOrReplaceTempView().
-
September 20, 2018 at 2:43 pm #5264DataFlair TeamSpectator
-
<li style=”list-style-type: none”>
- Its basic Dataset function.
- Its under org.apache.spark.sql
- def createOrReplaceTempView(viewName: String): Unit
- Creates a temporary view using the given name.
- The lifetime of this temporary view is tied to the SparkSession that was used to create this Dataset.
<br /> scala> val df = spark.read.csv("/home/hdadmin/titanic_data.txt")<br /> df: org.apache.spark.sql.DataFrame = [_c0: string, _c1: string ... 9 more fields]</p> <p>scala> df.printSchema<br /> root<br /> |-- _c0: string (nullable = true)<br /> |-- _c1: string (nullable = true)<br /> |-- _c2: string (nullable = true)<br /> |-- _c3: string (nullable = true)<br /> |-- _c4: string (nullable = true)<br /> |-- _c5: string (nullable = true)<br /> |-- _c6: string (nullable = true)<br /> |-- _c7: string (nullable = true)<br /> |-- _c8: string (nullable = true)<br /> |-- _c9: string (nullable = true)<br /> |-- _c10: string (nullable = true)</p> <p>scala> df.show<br /> +---+---+---+--------------------+-------+-----------+--------------------+-------+-----------------+-----+------+<br /> |_c0|_c1|_c2| _c3| _c4| _c5| _c6| _c7| _c8| _c9| _c10|<br /> +---+---+---+--------------------+-------+-----------+--------------------+-------+-----------------+-----+------+<br /> | 1|1st| 1|Allen, Miss Elisa...|29.0000|Southampton| St Louis, MO| B-5| 24160 L221| 2|female|<br /> | 2|1st| 0|Allison, Miss Hel...| 2.0000|Southampton|Montreal, PQ / Ch...| C26| null| null|female|<br /> | 3|1st| 0|Allison, Mr Hudso...|30.0000|Southampton|Montreal, PQ / Ch...| C26| null|(135)| male|<br /> | 4|1st| 0|Allison, Mrs Huds...|25.0000|Southampton|Montreal, PQ / Ch...| C26| null| null|female|<br /> | 5|1st| 1|Allison, Master H...| 0.9167|Southampton|Montreal, PQ / Ch...| C22| null| 11| male|<br /> | 6|1st| 1| Anderson, Mr Harry|47.0000|Southampton| New York, NY| E-12| null| 3| male|<br /> | 7|1st| 1|Andrews, Miss Kor...|63.0000|Southampton| Hudson, NY| D-7| 13502 L77| 10|female|<br /> | 8|1st| 0|Andrews, Mr Thoma...|39.0000|Southampton| Belfast, NI| A-36| null| null| male|<br /> | 9|1st| 1|Appleton, Mrs Edw...|58.0000|Southampton| Bayside, Queens, NY| C-101| null| 2|female|<br /> | 10|1st| 0|Artagaveytia, Mr ...|71.0000| Cherbourg| Montevideo, Uruguay| null| null| (22)| male|<br /> | 11|1st| 0|Astor, Colonel Jo...|47.0000| Cherbourg| New York, NY| null|17754 L224 10s 6d|(124)| male|<br /> | 12|1st| 1|Astor, Mrs John J...|19.0000| Cherbourg| New York, NY| null|17754 L224 10s 6d| 4|female|<br /> | 13|1st| 1|Aubert, Mrs Leont...| NA| Cherbourg| Paris, France| B-35| 17477 L69 6s| 9|female|<br /> | 14|1st| 1|Barkworth, Mr Alg...| NA|Southampton| Hessle, Yorks| A-23| null| B| male|<br /> | 15|1st| 0| Baumann, Mr John D.| NA|Southampton| New York, NY| null| null| null| male|<br /> | 16|1st| 1|Baxter, Mrs James...|50.0000| Cherbourg| Montreal, PQ|B-58/60| null| 6|female|<br /> | 17|1st| 0|Baxter, Mr Quigg ...|24.0000| Cherbourg| Montreal, PQ|B-58/60| null| null| male|<br /> | 18|1st| 0| Beattie, Mr Thomson|36.0000| Cherbourg| Winnipeg, MN| C-6| null| null| male|<br /> | 19|1st| 1|Beckwith, Mr Rich...|37.0000|Southampton| New York, NY| D-35| null| 5| male|<br /> | 20|1st| 1|Beckwith, Mrs Ric...|47.0000|Southampton| New York, NY| D-35| null| 5|female|<br /> +---+---+---+--------------------+-------+-----------+--------------------+-------+-----------------+-----+------+<br /> only showing top 20 rows</p> <p>scala> df.createOrReplaceTempView("titanicdata")</p> <p>
-
September 20, 2018 at 2:43 pm #5266DataFlair TeamSpectator
NOTE : Please note that, this API is metioned as ‘Experimental’ in Spark API documentation
-
-
AuthorPosts
Viewing 2 reply threads
- You must be logged in to reply to this topic.