- This topic has 2 replies, 1 voice, and was last updated 7 years, 10 months ago by
.
-
Posts
-
-
Hi,
I am getting below exceptions while running the Spark with Scala application,
1. Right click on Scala program -> Run As -> Scala application
Exception in thread “main” java.lang.IllegalArgumentException: System memory 259522560 must be at least 471859200. Please increase heap size using the –driver-memory option or spark.driver.memory in Spark configuration.
2. if i run spark-submit,
@localhost spark-1.5.2]$ bin/spark-submit –class com.scala.spark.demo.Wordcount /home/Desktop/scalasparkWC.jar /home/Desktop/wordcount.parquet /home/Desktop/Apr_2017_003
Error occurred during initialization of VM
Could not reserve enough space for object heap
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
[ubuntu@localhost spark-1.5.2]$ -
When you are executing program from the Eclipse, change following settings:
In eclipse go to Run configuration > Arguments > VM Arguments and set max heapsize like -Xmx512m
The code provided is for Spark 2.0 and you are using Spark 1.5, Use Spark 2.0, the error will be resolved.
Refer Create Spark Project in Scala with Eclipse without Maven
-
-
- You must be logged in to reply to this topic.
