Can't write output to file from Apache Flink program in Eclipse

Live instructor-led & Self-paced Online Certification Training Courses (Big Data, Hadoop, Spark) Forums Apache Flink Can't write output to file from Apache Flink program in Eclipse

Viewing 2 reply threads
  • Author
    Posts
    • #6608

      I am running Flink program in eclipse, I am not able to write the output to a file byoutput.writeAsCsv("/home/output/out000000334111.csv");however output.print(); showing correct output on console.

      I have also tried other variant like writeAsText() but it also not working

      How can I write output to a file in Apache Flink?

    • #6609

      The problem with your code is that you are not using ExecutionEnvironment. The ExecutionEnvironment provides methods to control the job execution and to access the data from other Environment.If you wants to write the data your program is incomplete without using env.execute(); , this will allow to interact with the outside world.

    • #6610

      Hi

      if we create ajar file from ecllipse and run in linux its prints the whole output in single file as expected but in case of windows its creating multiple partitions to store the results.
      -output.txt (a folder gets created)
      –a (this has some part of output)
      –b(this has some part of output)
      –c(this has some part of output)
      –d(this has some part of output)

      ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
      DataSet<String> text = env.readTextFile(“C:\\Downloads\\text.txt”);
      //do the filter and other functions here
      data.writeAsText(“C:\\Downloads\\output.txt”);
      env.execute(“Executing program”);

Viewing 2 reply threads
  • You must be logged in to reply to this topic.