0

I am trying to delete a csv file from scala I have tried this but it doesn't work, it doesn't delete the file; result is a dataset/dataframe:

result.withColumn("features", stringify(result.col("features")))
      .write
      .mode(SaveMode.Overwrite)
      .option("header","true")
      .csv("data/temporaryBasis")
...
    new File("data/temporaryBasis").delete()
Ivan Stanislavciuc
  • 7,140
  • 15
  • 18
informatic_aa
  • 11
  • 3
  • Welcome to SO. Can you clarify your question: you want to delete a file on HDFS from a Spark application? – Gaël J Jun 19 '22 at 09:15
  • Yes, that is what i want to do. – informatic_aa Jun 20 '22 at 12:36
  • Does this answer your question? [How do I delete files in hdfs directory after reading it using scala?](https://stackoverflow.com/questions/45104284/how-do-i-delete-files-in-hdfs-directory-after-reading-it-using-scala) – Gaël J Jun 20 '22 at 16:28
  • not exactly, this is how I read my cvs file: var data=spark.read.csv("data/dataset") – informatic_aa Jun 20 '22 at 21:03

0 Answers0