This article recommend using “return” to jump out of a PySpark application. But after I did by following what he said. It reports error:

  File "test.py", line 333
    return
    ^
SyntaxError: 'return' outside function

Seems it can’t work. After trying to run PySpark application on my own laptop, I finally got the correct answer:

import sys
if df.rdd.isEmpty():
  sys.exit(0)

Related Posts

  • A problem of using Pyspark SQL

    Here is the code: from pyspark.sql import SQLContext from pyspark.context import SparkContext from pyspark.sql.types import…

  • Some hints on Dataproc

    When running a job in the cluster of Dataproc, it reported: java.util.concurrent.ExecutionException: java.lang.ClassNotFoundException: Failed to…

  • Deploy Hive on Spark

    The Mapreduce framework is too small for realtime analytic query, so we need to change…