Why my Spark job hangs?

      No Comments on Why my Spark job hangs?

After running my small application for Spark of Machine Learning , the job hangs and the Spark UI for it display nothing for more than 5 minutes.
That is weird and I see some logs in UI of yarn:

I haven’t any IP looks like “110.75.x.x”. Why is the Spark job trying to connect it ?
After reviewing the code carefully, I find out the problem:

It is me who forget to add IP to URI of HDFS. Thus, the correct code should be:

Now the application runs correctly.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.