A example of Mesos Python Framework to calculate Pi

I have written an example of Mesos Framework by python. It simply calculate “Pi” by using Mento-Carlo algorithm
The whole source code is at https://github.com/RobinDong/mesos-python-examples/tree/master/calculate_pi
At beginning, I use python threading in “launchTask()”:

But I found out that the executors only spend a small part of CPU resource in slave machines (about 100%~150%, which is too low in a 8-cores server). First, I thought it may be limited by cgroup, but later, the answer is revealed: multi-threaded python application is actually single thread because of GIL.

With no choice, I have to change my code: the thread will launch a new process, then the new process will calculate the result and finally return the result to thread. It works fine in my Mesos cluster.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.