Robin on Linux – Page 33 – All about technology

A example of Mesos Python Framework to calculate Pi

I have written an example of Mesos Framework by python. It simply calculate “Pi” by using Mento-Carlo algorithm
The whole source code is at https://github.com/RobinDong/mesos-python-examples/tree/master/calculate_pi
At beginning, I use python threading in “launchTask()”:

class PiExecutor(mesos.interface.Executor):
    def launchTask(self, driver, task):
        def run_thread():
        ......
        thread = threading.Thread(target=run_thread)
        thread.start()

But I found out that the executors only spend a small part of CPU resource in slave machines (about 100%~150%, which is too low in a 8-cores server). First, I thought it may be limited by cgroup, but later, the answer is revealed: multi-threaded python application is actually single thread because of GIL.
With no choice, I have to change my code: the thread will launch a new process, then the new process will calculate the result and finally return the result to thread. It works fine in my Mesos cluster.

Override the init of BaseHTTPRequestHandler in python

Currently, I am writing some python program for the front of our storage system. By using ‘BaseHTTPRequestHandler’ and ‘ThreadedHTTPServer’, we could implement a simple multi-thread http server quickly. But after add ‘__init__()’ for our ‘MyHandler’, it doesn’t work correctly now:

class MyHandler(BaseHTTPRequestHandler):
    def __init__(self):
        self.request_id = ''

Then I found this statement in python docs:

Subclasses should not need to override or extend the __init__() method.

But thanks for the code example, we could override ‘__init__’ of BaseHTTPRequestHandler’ this way:

class MyHandler(BaseHTTPRequestHandler):
    def __init__(self, request, client_address, server):
        BaseHTTPRequestHandler.__init__(self, request, client_address, server)
        self.request_id = ''

Solve a USB network card problem

I am doing some source code porting works on linux kernel recently.
After I reboot my server (ubuntu 14.04) into new kernel version of 3.19.8, it can’t be connected by using ssh but only by using serial port.
The server is using a USB network card, so firstly I suspect some kernel driver for USB NIC has missing in .config file. Therefore I boot back into the old version kernel and try to find some useful information in ‘dmesg’:

asix 3-1.4:1.0 eth5: register 'asix' at usb-0000:00:11.0-1.4, ASIX AX88772B USB 2.0 Ethernet, 00:24:9c:04:44:28
......
eth3: link up, 100Mbps, full-duplex, lpa 0x45E1

The eth3 is using MII port, so when I try to grep “link up …lpa” in 3.19.8 kernel source code, I find out it must be printed by these codes in drivers/net/mii.c:

unsigned int mii_check_media (struct mii_if_info *mii,
                  unsigned int ok_to_print,
                  unsigned int init_media)
{
......
    if (ok_to_print)
        netdev_info(mii->dev, "link up, %uMbps, %s-duplex, lpa 0x%04X\n",
                lpa2 & (LPA_1000FULL | LPA_1000HALF) ? 1000 :
                media & (ADVERTISE_100FULL | ADVERTISE_100HALF) ?
                100 : 10,
                duplex ? "full" : "half",
                lpa);
......

The only place “ASIX AX88772B” driver call mii_check_media is in drivers/net/usb/asix_devices.c:

static int ax88772_link_reset(struct usbnet *dev)
{
    u16 mode;
    struct ethtool_cmd ecmd = { .cmd = ETHTOOL_GSET };
    mii_check_media(&dev->mii, 1, 1);
    mii_ethtool_gset(&dev->mii, &ecmd);
    mode = AX88772_MEDIUM_DEFAULT;
......
static const struct driver_info ax88772b_info = {
    .description = "ASIX AX88772B USB 2.0 Ethernet",
    .bind = ax88772_bind,
    .unbind = ax88772_unbind,
    .status = asix_status,
    .link_reset = ax88772_link_reset,
    .reset = ax88772_reset,
    .flags = FLAG_ETHER | FLAG_FRAMING_AX | FLAG_LINK_INTR |
             FLAG_MULTI_PACKET,
    .rx_fixup = asix_rx_fixup_common,
    .tx_fixup = asix_tx_fixup,
    .data = FLAG_EEPROM_MAC,
};

So far, the reason is that: the system does not “reset” the USB network card after booting up. But why it only forget to reset USB NIC in 3.19.8 kernel? After checking the /etc/network/interfaces:

auto eth0
iface eth0 inet dhcp
auto eth3
iface eth3 inet dhcp
auto usbnet0

the answer is: the device name of the USB NIC has been changed to “eth5” by udevd in 3.9.18 kernel (new version kernel recognise new network port so eth0/eth1/eth2/eth3/eth4 all has been occupied by system) so the network scripts can’t start it up.
Solution:
Fix the name of USB NIC by adding below content into /etc/udev/rules.d/70-persistent-net.rules:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:24:9c:04:44:28", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="usbnet0"

and add configurations into /etc/network/interfaces to start “usbnet0” up automatically:

auto usbnet0
iface usbnet0 inet dhcp

puppet 3 certification problem on centos 7

I configure the puppet master and agent followed by this step. But when I run “puppet agent -t”, it report error:

[root@iZ25s7c99ynZ ~]# puppet agent --test
Info: Creating a new SSL key for iz25s7c99ynz
Info: Caching certificate for ca
Info: csr_attributes file loading from /etc/puppet/csr_attributes.yaml
Info: Creating a new SSL certificate request for iz25s7c99ynz
Info: Certificate Request fingerprint (SHA256): E0:17:BA:8F:0A:97:FC:90:84:75:96:75:CC:DB:B8:07:C8:48:AD:CA:26:5C:E5:5B:01:4B:9E:71:6A:52:C9:AE
Info: Caching certificate for iz25s7c99ynz
Info: Caching certificate_revocation_list for ca
Error: Could not request certificate: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [certificate revoked for /CN=puppet.builder1.com]

My OS version is “Centos 7” and puppet version is “3.7.5”. After I have tried the way as this page answered, the problem still exists. Therefore, I write down the final correct operations here for future reference:

#In master
rm -rf /var/lib/puppet/ssl/*
service httpd stop
## use Ctrl + c to break when you see "Notice: Starting Puppet master version..."
puppet master --no-daemonize --verbose
service httpd start
#In agent
rm -rf /var/lib/puppet/ssl/*
puppet agent -t

How to set the value of “$releasever” permanently for yum

In a test server I typed “sudo yum update”, it reported errors like:

$sudo yum update
Loaded plugins: branch, fastestmirror, security
Setting up Update Process
Bad id for repo: base.$releasever.x86_64, byte = $ 8
Bad id for repo: base.$releasever.noarch, byte = $ 8
Bad id for repo: ops.$releasever.x86_64, byte = $ 4
Bad id for repo: ops.$releasever.noarch, byte = $ 4
Loading mirror speeds from cached hostfile
http://mirrors.xxx.com/centos/%24releasever/os/x86_64/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found"
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: base. Please verify its path and try again<

Then I found this web in google for introducing how to get the value of "$releasever", but it does not tell us how to set "$releasever" permanently. Therefore, I have to search word like "releasever" in the source code of 'yum':

rpm -ql yum|xargs sudo grep releasever * -rn

Finally the source code in '/usr/lib/python2.6/site-packages/yum/__init__.py' comes out:

releasever=self.conf.yumvar['releasever']

But where does 'self.conf.yumvar' get its values? The answer is '/etc/yum/vars/'. After

sudo bash -c "echo 6 > /etc/yum/vars/releasever"

My 'sudo yum update' works correctly now.

Vault Conference 2015, in Boston

Vault Conference 2015

Registration
vault

First talk, and the sponsors
vault

John Spray and his talk about development updates of Ceph
vault

I and Coly also give a presentation about our Cold Storage project, the slide is here: Lambert: Achieve High Durability, Low Cost & Flexibility at Same Time — open source cold storage engine for ExaBytes data in Alibaba
vault

I am appreciating the help from open source community, seems my oral English is not bad 🙂

Trip to Boston

Last week, I flight to Boston, MA to attend the Vault Conference 2015. This is my first time for coming to Boston.

The Boston Common
Boston

The Charles River, from MIT side.
Boston

The Hall of MIT
Boston

In the Havard Square
Boston

The well-known Havard Book Store
Boston

The Old Corner Book Store, it used to be a real book store about 100 years ago 🙂

128bit atomic operation in arm64

In x86_64 architecture, we could use cmpxchg16b to compare and exchange a 128bit value in one atomic operation. But how to operate a 128bit value in aarch64 architecture (arm64) machine ? The answer is __atomic_compare_exchange().
As below code:

#include 
....
__int128 value = 0, value1 = 0, value2 = 1;
__int128 ptr = &value;
__int128 expected = &value1;
__int128 desired = &value2;
__atomic_compare_exchange(ptr, expected, desitre, 0, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);
....

use “gcc test.c -o test” to compile code, but it reports:

 undefined reference to `__atomic_compare_exchange_16'

The reason is we have not use “libatomic.so”, so use “gcc test.c -o test -latomic” to compile and link the code agin. It works in Fedora 21 (aarch64).

Books I read in year 2014

Buried Cities: Pompeii, Olympia, Mycenae

I have watched “Fantastic Mr Fox” four years ago, but just read its novel this year. It’s really a interesting story even I am not a child anymore. The three bad farmers want to kill Mr Fox and his whole family, but Mr Fox just steal their goods without hurting anybody, which is a good idea to me 🙂
“Interstellar” is no doubt the most amazing science fiction this year. This is a story about a father travel a very very long time and space to rescue his children, and also, his descendants. The most attractive part of this novel for me is not those fashion or professional words like “wormhole”, “black hole” etc, but the struggling of the lone father (or the human being). It also give me much power and courage to scrabble out of the traps in my own life.

A fight to save a old server ….

Coly, who is my old colleague, bought a server computer in 2010. This server, which worth about 30,000 Yuan(RMB) 4 years ago, is composed by a ASUS KGPE-D16 motherboard, two AMD-6172 CPUs, and a 670W power supply. Recently, we need many servers to build some software, so I beg Coly to bring his 4 years old server to our small computer room in our office, and he promised.
After installing Linux on this server, I found out that only one CPU could boot up and run, the other one can’t be recognized by operation system. I asked Coly why, and he answered me maybe one CPU was broken.
I thought a mother-board with two sockets can only run one CPU was a great waste; therefore I bought a new CPU and setup it on the mother-board. But after that, the whole server couldn’t boot up totally. Having tried many ways but all failed, I turn to ask Coly to rescue his own old server.
At last, Coly come here and open the lid of the server case, and becomes astonishing very soon.
Coly: What on earth have you done to my server! Have you killed a man who sales silicone grease? You paste too much silicone grease on the CPU, even the edge of the sockets also be contaminated!
Me: Sorry, I just want to make sure the CPU will not be too hot.
Coly: Yeah, they will never be hot now —— because they can’t even run.
Then, Coly kneels on the floor and begin to clean all the silicon grease on the mother-board, which is a really hard work.

But, after clearing, the server still could not boot up.
Coly: Maybe the mother-board has broken, or maybe power supply got a shot-circuit. How much is a KGPE-D16 motherboard in TAOBAO (a very famous e-commerce website in China)? It cost me 8,000 Yuan 4 years ago.
Me: About 800 Yuan.
Coly: ….Mother board becomes cheap so fast. Then, how much is a 670W power supply in TAOBAO? It was 800 Yuan before.
Me: En, about 600 Yuan.
Coly: Ah-ha, looks power supply is a hedging tool for preserving monetary value. I reckon I will buy a large mount of power supply instead of stocks or gold to preserve my money .
In my opinion, a power supply is more stable than a mother board, so I bought a new mother board and luckily boot up the server with two CPU (24 cores). At last, no servers or machines are wasted, in my hand 🙂