Author Archives: Robin Dong

Source code analysis for Autograd

Autograd is a convenient tool to automatically differentiate native Python and Numpy code. Let’s look at an example first:

The result is 3.2 f(x) = sqaure(x) + 1, its derivative is 2*x, so the result is correct. Function grad() actually return a ‘function object’, which is ‘grad_f’. When we… Read more »

Prediction of Red Wine Quality

In Kaggle platform, there is an example dataset about Quality of Red Wine. I wrote some code for it by using scikit-learn and pandas:

The results reported by snippet above:

Looks the most important feature to predict quality of red wine is ‘alcohol’. Intuitively, right?

Use PCA (Principal Component Analysis) to blur color image

I wrote an example of blurring color picture by using PCA from scikit-learn:

But it reports

The correct solution is transforming image to 2 dimensions shape, and inverse transform it after PCA:

It works very well now. Let’s see the original image and blurring image: Original Image… Read more »

Do tf.random_crop() operation on GPU

When I run code like:

it reports:

Looks operation tf.random_crop() doen’t have CUDA kernel implementation. Therefore I need to write it myself. The solution is surprisingly simple: write a function to do random_crop on one image by using tf.random_uniform() and tf.slice(), and then use tf.map_fn() to apply it… Read more »

Regularization loss in ‘slim’ library of Tensorflow

My python code using slim library to train classification model in Tensorflow:

It works fine. However, no matter what value the ‘weight_decay’ is, the training accuracy of the model could reach higher than 90% easily. It seems ‘weight_decay’ just doesn’t work. In order to find out the reason, I… Read more »

Using multi-GPUs for training in distributed environment of Tensorflow

I am trying to write code for training on multi-GPUs. The code is mainly from the example of ‘Distributed Tensorflow‘. I have changed the code slightly for runing on GPU:

But after launch the script below:

it reports:

Seems one MonitoredTrainingSession will occupy all the memory of… Read more »

Reinforcement Learning example for tree search

I have been learning Reinforcement Learning for about two weeks. Although haven’t go through all the course of Arthur Juliani, I had been able to write a small example of Q-learning now. This example is about using DNN for Q-value table to solve a path-finding-problem. Actually, the path is more… Read more »

Problems and solutions about building Tensorflow-1.8 with TensorRT 4.0

Problem: When compiling Tensorflow-1.8 with CUDA-9.2, it reports:

Solution: Add ‘/usr/local/cuda-9.2/lib64’ into ‘/etc/’ and run ‘sudo ldconfig’ to make it works. Problem: When compiling Tensorflow-1.8, it reports:

Solution: In ‘.tf_configure.bazelrc’ file, use real python location instead of soft link:

Problem: When running TensorRT, it reports:

Solution:… Read more »

Testing performance of Tensorflow’s fixed-point-quantization on x86_64 cpu

Google has published their quantization method on this paper. It use int8 to run feed-forward but float32 for back-propagation, since back-propagation need more accurate to accumulate gradients. I got a question right after reading the paper: why all the performance test works are on platform of mobile-phone (ARM architecture)? The… Read more »