Tag Archives: pandas

Some tips about BigQuery on GCP

Migrate SQL script from AWS Redshift to BigQuery

in Redshift should be changed to

in BigQuery. Since BigQuery doesn’t force type conversion, some NULL value in Redshift could be a NULL value or a ‘NULL’ string in BigQuery. Make sure you use both

and

for checking…. Read more »

Recent learned tips abou Numpy and Pandas

Precision After running this snippet:

It print out:

Why np.float32 and np.float64 have the same output? The answer is: displaying of numpy array need to set options. Let’s set option before print:

The result has became:

which looks much reasonable. Furthermore, why it prints out ‘0.1122334455667789’… Read more »

To construct DataFrame more effectively

The old code of python looks like:

This snippet above will cost 7 seconds to run on my laptop. Actually, pd.concat() is an expensive operation for CPU. So let’s replace it with common python dictionary:

This snippet only costs 0.03 seconds, which is more effective.

Some tips about Python, Pandas, and Tensorflow

There are some useful tips for using Keras and Tensorflow to build models. 1. Using applications.inception_v3.InceptionV3(include_top = False, weights = ‘Imagenet’) to get pretrained parameters for InceptionV3 model, the console reported:

The solution is here. Just install some packages:

2. Could we use ‘add’ to merge two DataFrames… Read more »

Use pandas and matplotlib to draw line chart

I have two CSV files. Their content looks like:

The simplest way to load and draw them is by using pandas and matplotlib.

The figure draw out by this snippet is shown below: