Author Archives: Robin Dong

A problem about running Argo

After I launched an Argo workflow, it just hanged on ContainerCreating stage. After waiting for more than 10 minutes, it hasn’t changed at all. Then I found this article. After using

I finally found out the problem in Message column:

That’s because I forgot to add secret to… Read more »

Some tips about BigQuery on GCP

Migrate SQL script from AWS Redshift to BigQuery

in Redshift should be changed to

in BigQuery. Since BigQuery doesn’t force type conversion, some NULL value in Redshift could be a NULL value or a ‘NULL’ string in BigQuery. Make sure you use both

and

for checking…. Read more »

Use `psql` to download data as CSV file

Although SQL WorkBench is a handy tool for querying AWS Redshift, we still need to CLI tool for automation. To install psql on MacOS, we need to

Then we could download data without using \copy but only –csv

Recent learned tips abou Numpy and Pandas

Precision After running this snippet:

It print out:

Why np.float32 and np.float64 have the same output? The answer is: displaying of numpy array need to set options. Let’s set option before print:

The result has became:

which looks much reasonable. Furthermore, why it prints out ‘0.1122334455667789’… Read more »

Use both ‘withParam’ and ‘when’ in Argo Workflows (on Kubernetes)

In Argo, we can use ‘withParam’ to create loop logic:

But in my YAML, it also use when in Argo:

When the NEED_RUN is 0, the Argo will report error since it can’t find the {{steps.generate.outputs.result}}. Seems the YAML parser of Argo will try to parse withParam before… Read more »

Be careful of the ternary operator in Python

The result will be:

Where is the last go? It goes with the no. The python interpreter will consider "no" / "last" under the else condition even it actually break the syntax rule. The correct way to write the ternary operator should be:

Now the result become:… Read more »