Robin on Linux – Page 3 – All about technology

Timezone in pods of Argo

Last week I noticed that the pod in Argo would give a UTC timezone even though the Argo configuration has set a AEDT timezone.

apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
  name: my-cron-workflow
  namespace: MYNAMESPACE
spec:
  schedule: "0 10 * * *"
  timezone: "Australia/Sydney"

Actually, this timezone in configuration is only used to give the time this scheduled job would run at, not the timezone in every pod.

To set up a unified timezone for all the pods, we need to use volume.

apiVersion: v1
kind: Pod
metadata:
  name: busybox-sleep
spec:
  containers:
  - name: busybox
    image: busybox
    args:
    - sleep
    - "1000000"
    volumeMounts:
    - name: tz-config
      mountPath: /etc/localtime
  volumes:
    - name: tz-config
      hostPath:
        path: /usr/share/zoneinfo/Australia/Sydney
        type: File

Install new driver for old Nvidia Tesla P100

I was trying to launch a VM instance with GPU on Google Cloud. But after trying T4, L4, and V100, they all reported “exceeding resource limit”, which means a lot of people in my region are using these types of GPUs.

Without choice, I launched a VM instance with an old Nvidia Tesla P100 (I first used it about 5 years ago). Then, I need to install its driver. But the installation process reported errors:

   *** Failed CC version check. ***

     SYMLINK /tmp/selfgz26389/NVIDIA-Linux-x86_64-515.105.01/kernel/nvidia/nv-kernel.o
     SYMLINK /tmp/selfgz26389/NVIDIA-Linux-x86_64-515.105.01/kernel/nvidia-modeset/nv-modeset-kernel.o
    CONFTEST: hash__remap_4k_pfn
    CONFTEST: set_pages_uc
    CONFTEST: list_is_first
    CONFTEST: set_memory_uc
...
	cc: error: unrecognized command-line option '-ftrivial-auto-var-init=zero'

At first glance, I suspect the GCC compiler is too old. After downgrading the GCC to gcc-10 and gcc-9, the error still existed.

Finally, I noticed that the driver of the Tesla P100 is very new (Release Date: 2023.3.30) and this page mentioned “gcc-12”. Therefore I upgraded the GCC to 12:

sudo apt install gcc-12
sudo ln -sf /usr/bin/gcc-12 /etc/alternatives/cc

Now the driver can be installed successfully.

The experience of using Google Cloud’s Text-to-Speech AI

Just using the Python API of Text-to-Speech AI to transform a PDF file to mp3 audio, as the example:

from google.cloud import texttospeech
from PyPDF2 import PdfReader

client = texttospeech.TextToSpeechClient()

reader = PdfReader("xxx.pdf")

voice = texttospeech.VoiceSelectionParams(
    language_code="cmn-CN", name="cmn-CN-Wavenet-B", ssml_gender=texttospeech.SsmlVoiceGender.MALE
)

audio_config = texttospeech.AudioConfig(
    audio_encoding=texttospeech.AudioEncoding.MP3,
    speaking_rate=0.8,
)

text = ""
index = 1
# try first 10 pages
for page in reader.pages[:10]:
    text += page.extract_text()

print(len(text))
synthesis_input = texttospeech.SynthesisInput(text=text)
response = client.synthesize_speech(
    input=synthesis_input, voice=voice, audio_config=audio_config
)

with open("outout.mp3", "wb") as out:
    out.write(response.audio_content)
    print("Written")

Very simple, right? But it just reported an error:

google.api_core.exceptions.InvalidArgument: 400 Either `input.text` or `input.ssml` is longer than the limit of 5000 bytes. This limit is different from quotas. To fix, reduce the byte length of the characters in this request, or consider using the Long Audio API: https://cloud.google.com/text-to-speech/docs/create-audio-text-long-audio-synthesis.

It seems the request is too long. Let’s use the “Long Audio API”:

from google.cloud import texttospeech
from PyPDF2 import PdfReader

client = texttospeech.TextToSpeechLongAudioSynthesizeClient()

reader = PdfReader("xxx.pdf")

voice = texttospeech.VoiceSelectionParams(
    language_code="cmn-CN", name="cmn-CN-Wavenet-B", ssml_gender=texttospeech.SsmlVoiceGender.MALE
)

audio_config = texttospeech.AudioConfig(
    audio_encoding=texttospeech.AudioEncoding.LINEAR16,
    speaking_rate=0.8,
)

text = ""
index = 1
for page in reader.pages[:10]:
    text += page.extract_text()

print(len(text))
synthesis_input = texttospeech.SynthesisInput(text=text)
request = texttospeech.SynthesizeLongAudioRequest(
    parent="projects/robin-00000/locations/us",
    input=synthesis_input, voice=voice, audio_config=audio_config,
    output_gcs_uri="gs://robin_tts/xxx.mp3"
)

operation = client.synthesize_long_audio(request=request)
result = operation.result(timeout=300)
print(result)

It couldn’t work still:

google.api_core.exceptions.InvalidArgument: 400 The long audio API does not support the language zh. Supported languages: en, es.

Okay. It doesn’t support the Chinese language. Then, what should I do if I want to translate a Chinese pdf to mp3? Convert them page by page into 500 mp3 files? This is terrible. Even for the short mp3 it generated, it definitely sounds like a machine, not a human.

Google has the state-of-the-art technology of deep learning but some of their products in the cloud are ridiculously hard to use (such as Vertex AI, and this Text-to-Speech).

After some searching (at least Google search is perfect as before), I found this NaturalReader. Surprisingly, it supports the Chinese language and the voice is as well as a real human. The only problem is it is very expensive for individual users.

Using Javascript to load ONNX model for Object Detection job

Although having used the YOLOv5 model several times, I haven’t used its corresponding ONNX model before. This time, I met a use case to run its ONNX model using Javascript.

To learn and debug the code, I installed node.js and started my Javascript trip. This snippet helps me a lot to understand the Non-Max Suppression algorithm again (I used NMS algo many years). After studying it and also the implementation in the YOLOv5, I finally knocked together the Javascript code to use the ONNX model of YOLOv5 to detect objects:

https://github.com/RobinDong/onnx_js/blob/main/detect.js
Why the “YOLOV5S_CLASSES” is 85? That’s because in YOLO series it uses the first four floats as coordinates of boxes (center x, center y, width, height), the fifth as “object confidence”, and the left 80 floats as “class confidence”.

For anyone who is interested in it, you could use the below steps to run:

Install node.js on your computer or laptop
Run npm install fs inkjet onnxruntime-node to install js packages
Run node detect.js lorikeet.jpg to detect objects in the image “lorikeet.jpg” and output to file “output.txt”
Run python3 draw.py lorikeet.jpg to show the image of the result by using data in “output.txt”

Hope it could be useful for someone.

Accelerate augmentation of bird audio

audiomentions is a very convenient library for my bird sound classification. As the code below:

from audiomentations import Compose, AddGaussianNoise, AddGaussianSNR, TimeStretch, PitchShift

        self.augment = Compose([
            AddGaussianNoise(min_amplitude=0.005, max_amplitude=0.015, p=poss),
            AddGaussianSNR(min_snr_in_db=5.0, max_snr_in_db=40.0, p=poss),
            TimeStretch(min_rate=0.8, max_rate=1.2, p=poss),
            PitchShift(min_semitones=-2, max_semitones=2, p=poss)
        ])

These four augmentation methods are enough for current training. But the PitchShift method will cost a lot of CPU resources therefore the GPU couldn’t run to full load and the CPU usage jumps to 100%.

Failed to find an audio augmentation library that uses GPU, I started to check the source code of “audiomentions” and noticed that it uses librosa as its implementation:

        try:
            pitch_shifted_samples = librosa.effects.pitch_shift(
                samples, sr=sample_rate, n_steps=self.parameters["num_semitones"]
            )
        except librosa.util.exceptions.ParameterError:

Then the code of “librosa” for “pitch_shift”:

def pitch_shift(
    y: np.ndarray,
    *,
    sr: float,
    n_steps: float,
    bins_per_octave: int = 12,
    res_type: str = "soxr_hq",
    scale: bool = False,
    **kwargs: Any,
) -> np.ndarray:

The default “res_type” for “pitch_shift” is “soxr_hq”. This is a slow resource. After changing “it”res_type” to “linear” in “audiomentions”, the CPU usage jumps back to 50% on my desktop and the GPU ramps up to 100% when training.

—— 2023.07.28 ——

Thanks for the correction from Iver.

After I run this test snippet:

import time
import librosa

sound, sr = librosa.load("./song/background/AirportAnnouncements_1.wav")

for resource in [None, "linear", "soxr_hq", "kaiser_best"]:
    begin = time.time()
    for _ in range(10):
        if resource:
            librosa.effects.pitch_shift(sound, sr=sr, n_steps=1, res_type=resource)
        else:
            librosa.effects.pitch_shift(sound, sr=sr, n_steps=1)
    if resource:
        print(f"{resource} time:", time.time() - begin)
    else:
        print("default time:", time.time() - begin)

and got the result

default time: 8.455572366714478
linear time: 3.3037502765655518
soxr_hq time: 3.3474862575531006
kaiser_best time: 8.467342615127563

Iver is right: the soxr_hq is as fast as linear. And the actual default res_type of librosa which I was using is kaiser_best.

Simple Python code of Knapsack Problem

Just write this snippet for my practice of 0-1 Knapsack Problem:

values  = [1, 2, 3, 4, 5]
weights = [3, 2, 1, 9, 6]
max_weight = 12

def knappack():
    n = len(values)
    dp = [[0] * (max_weight+1) for _ in range(n)]
    print(dp)

    for i in range(n):
        for j in range(1, max_weight+1):
            if weights[i] > j:
                dp[i][j] = dp[i-1][j]
            else:
                dp[i][j] = max(dp[i-1][j], dp[i-1][j-weights[i]] + values[i])

    print(dp[-1][-1])

knappack()

An incorrect understanding of me for Skip-list

After reading the classic paper about Skip-list, I tried to implement it by myself. But then I found a line of pseudo-code in the “Delete” function that I couldn’t understand:

Seems all the elements in “update” will point to “x”, so why do we need to check here? Maybe I can ignore this checking. Now here comes a part of my code:

    def erase(self, num: int) -> bool:
        update = Node(-1)
        
        curr = self._head
        for level in range(MAX_LEVEL-1, -1, -1):
            while curr._forward[level] and curr._forward[level]._key < num:
                curr = curr._forward[level]
            update._forward[level] = curr
        
        curr = curr._forward[0]
        if curr == None or curr._key != num:
            return False
        curr._count -= 1
        if curr._count > 0:
            return True
        
        for level in range(MAX_LEVEL):
            update._forward[level]._forward[level] = curr._forward[level]
            
        del curr
        return True

But unfortunately, it failed for the test case. In the debugging process, I realized that not all elements in “update” will point to “x”. Le’ts just take the figure-1 from the paper as my example:

As above, imaging we are deleting node “17”. The “forward[1]” (index start from 0, this is the difference of my code with the paper) of node “9” is pointing to node “17” so it should be redirect to node “25”. But the “forward[3]” of node “6” is pointing to “NIL”, and shouldn’t be redirected to node “25” because “node6._forward[3]” didn’t point to node “17”. The situation is the same for “forward[4]” and beyond, of node “6”.

This is why the last few lines of my code should be:

......
        
        for level in range(MAX_LEVEL):
            if update._forward[level]._forward[level] != curr:
                break
            update._forward[level]._forward[level] = curr._forward[level]
            
        del curr
        return True

Just like the pseudo-code in the paper!

I am really respect to these academic guys — everytime I thought they were wrong is actually I missed something.

Extract only birds sound from audio

This paper introduced a method to extract only segments with bird sound from an audio file. Since the paper didn’t give any code, I started to write it by myself.

Here is the Python implementation:

import cv2
import time
import torch
import librosa
import soundfile as sf
import numpy as np

from torchlibrosa.stft import LogmelFilterBank, Spectrogram

class CFG:
    n_fft = 2048
    hop_length = 512
    sample_rate = 32000
    n_mels = 64
    fmin = 150
    fmax = 150000

class SignalExtractor:
    def __init__(self):
        self.spectrogram_extractor = Spectrogram(
            n_fft=CFG.n_fft, hop_length=CFG.hop_length, win_length=CFG.n_fft, window="hann",
            center=True, pad_mode="reflect", freeze_parameters=True)
        # Logmel feature extractor
        self.logmel_extractor = LogmelFilterBank(sr=CFG.sample_rate, n_fft=CFG.n_fft,
            n_mels=CFG.n_mels, fmin=CFG.fmin, fmax=CFG.fmax, ref=1.0, amin=1e-10, top_db=None, freeze_parameters=True)
        self.factors = [2.0, 1.8, 1.6, 1.4, 1.2, 1.1]
        self.kernel_size = 15
        self.sn_threshold = 0.2

    def extract(self, input):
        x = torch.from_numpy(input)
        x = x[None, :].float()

        x = self.spectrogram_extractor(x)
        x = self.logmel_extractor(x)

        x = x.squeeze(0).squeeze(0)
        x = x.permute(1, 0).numpy()
        x = x - np.amin(x)

        for factor in self.factors:
            sound, sn_ratio = self._factor_extract(input, x, factor)
            if sn_ratio >= self.sn_threshold:
                break

        return sound, sn_ratio

    def _factor_extract(self, input, x, factor: float):
        rows, cols = x.shape
        row_median = np.median(x, axis=1)
        row_median_matrix = np.tile(row_median, (cols, 1)).T * factor
        col_median = np.median(x, axis=0)
        col_median_matrix = np.tile(col_median, (rows, 1)) * factor

        y = x > row_median_matrix
        z = x > col_median_matrix
        res = np.logical_and(y, z) + np.zeros(x.shape)

        kernel = np.ones((self.kernel_size, self.kernel_size), np.uint8)
        img = cv2.dilate(res, kernel, iterations=1)

        indicator = np.sum(img, axis=0)
        chunk_size = input.shape[0] // indicator.shape[0]
        sounds = []
        for index, chunk in enumerate(indicator):
            if chunk > 0:
                sounds.append(input[index*chunk_size:(index+1)*chunk_size])
        if len(sounds) <= 0:
            return None, 0.0
        sound = np.concatenate(sounds)
        return sound, sound.shape[0]/input.shape[0]

The implementation has some differences from the method in the paper:

I didn’t use erosion since dilation is good enough for picking up the bird-sound segment
three times bigger than median is too strict for most audio files, so I use an array of ratios. When the 2.0 ratio couldn’t pick up any bird sound, the code will automatically try a 1.8 ratio etc.
I used a big kernel (15, 15) for dilation since it works well in my samples

The original sample:

After extracteing only bird sounds:

Intel extension for PyTorch

Trying to test the Intel extension for PyTorch in my project, but it reported errors:

Traceback (most recent call last):                                                                                                                                                                                                          
  File "reviewjpgs_optimaztion_testing.py", line 27, in <module>                                                                                                                                                                            
    import intel_extension_for_pytorch as ipex                                                                                                                                                                                              
  File "/home/hero/.pyenv/versions/3.8.12/lib/python3.8/site-packages/intel_extension_for_pytorch/__init__.py", line 11, in <module>                                                                                                        
    from .cpu import _cpu_isa                                                                                                                                                                                                               
  File "/home/hero/.pyenv/versions/3.8.12/lib/python3.8/site-packages/intel_extension_for_pytorch/cpu/__init__.py", line 1, in <module>                                                                                                     
    from . import runtime                                                                                                                                                                                                                   
  File "/home/hero/.pyenv/versions/3.8.12/lib/python3.8/site-packages/intel_extension_for_pytorch/cpu/runtime/__init__.py", line 3, in <module>                                                                                             
    from .multi_stream import MultiStreamModule, get_default_num_streams, \
  File "/home/hero/.pyenv/versions/3.8.12/lib/python3.8/site-packages/intel_extension_for_pytorch/cpu/runtime/multi_stream.py", line 4, in <module>
    import intel_extension_for_pytorch._C as core
ImportError: /home/hero/.pyenv/versions/3.8.12/lib/python3.8/site-packages/intel_extension_for_pytorch/lib/libintel-ext-pt-cpu.so: undefined symbol: _ZNK3c1010TensorImpl22is_strides_like_customENS_12MemoryFormatE

The answer is quite tricky: need to install the IPEX package with the same version of PyTorch.

After the testing of both torch.jit.trace and this IPEX, we found out that `torch.jit.trace` could boost the performance of prediction significantly but IPEX could not.

Use a specific service account in the Argo job

I created a simple Argo job to pull messages from a Google Cloud Pub/Sub topic. Permission has been given to the service account of GKE’s workload identity. But the Argo job failed with errors:

argo submit example.json -n argoproj

hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
hello-world-pqbm5:     return callable_(*args, **kwargs)
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/grpc/_channel.py", line 1030, in __call__
hello-world-pqbm5:     return _end_unary_response_blocking(state, call, False, None)
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/grpc/_channel.py", line 910, in _end_unary_response_blocking
hello-world-pqbm5:     raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
hello-world-pqbm5: grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
hello-world-pqbm5:      status = StatusCode.PERMISSION_DENIED
hello-world-pqbm5:      details = "User not authorized to perform this action."
hello-world-pqbm5:      debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.69.95:443 {grpc_message:"User not authorized to perform this action.", grpc_status:7, created_time:"2023-05-15T01:10:43.128528579+00:00"}"
hello-world-pqbm5: >
hello-world-pqbm5: 
hello-world-pqbm5: The above exception was the direct cause of the following exception:
hello-world-pqbm5: 
hello-world-pqbm5: Traceback (most recent call last):
hello-world-pqbm5:   File "<string>", line 26, in <module>
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/google/pubsub_v1/services/subscriber/client.py", line 1495, in pull
hello-world-pqbm5:     response = rpc(
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
hello-world-pqbm5:     return wrapped_func(*args, **kwargs)
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/google/api_core/retry.py", line 349, in retry_wrapped_func
hello-world-pqbm5:     return retry_target(
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/google/api_core/retry.py", line 191, in retry_target
hello-world-pqbm5:     return target()
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/google/api_core/timeout.py", line 120, in func_with_timeout
hello-world-pqbm5:     return func(*args, **kwargs)
hello-world-pqbm5:   File "/usr/local/lib/python3.9/dist-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
hello-world-pqbm5:     raise exceptions.from_grpc_error(exc) from exc
hello-world-pqbm5: google.api_core.exceptions.PermissionDenied: 403 User not authorized to perform this action.

Thanks to my colleagues. They remind me that an Argo job needs to specify a service account when running in the workload identity namespace.

argo submit example.json -n argoproj --serviceaccount argo-workflow

Or, I can add this service account to the YAML file:

apiVersion: argoproj.io/v1alpha1
kind: Workflow                  # new type of k8s spec
metadata:
  generateName: hello-world-    # name of the workflow spec
spec:
  entrypoint: whalesay          # invoke the whalesay template
  serviceAccountName: argo-workflow