I already knew that Vertex AI is a new product of GCP so it tends to be unstable. But I didn’t imagine it lacks a vital function: log errors or exceptions thrown by python code.

After I added code from this article, some of my steps, fortunately, reported error logs. But for some other steps, they jump out with just a simple message:

Don’t be confused. The blank above is from the GUI of GCP because there is no record.

If a pipeline framework couldn’t even capture all the errors from its steps, I would not dare to use it in production.