I tried to run a basic example of the Google Vertex AI pipeline:

import kfp
from kfp.v2 import compiler
from kfp.v2.google.client import AIPlatformClient
from google.cloud import aiplatform
from google_cloud_pipeline_components import aiplatform as gcc_aip
project_id = PROJECT_ID
region = REGION
pipeline_root_path = PIPELINE_ROOT
@kfp.dsl.pipeline(
    name="automl-image-training-v2",
    pipeline_root=pipeline_root_path)
def pipeline(project_id: str):
    ds_op = gcc_aip.ImageDatasetCreateOp(
        project=project_id,
        display_name="flowers",
        gcs_source="gs://cloud-samples-data/vision/automl_classification/flowers/all_data_v2.csv",
        import_schema_uri=aiplatform.schema.dataset.ioformat.image.single_label_classification,
    )
    training_job_run_op = gcc_aip.AutoMLImageTrainingJobRunOp(
        project=project_id,
        display_name="train-iris-automl-mbsdk-1",
        prediction_type="classification",
        model_type="CLOUD",
        base_model=None,
        dataset=ds_op.outputs["dataset"],
        model_display_name="iris-classification-model-mbsdk",
        training_fraction_split=0.6,
        validation_fraction_split=0.2,
        test_fraction_split=0.2,
        budget_milli_node_hours=8000,
    )
    endpoint_op = gcc_aip.ModelDeployOp(
        project=project_id, model=training_job_run_op.outputs["model"]
    )
compiler.Compiler().compile(pipeline_func=pipeline,
        package_path='image_classif_pipeline.json')
api_client = AIPlatformClient(project_id=project_id, region=region)
response = api_client.create_run_from_job_spec(
    'image_classif_pipeline.json',
    pipeline_root=pipeline_root_path,
    parameter_values={
        'project_id': project_id
    })

but the script reported an error:

google.api_core.exceptions.PermissionDenied: 403 Permission 'aiplatform.pipelineJobs.create' denied on resource '//aiplatform.googleapis.com/projects/my_project/locations/us-central1' (or it may not exist).

I am using my own google account and could submit the job from GUI of GCP. But why didn’t I have permission to launch the job in my console?

Finally, the ops helped me found out the problem. By running

gcloud config set project my_project
gcloud auth application-default login

again. I noticed that I had set the environment variable GOOGLE_APPLICATION_CREDENTIALS to another account… Now the reason is obvious.

After unsetting the GOOGLE_APPLICATION_CREDENTIALS and re-login the default account. The Vertex AI pipeline job could be launched from my laptop now.

Related Posts