I created a simple Argo job to pull messages from a Google Cloud Pub/Sub topic. Permission has been given to the service account of GKE’s workload identity. But the Argo job failed with errors:
Shell
x
1
1
argo submit example.json -n argoproj
Console
1
29
29
1
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
2
hello-world-pqbm5: return callable_(*args, **kwargs)
3
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/grpc/_channel.py", line 1030, in __call__
4
hello-world-pqbm5: return _end_unary_response_blocking(state, call, False, None)
5
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/grpc/_channel.py", line 910, in _end_unary_response_blocking
6
hello-world-pqbm5: raise _InactiveRpcError(state) # pytype: disable=not-instantiable
7
hello-world-pqbm5: grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
8
hello-world-pqbm5: status = StatusCode.PERMISSION_DENIED
9
hello-world-pqbm5: details = "User not authorized to perform this action."
10
hello-world-pqbm5: debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.69.95:443 {grpc_message:"User not authorized to perform this action.", grpc_status:7, created_time:"2023-05-15T01:10:43.128528579+00:00"}"
11
hello-world-pqbm5: >
12
hello-world-pqbm5:
13
hello-world-pqbm5: The above exception was the direct cause of the following exception:
14
hello-world-pqbm5:
15
hello-world-pqbm5: Traceback (most recent call last):
16
hello-world-pqbm5: File "<string>", line 26, in <module>
17
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/google/pubsub_v1/services/subscriber/client.py", line 1495, in pull
18
hello-world-pqbm5: response = rpc(
19
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
20
hello-world-pqbm5: return wrapped_func(*args, **kwargs)
21
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/google/api_core/retry.py", line 349, in retry_wrapped_func
22
hello-world-pqbm5: return retry_target(
23
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/google/api_core/retry.py", line 191, in retry_target
24
hello-world-pqbm5: return target()
25
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/google/api_core/timeout.py", line 120, in func_with_timeout
26
hello-world-pqbm5: return func(*args, **kwargs)
27
hello-world-pqbm5: File "/usr/local/lib/python3.9/dist-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
28
hello-world-pqbm5: raise exceptions.from_grpc_error(exc) from exc
29
hello-world-pqbm5: google.api_core.exceptions.PermissionDenied: 403 User not authorized to perform this action.
Thanks to my colleagues. They remind me that an Argo job needs to specify a service account when running in the workload identity namespace.
Shell
1
1
1
argo submit example.json -n argoproj --serviceaccount argo-workflow
Or, I can add this service account to the YAML file:
YAML
1
7
1
apiVersion argoproj.io/v1alpha1
2
kind Workflow # new type of k8s spec
3
metadata
4
generateName hello-world- # name of the workflow spec
5
spec
6
entrypoint whalesay # invoke the whalesay template
7
serviceAccountName argo-workflow