创建google cloud storage notification 的权限问题

news2024/11/26 15:31:51

问题

根据google 的文档：
https://cloud.google.com/storage/docs/reporting-changes#command-line

明确表示，要创建storage notificaiton ，创建者(or service account) 只需要bucket 和 pubsub admin roles

在这里插入图片描述
但是实际上我在公司尝试为1个bucket 创建 notification 时遇到了错误：


ERROR: (gcloud.storage.buckets.notifications.create) User [xxxxx] does not have permission to access b instance [xxxx] (or it may not exist): 
xxxxx does not have serviceusage.services.use access to the Google Cloud project. Permission 'serviceusage.services.use' denied on resource (or it may not exist). This command is authenticated as xxxxx which is the active account specified by the [core/account] property.

首先什么是b instance, serviceusage.services.us 又是什么权限

由于我在公司所用的账号是terraform用的账号，权限我相信是足够的，也具有pubsub 和 bucket admin 的roles.

而且这个账号之前是创建过其他notification的

尝试几次后，够钟收工

尝试在家里reproduce 这个issue

首先创建 1个service account, 并且分配权限

account name 就叫 pubsub-bucket-adm

tf 脚本：

# create a resource for a new service account
resource "google_service_account" "service_account_pubsub_bucket_adm" {
  account_id   = "pubsub-bucket-adm"
  display_name = "service count for pubsub bucket admin"
  project      = var.project_id
}

# define the roles for the service account
resource "google_project_iam_binding" "service_account_pubsub_bucket_adm" {
  project = var.project_id
  role    = "roles/pubsub.admin"

  members = [
    "serviceAccount:${google_service_account.service_account_pubsub_bucket_adm.email}"
  ]
}

resource "google_project_iam_binding" "service_account_pubsub_bucket_adm_storage" {
  project = var.project_id
  role    = "roles/storage.admin"

  members = [
    "serviceAccount:${google_service_account.service_account_pubsub_bucket_adm.email}"
  ]
}

检查账号和权限

正常

[gateman@manjaro-x13 keys]$ list_service_account
+ gcloud iam service-accounts list
DISPLAY NAME                            EMAIL                                                  DISABLED
owner-test                              owner-test@jason-hsbc.iam.gserviceaccount.com          False
App Engine default service account      jason-hsbc@appspot.gserviceaccount.com                 False
terraform                               terraform@jason-hsbc.iam.gserviceaccount.com           False
vm-common                               vm-common@jason-hsbc.iam.gserviceaccount.com           False
service count for pubsub bucket admin   pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com   False
pubsub publisher a                      pubsub-publisher-a@jason-hsbc.iam.gserviceaccount.com  False
non-access                              non-access@jason-hsbc.iam.gserviceaccount.com          False
terraform2                              terraform2@jason-hsbc.iam.gserviceaccount.com          False
Compute Engine default service account  912156613264-compute@developer.gserviceaccount.com     False
[gateman@manjaro-x13 keys]$ list_roles pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com
+ gcloud projects get-iam-policy jason-hsbc '--flatten=bindings[].members' '--format=table(bindings.role)' --filter=bindings.members:pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com
ROLE
roles/pubsub.admin
roles/storage.admin
[gateman@manjaro-x13 keys]$

创建1个bucket

tf 脚本

resource "google_storage_bucket" "bucket-jason-hsbc-test" {
  name     = "${var.project_id}-test"
  project  = var.project_id
  location = var.region_id
}

[gateman@manjaro-x13 keys]$ gsutil ls
gs://dataflow-staging-europe-west2-912156613264/
gs://gcf-v2-sources-912156613264-europe-west2/
gs://gcf-v2-uploads-912156613264-europe-west2/
gs://jason-hsbc/
gs://jason-hsbc-dataflow/
gs://jason-hsbc-demo-src/
gs://jason-hsbc-demo-target/
gs://jason-hsbc-des/
gs://jason-hsbc-learning/
gs://jason-hsbc-raw/
gs://jason-hsbc-src/
gs://jason-hsbc-test/
gs://jason-hsbc_cloudbuild/
gs://linkedin_learning_56/
[gateman@manjaro-x13 keys]$

gs://jason-hsbc-test 这个新的bucket 已被创建

尝试去为这个bucket 创建notification

命令

gcloud storage buckets notifications create gs://jason-hsbc-test --topic=projects/jason-hsbc/topics/TopicA

错误重现

[gateman@manjaro-x13 keys]$ gcloud storage buckets notifications create gs://jason-hsbc-test --topic=projects/jason-hsbc/topics/TopicA
WARNING: Topic already exists: projects/jason-hsbc/topics/TopicA
WARNING: Retrying create notification request because topic changes may take up to 10 seconds to process.
ERROR: (gcloud.storage.buckets.notifications.create) User [pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com] does not have permission to access b instance [jason-hsbc-test] (or it may not exist): pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com does not have serviceusage.services.use access to the Google Cloud project. Permission 'serviceusage.services.use' denied on resource (or it may not exist). This command is authenticated as pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com which is the active account specified by the [core/account] property.
[gateman@manjaro-x13 keys]

解决

看error message 应该是缺了 serviceusage.services.use 这个permisson
就从点入手

查看有那些role 具有permisson serviceusage.services.use

https://console.cloud.google.com/iam-admin/roles?referrer=search&project=jason-hsbc

发现 Service Usage Consumer 应该是候选role
https://console.cloud.google.com/iam-admin/roles/details/roles%3Cserviceusage.serviceUsageConsumer?project=jason-hsbc

添加角色 Service Usage Consumer 给 pubsub-bucket-adm

tf 脚本

resource "google_project_iam_binding" "service_account_pubsub_bucket_adm_service_usage_comsumer" {
  project = var.project_id
  role    = "roles/serviceusage.serviceUsageConsumer"

  members = [
    "serviceAccount:${google_service_account.service_account_pubsub_bucket_adm.email}"
  ]
}

检查

[gateman@manjaro-x13 keys]$ list_roles pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com
+ gcloud projects get-iam-policy jason-hsbc '--flatten=bindings[].members' '--format=table(bindings.role)' --filter=bindings.members:pubsub-bucket-adm@jason-hsbc.iam.gserviceaccount.com
ROLE
roles/pubsub.admin
roles/serviceusage.serviceUsageConsumer
roles/storage.admin

重新尝试创建notification

[gateman@manjaro-x13 keys]$ gcloud storage buckets notifications create gs://jason-hsbc-test --topic=projects/jason-hsbc/topics/TopicA
WARNING: Topic already exists: projects/jason-hsbc/topics/TopicA
etag: '1'
id: '1'
kind: storage#notification
payload_format: JSON_API_V1
selfLink: https://www.googleapis.com/storage/v1/b/jason-hsbc-test/notificationConfigs/1
topic: //pubsub.googleapis.com/projects/jason-hsbc/topics/TopicA

[gateman@manjaro-x13 keys]$ gsutil notification list gs://jason-hsbc-test
projects/_/buckets/jason-hsbc-test/notificationConfigs/1
	Cloud Pub/Sub topic: projects/jason-hsbc/topics/TopicA

成功！