aws eks理解和使用podidentity为pod授权

news2024/9/21 8:03:43

参考链接

https://www.amazonaws.cn/new/2024/amazon-eks-introduces-eks-pod-identity/
https://aws.amazon.com/cn/blogs/aws/amazon-eks-pod-identity-simplifies-iam-permissions-for-applications-on-amazon-eks-clusters/

先决条件

集群版本需要符合要求，如果在addon上使用，则addon版本也要符合要求，版本要求参考
eks节点的托管策略AmazonEKSWorkerNodePolicy中新增了权限eks-auth:AssumeRoleForPodIdentity授权PIA进行该API的调用。因此可以通过标签限制可以assume的角色范围，相关的条件键参考。

默认凭证链中Assume role with web identity是在Container credential provider之前的，因此如果使用默认凭证链，逻辑上按照优先级要先关闭IRSA功能，但是实际测试Container的优先级更高

2024-05-09 04:40:59,277 - MainThread - botocore.hooks - DEBUG - Event calling-command.sts.get-caller-identity: calling handler <bound method GenerateCliSkeletonArgument.generate_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x7f874ea2f690>>
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: custom-process
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: config-file
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: ec2-credentials-file
2024-05-09 04:40:59,278 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: boto-config
2024-05-09 04:40:59,279 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: container-role
2024-05-09 04:40:59,280 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): 169.254.170.23:80
2024-05-09 04:40:59,281 - MainThread - urllib3.connectionpool - DEBUG - http://169.254.170.23:80 "GET /v1/credentials HTTP/1.1" 200 905

PIA实际上是在源有的凭证链上增加的新环节

配置PIA后，eks会为pod注入AWS_CONTAINER_CREDENTIALS_FULL_URI和AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE环境变量。默认token的有效期为24小时。

    env:
    - name: AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE
      value: "/var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token"
    - name: AWS_CONTAINER_CREDENTIALS_FULL_URI
      value: "http://169.254.170.23/v1/credentials"
    volumeMounts:
    - mountPath: "/var/run/secrets/pods.eks.amazonaws.com/serviceaccount/"
      name: eks-pod-identity-token

PIA会通过AssumeRoleForPodIdentity为pod获取临时凭证。AssumeRoleForPodIdentityapi仅仅由PIA使用，需要使用最新版本的sdk。相比IRSA调用的api由AssumeRoleWithWebIdentity变为AssumeRoleForPodIdentity
部分aws sdk版本是不支持PIA的（例如SDK for Java 1.x和SDK for JavaScript 2.x），建议使用高于版本列表的sdk。boto3的版本要求1.34，但是实际测试1.33也有效
目前私有集群不支持，因为中国区无法创建eks-auth终端节点

PIA日志如下，需要查看pod所在节点对应的PIA日志

// 初始化PIA
{"bind-addr":"localhost:2703","level":"info","msg":"Starting server...","time":"2024-05-09T03:50:17Z"}   
{"bind-addr":"[fd00:ec2::23]:80","level":"info","msg":"Starting server...","time":"2024-05-09T03:50:17Z"} 
{"bind-addr":"169.254.170.23:80","level":"info","msg":"Starting server...","time":"2024-05-09T03:50:17Z"} 
// 申请临时凭证
// 192.168.22.38地址是使用PIA申请凭证的pod IP地址
// pod中请求的地址实际上就是PIA
{"client-addr":"192.168.22.38:35160","cluster-name":"test127","level":"info","msg":"handling new request request from 192.168.22.38:35160","time":"2024-05-09T06:02:47Z"}                                                                 
{"client-addr":"192.168.22.38:35160","cluster-name":"test127","level":"info","msg":"Calling EKS Auth to fetch credentials","time":"2024-05-09T06:02:47Z"}                                                                               {"client-addr":"192.168.22.38:35160","cluster-name":"test127","fetched_role_arn":"arn:aws-cn:sts::xxxxxxxxxxx:assumed-role/myPIArole/eks-test127-test-irsa--2b5522a3-04be-40cf-8ab2-3120d55a1308","fetched_role_id":"AROAxxxIEQW2:eks-test127-test-irsa--2b5522a3-04be-40cf-8ab2-3120d55a1308","level":"info","msg":"Successfully fetched credentials from EKS Auth","request_time_ms":248,"time":"2024-05-09T06:02:47Z"}               
{"client-addr":"192.168.22.38:35160","cluster-name":"test127","level":"info","msg":"Storing creds in cache","refreshTtl":10800000000000,"time":"2024-05-09T06:02:47Z"}

在cloudtrail中查看调用

{
    "eventVersion": "1.09",
    "userIdentity": {
        "type": "AssumedRole",
        "arn": "arn:aws-cn:sts::xxxxxxxxxxx:assumed-role/myEKSNodeRole/i-0xxxa8ebbbd",
        }
    },
    "eventSource": "eks-auth.amazonaws.com",
    "eventName": "AssumeRoleForPodIdentity",
    "userAgent": "aws-sdk-go-v2/1.21.2 os/linux lang/go#1.20.12 md/GOOS#linux md/GOARCH#amd64 api/eksauth#1.0.0-zeta.e49712bf27d5",
    "requestParameters": {
        "clusterName": "test127",
        "token": "HIDDEN_DUE_TO_SECURITY_REASONS"
    }
}

重启pod不会导致临时凭证再次申请

Each set of temporary credentials are assumed by the EKS Auth service in EKS Pod Identity, instead of each Amazon SDK that you run in each pod. Then, the Amazon EKS Pod Identity Agent that runs on each node issues the credentials to the SDKs. Thus the load is reduced to once for each node and isn't duplicated in each pod.

实际上请求是eks node发送的，我们可以手动模拟这一过程

尽管文档提到这个api专供PIA使用，但是实际测试可以手动调用，过期时间6小时

$ aws eks-auth assume-role-for-pod-identity --cluster-name test127 --token eyJhbGciOiJSUzxxxxxxxxxLUbg
{
    "subject": {
        "namespace": "default",
        "serviceAccount": "test-irsa"
    },
    "audience": "pods.eks.amazonaws.com",
    "podIdentityAssociation": {
        "associationArn": "arn:aws-cn:eks:cn-north-1:xxxxxxxxxxx:podidentityassociation/test127/a-ljfzpyuh61pknjtiq",
        "associationId": "a-ljfzpyuh61pknjtiq"
    },
    "assumedRoleUser": {
        "arn": "arn:aws-cn:sts::xxxxxxxxxxx:assumed-role/myPIArole/eks-test127-test-irsa--b95c1a6e-74b2-4da5-97e0-4b105da0638f",
        "assumeRoleId": "AROAQxxxx4IEQW2:eks-test127-test-irsa--b95c1a6e-74b2-4da5-97e0-4b105da0638f"
    },
    "credentials": {
        "sessionToken": "FwoDYXdztjf4=",
        "secretAccessKey": "wNqAp8xxxxxxxxxxxyoa+zpWJ",
        "accessKeyId": "ASIAQRIBWRJKFSMXKL5U",
        "expiration": "2024-05-09T12:23:23+00:00"
    }
}