94 、k8s之rbac

一、rbac----安全机制

赋权机制

集群是按照用户名进行登录，按照项目名称进行命名空间的分类。

配电云主站------62天 8个人高温补贴

一主2从

user pdyzz

pdyzz

-n pdyzz

资源空间

pod数量

1.1、k8s的安全机制：

apiserver------>集群内和外部的通信都需要apiserver进行调度，所有的安全机制都是围绕apiserver展开的。

1.2、apiserver进行通信：

1、认证：Authentication

2、鉴权：Authorzation你在集群当中的权限的控制

3、准入控制admission control 你能做的哪些

1.3、认证：

1、token

token：Http token ，是一个很长的特殊编码方式的而且难以被模仿的特殊字符串，来表达客户端的一种方式。

每一个token都会对应一个用户，存储在apiserver能够访问的文件中。客户端发起对apiserver的请求时，在http header当中必须加入token。

2、http base认证

http base认证：用户名加密码进行认证

3、https证书认证

https证书认证：基于ca证书签名的客户端身份认证方式。最严格得到方式。

http token和http base都是服务端对客户端的单向认证，https是双向认证的方式。

1.4、认证的资源类型：

kubectl kubelet kube-proxy

kubectl对pod进行管理也需要认证

service Account：是为了方便访问pod中的容器，以及容器访问apiserver专门创建的。

service Acount---->每创建一个pod就会自动创建。

1、token：和apiserver认证的私钥

2、ca.crt：认证apiserver的证书

3、namespace：service Account的命名空间

[root@master01 opt]# kubectl exec -it nginx1-bd76c7b4-jp445 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@nginx1-bd76c7b4-jp445:/# cd /var/run/secrets/kubernetes.io/serviceaccount/
root@nginx1-bd76c7b4-jp445:/var/run/secrets/kubernetes.io/serviceaccount# ls
ca.crt	namespace  token  ##每个pod都有

1.5、鉴权：

认证过后，就到了鉴权。

确定请求方有哪些资源的权限。

1.20版本，鉴权统一使用RBAC进行。

1、角色：

Role：指定命名空间的资源控制权限

ClusterRole：可以指定所有的命名空间的资源控制权限

2、角色绑定：

Rolebinding：将角色绑定到主体用户-------subject

clusterRolebinding：将集群角色绑定到主体

3、主体

user：用户

service account 服务账号（集群的服务账号）

用户组 group

准入机制：

[root@master01 rabc]# kubectl explain role
KIND:     Role
VERSION:  rbac.authorization.k8s.io/v1


[root@master01 opt]# mkdir rabc
[root@master01 opt]# cd rabc/
[root@master01 rabc]# vim test1.yaml


apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test1
#创建的角色名称
  namespace: default
#这个是必须要有的字段，只能有一个命名空间
rules:
#定义规则
- apiGroups: [""]
#rbac.authorization.k8s.io/v1,默认就是对apiserver的请求权限
  resources: ["pod","services"]  
#给主体也就是用户可以在指定的命名空间内对哪些资源对象进行操作。
  verbs: ["get","watch","list"]


权限
-------------------------------
#rules.verbs有："get", "list", "watch", "create", "update", "patch", "delete", "exec"
#rules.resources有："services", "endpoints", "pods", "secrets", "configmaps", "crontabs", "deployments", "jobs", "nodes", "rolebindings", "clusterroles", "daemonsets", "replicasets", "statefulsets", "horizontalpodautoscalers", "replicationcontrollers", "cronjobs"
-----------------------------------


----------------只是类型，格式不对---------------------
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: test1
#创建的角色名称
  namespace: default
#这个是必须要有的字段，只能有一个命名空间
subject:
- kind: User
        Group
        ServerAccount
rules:
#定义规则
- apiGroup: [""]
#rbac.authorization.k8s.io/v1,默认就是对apiserver的请求权限
  resources: ["pod","services"]  
#给主体也就是用户可以在指定的命名空间内对哪些资源对象进行操作。
  verbs: ["get","watch","list"]
--------------------------------------

在这里插入图片描述

[root@master01 rabc]# kubectl apply -f test1.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test1
#创建的角色名称
  namespace: default
#这个是必须要有的字段，只能有一个命名空间
rules:
#定义规则
- apiGroups: [""]
#rbac.authorization.k8s.io/v1,默认就是对apiserver的请求权限
  resources: ["pods","services"]
#给主体也就是用户可以在指定的命名空间内对哪些资源对象进行操作。
  verbs: ["get","watch","list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: test2
#创建的角色名称
  namespace: default
#这个是必须要有的字段，只能有一个命名空间
subjects:
- kind: User
  name: xy102
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: test1
  apiGroup: rbac.authorization.k8s.io



[root@master01 rabc]# useradd lucky
[root@master01 rabc]# passwd lucky 
更改用户 lucky 的密码 。
新的 密码：
无效的密码： 密码少于 8 个字符
重新输入新的 密码：
passwd：所有的身份验证令牌已经成功更新。
[root@master01 rabc]# su - lucky
上一次登录：五 9月 13 10:15:33 CST 2024pts/2 上
[lucky@master01 ~]$ kubectl get pod
error: the server doesn't have a resource type "pod"
[lucky@master01 ~]$ kubectl get pods



[root@master01 rabc]# cd /usr/local/bin/
[root@master01 bin]# ls
helm
[root@master01 bin]# rz -E
rz waiting to receive.
[root@master01 bin]# rz -E
rz waiting to receive.
[root@master01 bin]# rz -E
rz waiting to receive.
[root@master01 bin]# ls
cfssl  cfssl-certinfo  cfssljson  helm
#apiserver和用户之间连接的认证证书

[root@master01 bin]# chmod +x /usr/local/bin/cfssl*
[root@master01 bin]# cd /opt/rabc/

#客户端的签名证书

cat > lucky-csr.json <<EOF
{
  "CN": "lucky",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
	  "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
	  "OU": "System"
    }
  ]
}
EOF


[root@master01 rabc]# cd /etc/kubernetes/pki/
[root@master01 pki]# ls
apiserver.crt                 ca.crt              front-proxy-ca.srl
apiserver-etcd-client.crt     ca.key              front-proxy-client.crt
apiserver-etcd-client.key     ca.srl              front-proxy-client.key
apiserver.key                 etcd                sa.key
apiserver-kubelet-client.crt  front-proxy-ca.crt  sa.pub
apiserver-kubelet-client.key  front-proxy-ca.key

--------------错误--------------------------
[root@master01 pki]# cfssl gencert -ca=ca.crt -ca-key=ca.key -profile=kubernetes /opt/rbac/lucky-csr.json | cfssljson -bare lucky 
open /opt/rbac/lucky-csr.json: no such file or directory
Failed to parse input: unexpected end of JSON input
------------------------------------

[root@master01 pki]# cd /opt/rabc/
[root@master01 rabc]# ls
lucky-csr.json  test1.yaml
[root@master01 rabc]# vim lucky-csr.json 
[root@master01 rabc]# vim lucky-csr.json 

{
  "CN": "lucky",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

[root@master01 pki]# cfssl gencert -ca=ca.crt -ca-key=ca.key -profile=kubernetes /opt/rabc/lucky-csr.json | cfssljson -bare lucky 
2024/09/13 11:27:03 [INFO] generate received request
2024/09/13 11:27:03 [INFO] received CSR
2024/09/13 11:27:03 [INFO] generating key: rsa-2048
2024/09/13 11:27:03 [INFO] encoded CSR
2024/09/13 11:27:03 [INFO] signed certificate with serial number 270213834320680597090887969272870496291196286104
2024/09/13 11:27:03 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

[root@master01 pki]# ls
apiserver.crt                 ca.key                  front-proxy-client.key
apiserver-etcd-client.crt     ca.srl                  lucky.csr
apiserver-etcd-client.key     etcd                    lucky-key.pem
apiserver.key                 front-proxy-ca.crt      lucky.pem
apiserver-kubelet-client.crt  front-proxy-ca.key      sa.key
apiserver-kubelet-client.key  front-proxy-ca.srl      sa.pub
ca.crt                        front-proxy-client.crt


[root@master01 pki]# cd -
/opt/rabc
[root@master01 rabc]# vim rabc-config.sh

APISERVER=$1
# 设置集群参数
export KUBE_APISERVER="https://$APISERVER:6443"
kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/kubernetes/pki/ca.crt \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=lucky.kubeconfig

# 设置客户端认证参数
kubectl config set-credentials lucky \
  --client-key=/etc/kubernetes/pki/lucky-key.pem \
  --client-certificate=/etc/kubernetes/pki/lucky.pem \
  --embed-certs=true \
  --kubeconfig=lucky.kubeconfig

# 设置上下文参数
kubectl config set-context kubernetes \
  --cluster=kubernetes \
  --user=lucky \
  --namespace=lucky-cloud \
  --kubeconfig=lucky.kubeconfig


[root@master01 rabc]# chmod 777 rabc-config.sh 
[root@master01 rabc]# ls
lucky-csr.json  rabc-config.sh  test1.yaml

[root@master01 rabc]# ./rabc-config.sh 192.168.168.81
Cluster "kubernetes" set.
User "lucky" set.
Context "kubernetes" created.

[root@master01 rabc]# cd /opt/rabc/
[root@master01 rabc]# ls
lucky-csr.json  lucky.kubeconfig  rabc-config.sh  test1.yaml

[root@master01 rabc]# kubectl config use-context kubernetes --kubeconfig=lucky.kubeconfig
Switched to context "kubernetes".
[root@master01 rabc]# mkdir /home/lucky/.kube
mkdir: 无法创建目录"/home/lucky/.kube": 文件已存在
[root@master01 rabc]# ls
lucky-csr.json  lucky.kubeconfig  rabc-config.sh  test1.yaml
[root@master01 rabc]# cp lucky.kubeconfig /home/lucky/.kube/config
[root@master01 rabc]# chown -R lucky:lucky  /home/lucky/.kube/

##更改用户
[root@master01 rabc]# vim test1.yaml 

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test1
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
rules:
#定义规则
- apiGroups: [""]
#rbac.authorization.k8s.io/v1,默认就是对apiserver的请求权限
  resources: ["pods","services"]
#给主体也就是用户可以在指定的命名空间内对哪些资源对象进行操作。
  verbs: ["get","watch","list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: test2
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
subjects:
- kind: User
  name: lucky
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: test1
  apiGroup: rbac.authorization.k8s.io

[root@master01 rabc]# kubectl create ns lucky-cloud
namespace/lucky-cloud created
[root@master01 rabc]# kubectl apply -f test1.yaml 
role.rbac.authorization.k8s.io/test1 created
rolebinding.rbac.authorization.k8s.io/test2 created
[root@master01 rabc]# kubectl get role,rolebinding -n lucky-cloud 
NAME                                   CREATED AT
role.rbac.authorization.k8s.io/test1   2024-09-13T03:41:51Z

NAME                                          ROLE         AGE
rolebinding.rbac.authorization.k8s.io/test2   Role/test1   42s

[root@master01 rabc]# su - lucky
上一次登录：五 9月 13 11:43:13 CST 2024pts/1 上
[lucky@master01 ~]$ exit
登出
[root@master01 rabc]# kubectl create deployment nginx1 --image=nginx:1.22 --replicas=3 -n lucky-cloud 
deployment.apps/nginx1 created


[root@master01 rabc]# kubectl get pod -n lucky-cloud 
NAME                     READY   STATUS    RESTARTS   AGE
nginx1-654cb56c4-4q4g7   1/1     Running   0          74s
nginx1-654cb56c4-9qpdg   1/1     Running   0          74s
nginx1-654cb56c4-hzx8x   1/1     Running   0          74s

[root@master01 rabc]# su - lucky 
上一次登录：五 9月 13 11:46:49 CST 2024pts/1 上
[lucky@master01 ~]$ kubectl get pod
NAME                     READY   STATUS    RESTARTS   AGE
nginx1-654cb56c4-4q4g7   1/1     Running   0          114s
nginx1-654cb56c4-9qpdg   1/1     Running   0          114s
nginx1-654cb56c4-hzx8x   1/1     Running   0          114s

[lucky@master01 ~]$ kubectl exec -it nginx1-654cb56c4-4q4g7 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Error from server (Forbidden): pods "nginx1-654cb56c4-4q4g7" is forbidden: User "lucky" cannot create resource "pods/exec" in API group "" in the namespace "lucky-cloud"



##添加exec权限
[root@master01 rabc]# vim test1.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test1
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
rules:
#定义规则
- apiGroups: [""]
#rbac.authorization.k8s.io/v1,默认就是对apiserver的请求权限
  resources: ["pods","services"]
#给主体也就是用户可以在指定的命名空间内对哪些资源对象进行操作。
  verbs: ["get","watch","list","exec"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: test2
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
subjects:
- kind: User
  name: lucky
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: test1
  apiGroup: rbac.authorization.k8s.io

[root@master01 rabc]# kubectl apply -f test1.yaml 
role.rbac.authorization.k8s.io/test1 configured
rolebinding.rbac.authorization.k8s.io/test2 unchanged

[lucky@master01 ~]$ kubectl exec -it nginx1-654cb56c4-4q4g7 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
Error from server (Forbidden): pods "nginx1-654cb56c4-4q4g7" is forbidden: User "lucky" cannot create resource "pods/exec" in API group "" in the namespace "lucky-cloud"

##更改
[root@master01 rabc]# vim test1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test1
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
rules:
#定义规则
- apiGroups: [""]
#rbac.authorization.k8s.io/v1,默认就是对apiserver的请求权限
  resources: ["pods","services","deployments","pods/exec","pods/log"]
#给主体也就是用户可以在指定的命名空间内对哪些资源对象进行操作。
  verbs: ["get","watch","list","exec"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: test2
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
subjects:
- kind: User
  name: lucky
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: test1
  apiGroup: rbac.authorization.k8s.io


[root@master01 rabc]# kubectl apply -f test1.yaml
role.rbac.authorization.k8s.io/test1 configured
rolebinding.rbac.authorization.k8s.io/test2 unchanged

[root@master01 rabc]# su - lucky 
上一次登录：五 9月 13 13:01:36 CST 2024pts/0 上
[lucky@master01 ~]$ kubectl logs -f nginx1-654cb56c4-4q4g7
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/09/13 03:47:32 [notice] 1#1: using the "epoll" event method
2024/09/13 03:47:32 [notice] 1#1: nginx/1.22.1
2024/09/13 03:47:32 [notice] 1#1: built by gcc 10.2.1 20210110 (Debian 10.2.1-6) 
2024/09/13 03:47:32 [notice] 1#1: OS: Linux 3.10.0-957.el7.x86_64
2024/09/13 03:47:32 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 65536:65536
2024/09/13 03:47:32 [notice] 1#1: start worker processes
2024/09/13 03:47:32 [notice] 1#1: start worker process 29
2024/09/13 03:47:32 [notice] 1#1: start worker process 30
2024/09/13 03:47:32 [notice] 1#1: start worker process 31
2024/09/13 03:47:32 [notice] 1#1: start worker process 32





[root@master01 rabc]# vim test1.yaml 

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test1
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
rules:
#定义规则
- apiGroups: [""]
#rbac.authorization.k8s.io/v1,默认就是对apiserver的请求权限
  resources: ["pods","services","deployments","pods/exec","pods/log"]
#给主体也就是用户可以在指定的命名空间内对哪些资源对象进行操作。
  verbs: ["get","watch","list","exec","create"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: test2
#创建的角色名称
  namespace: lucky-cloud
#这个是必须要有的字段，只能有一个命名空间
subjects:
- kind: User
  name: lucky
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: test1
  apiGroup: rbac.authorization.k8s.io



[root@master01 rabc]# kubectl apply -f test1.yaml 
role.rbac.authorization.k8s.io/test1 configured
rolebinding.rbac.authorization.k8s.io/test2 unchanged
[root@master01 rabc]# su - lucky 
上一次登录：五 9月 13 13:03:22 CST 2024pts/0 上
[lucky@master01 ~]$ kubectl exec -it nginx1-654cb56c4-4q4g7 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@nginx1-654cb56c4-4q4g7:/# 



#rules.verbs有："get", "list", "watch", "create", "update", "patch", "delete", "exec"
#rules.resources有："services", "endpoints", "pods", "secrets", "configmaps", "crontabs", "deployments", "jobs", "nodes", "rolebindings", "clusterroles", "daemonsets", "replicasets", "statefulsets", "horizontalpodautoscalers", "replicationcontrollers", "cronjobs"

面试1

1、describe pod 查询容器的 container ID
2、到pod部署的节点，根据containerID获取容器在节点的进程号
3、根据进程号进入容器的网络命名空间

4、tcpdump -i 网卡

[root@master01 opt]# kubectl create deployment test1 --image=nginx:1.22 --replicas=1
deployment.apps/test1 created

[root@master01 opt]# kubectl get pod -o wide
NAME                    READY   STATUS    RESTARTS   AGE     IP             NODE     NOMINATED NODE   READINESS GATES
hpa-test2-0             1/1     Running   9          25h     10.244.2.20    node02   <none>           <none>
nfs1-76f66b958-68wpl    1/1     Running   0          7d4h    10.244.2.173   node02   <none>           <none>
nginx1-bd76c7b4-jp445   1/1     Running   0          21h     10.244.1.11    node01   <none>           <none>
test1-86776958-swshz    1/1     Running   0          2m57s   10.244.1.15    node01   <none>           <none>

[root@master01 opt]# kubectl describe pod test1-86776958-swshz 
Name:         test1-86776958-swshz
Namespace:    default
Priority:     0
Node:         node01/192.168.168.82
Start Time:   Fri, 13 Sep 2024 13:57:52 +0800
Labels:       app=test1
              pod-template-hash=86776958
Annotations:  <none>
Status:       Running
IP:           10.244.1.15
IPs:
  IP:           10.244.1.15
Controlled By:  ReplicaSet/test1-86776958
Containers:
  nginx:
    Container ID:   docker://c4f09342136d40a3134c3d76cec8678d0514e9e78320fcaac707a5bee29ad5fd
    Image:          nginx:1.22
    Image ID:       docker-pullable://192.168.168.71/test1/nginx@sha256:9081064712674ffcff7b7bdf874c75bcb8e5fb933b65527026090dacda36ea8b
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Fri, 13 Sep 2024 13:57:54 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-4s6fz (ro)
Conditions:


##查看进程号
[root@node01 ~]# docker inspect --format '{{.State.Pid}}' c4f09342136d40a3134c3d76cec8678d0514e9e78320fcaac707a5bee29ad5fd
97062


##进入这个容器的网络命名空间
[root@node01 ~]# nsenter -n -t 97062
[root@node01 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if276: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default 
    link/ether 82:ac:76:b0:45:30 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.244.1.15/24 brd 10.244.1.255 scope global eth0
       valid_lft forever preferred_lft forever

[root@node01 ~]# tcpdump -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes


[root@master01 opt]# curl 10.244.1.15


##查看抓包情况

在这里插入图片描述