CKA证书模拟24道题-题解

快捷别名

alias k=kubectl                         # will already be pre-configured
export do="--dry-run=client -o yaml"    # k create deploy nginx --image=nginx $do
export now="--force --grace-period 0"   # k delete pod x $now

vim设置

set tabstop=2
set expandtab
set shiftwidth=2

题库1

任务权重：1%

您可以通过上下文从主终端访问多个集群。将所有这些上下文名称写入 。kubectl/opt/course/1/contexts

接下来编写一个命令以将当前上下文显示到 ，该命令应使用 。/opt/course/1/context_default_kubectl.shkubectl

最后在 中编写执行相同操作的第二个命令，但不使用 ./opt/course/1/context_default_no_kubectl.shkubectl

You have access to multiple clusters from your main terminal through kubectl contexts. Write all those context names into /opt/course/1/contexts.

Next write a command to display the current context into /opt/course/1/context_default_kubectl.sh, the command should use kubectl.

Finally write a second command doing the same thing into /opt/course/1/context_default_no_kubectl.sh, but without the use of kubectl.

题解

k config get-contexts 
k config get-contexts -o name > /opt/course/1/contexts

或者
Or using jsonpath:
k config view -o yaml # overview
k config view -o jsonpath="{.contexts[*].name}"
k config view -o jsonpath="{.contexts[*].name}" | tr " " "\n" # new lines
k config view -o jsonpath="{.contexts[*].name}" | tr " " "\n" > /opt/course/1/contexts 

 #/opt/course/1/contexts
k8s-c1-H
k8s-c2-AC
k8s-c3-CCC

接下来创建第一个命令

# /opt/course/1/context_default_kubectl.sh
kubectl config current-context
➜ sh /opt/course/1/context_default_kubectl.sh
k8s-c1-H

第二个

# /opt/course/1/context_default_no_kubectl.sh
cat ~/.kube/config | grep current
➜ sh /opt/course/1/context_default_no_kubectl.sh
current-context: k8s-c1-H

第二个命令也可以改进为

# /opt/course/1/context_default_no_kubectl.sh
cat ~/.kube/config | grep current | sed -e "s/current-context: //"

题库2

任务权重：3%

使用上下文：kubectl config use-context k8s-c1-H

在命名空间中创建单个映像 Pod。应该命名 Pod，容器应该命名。此 Pod 应该只调度在控制平面节点上，不要在任何节点上添加新标签。httpd:2.4.41-alpinedefaultpod1pod1-container

Use context: kubectl config use-context k8s-c1-H

Create a single Pod of image httpd:2.4.41-alpine in Namespace default. The Pod should be named pod1 and the container should be named pod1-container. This Pod should only be scheduled on a controlplane node, do not add new labels any nodes.

题解

答：
首先，我们找到控制平面节点及其污点：

k get node # find controlplane node
k describe node cluster1-controlplane1 | grep Taint -A1 # get controlplane node taints
k get node cluster1-controlplane1 --show-labels # get controlplane node labels

接下来我们创建 Pod 模板：
# check the export on the very top of this document so we can use $do
k run pod1 --image=httpd:2.4.41-alpine $do > 2.yaml    #追加为yaml文件
vim 2.yaml

手动执行必要的更改。使用 Kubernetes 文档并搜索例如容忍和 nodeSelector 来查找示例：

# 2.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
spec:
  containers:
  - image: httpd:2.4.41-alpine
    name: pod1-container                       # change
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  tolerations:                                 # add
  - effect: NoSchedule                         # add
    key: node-role.kubernetes.io/control-plane # add
  nodeSelector:                                # add
    node-role.kubernetes.io/control-plane: ""  # add
status: {}

这里重要的是添加在控制平面节点上运行的容差，以及 nodeSelector，以确保它只在控制平面节点上运行。如果我们只指定容忍度，则可以 在控制平面或工作节点上调度 Pod。

现在我们创建它：

k -f 2.yaml create

让我们检查一下 Pod 是否已调度：

➜ k get pod pod1 -o wide
NAME   READY   STATUS    RESTARTS   ...    NODE                     NOMINATED NODE
pod1   1/1     Running   0          ...    cluster1-controlplane1   <none> 

题库3

任务权重：1%

使用上下文：kubectl config use-context k8s-c1-H

命名空间中有两个命名的 Pod。C13 管理层要求您将 Pod 缩减到一个副本以节省资源。o3db-*project-c13

Use context: kubectl config use-context k8s-c1-H

There are two Pods named in Namespace . C13 management asked you to scale the Pods down to one replica to save resources.o3db-*project-c13

题解

答：
如果我们检查 Pod，我们会看到两个副本：


➜ k -n project-c13 get pod | grep o3db
o3db-0                                  1/1     Running   0          52s
o3db-1                                  1/1     Running   0          42s


从它们的名字来看，这些似乎由StatefulSet管理。但是，如果我们不确定，我们还可以检查管理 Pod 的最常见资源：

➜ k -n project-c13 get deploy,ds,sts | grep o3db
statefulset.apps/o3db   2/2     2m56s
确认，我们必须使用StatefulSet。要找出这一点，我们还可以查看 Pod 标签：

➜ k -n project-c13 get pod --show-labels | grep o3db
o3db-0                                  1/1     Running   0          3m29s   app=nginx,controller-revision-hash=o3db-5fbd4bb9cc,statefulset.kubernetes.io/pod-name=o3db-0
o3db-1                                  1/1     Running   0          3m19s   app=nginx,controller-revision-hash=o3db-5fbd4bb9cc,statefulset.kubernetes.io/pod-name=o3db-1
为了完成任务，我们只需运行：

➜ k -n project-c13 scale sts o3db --replicas 1
statefulset.apps/o3db scaled

➜ k -n project-c13 get sts o3db
NAME   READY   AGE
o3db   1/1     4m39s

题库4

任务权重：4%

使用上下文：kubectl config use-context k8s-c1-H

在命名空间中执行以下操作。创建一个以 image 命名的 Pod 。配置一个只执行命令的活体探针。还要配置一个 准备探测 检查 url 是否可访问，您可以为此使用。启动 Pod 并确认由于准备情况探测而未准备就绪。defaultready-if-service-readynginx:1.16.1-alpinetruehttp://service-am-i-ready:80wget -T2 -O- http://service-am-i-ready:80

创建第二个以带有标签的映像命名的 Pod。现有的服务现在应该将第二个 Pod 作为端点。am-i-readynginx:1.16.1-alpineid: cross-server-readyservice-am-i-ready

现在第一个 Pod 应该处于就绪状态，确认这一点。

Use context: kubectl config use-context k8s-c1-H

 

Do the following in Namespace . Create a single Pod named of image . Configure a LivenessProbe which simply executes command . Also configure a ReadinessProbe which does check if the url is reachable, you can use for this. Start the Pod and confirm it isn't ready because of the ReadinessProbe.defaultready-if-service-readynginx:1.16.1-alpinetruehttp://service-am-i-ready:80wget -T2 -O- http://service-am-i-ready:80

Create a second Pod named of image with label . The already existing Service should now have that second Pod as endpoint.am-i-readynginx:1.16.1-alpineid: cross-server-readyservice-am-i-ready

Now the first Pod should be in ready state, confirm that.

题解

答：
对于一个 Pod 使用探针检查另一个 Pod 是否准备就绪有点反模式，因此通常可用的对 绝对远程 url 不起作用。尽管如此，此任务中请求的解决方法仍应显示探测器和 Pod<-> 服务通信的工作原理。readinessProbe.httpGet

首先，我们创建第一个 Pod：


k run ready-if-service-ready --image=nginx:1.16.1-alpine $do > 4_pod1.yaml

vim 4_pod1.yaml

# 4_pod1.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ready-if-service-ready
  name: ready-if-service-ready
spec:
  containers:
  - image: nginx:1.16.1-alpine
    name: ready-if-service-ready
    resources: {}
    livenessProbe:                                      # add from here
      exec:
        command:
        - 'true'
    readinessProbe:
      exec:
        command:
        - sh
        - -c
        - 'wget -T2 -O- http://service-am-i-ready:80'   # to here
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

然后创建 Pod

k -f 4_pod1.yaml create

并确认它处于未就绪状态：

➜ k get pod ready-if-service-ready
NAME                     READY   STATUS    RESTARTS   AGE
ready-if-service-ready   0/1     Running   0          7s

我们还可以使用描述来检查其原因：

➜ k describe pod ready-if-service-ready
 ...
  Warning  Unhealthy  18s   kubelet, cluster1-node1  Readiness probe failed: Connecting to service-am-i-ready:80 (10.109.194.234:80)
wget: download timed out

现在我们创建第二个 Pod：

k run am-i-ready --image=nginx:1.16.1-alpine --labels="id=cross-server-ready"

现有的服务 现在应该有一个终结点：service-am-i-ready

k describe svc service-am-i-ready
k get ep # also possible

这将导致我们的第一个 Pod 准备就绪，只需给它一分钟时间让就绪探测再次检查：

➜ k get pod ready-if-service-ready
NAME                     READY   STATUS    RESTARTS   AGE
ready-if-service-ready   1/1     Running   0          53s

看看这些 Pod 一起工作！

题库5

任务权重：1%

使用上下文：kubectl config use-context k8s-c1-H

所有命名空间中都有各种 Pod。编写一个命令，其中列出了按 AGE （） 排序的所有 Pod。/opt/course/5/find_pods.shmetadata.creationTimestamp

编写第二个命令，其中列出了按字段排序的所有 Pod。对这两个命令都使用排序。/opt/course/5/find_pods_uid.shmetadata.uidkubectl

Use context: kubectl config use-context k8s-c1-H

 

There are various Pods in all namespaces. Write a command into which lists all Pods sorted by their AGE ()./opt/course/5/find_pods.shmetadata.creationTimestamp

Write a second command into which lists all Pods sorted by field . Use sorting for both commands./opt/course/5/find_pods_uid.shmetadata.uidkubectl

题解

答：

这里的一个很好的资源（以及许多其他事情）是kubectl-cheat-sheet。在 Kubernetes 文档中搜索“备忘单”时，您可以快速访问它。

# /opt/course/5/find_pods.sh
kubectl get pod -A --sort-by=.metadata.creationTimestamp

并执行：

➜ sh /opt/course/5/find_pods.sh
NAMESPACE         NAME                                             ...          AGE
kube-system       kube-scheduler-cluster1-controlplane1            ...          63m
kube-system       etcd-cluster1-controlplane1                      ...          63m
kube-system       kube-apiserver-cluster1-controlplane1            ...          63m
kube-system       kube-controller-manager-cluster1-controlplane1   ...          63m
...

对于第二个命令：

# /opt/course/5/find_pods_uid.sh
kubectl get pod -A --sort-by=.metadata.uid

并执行：

➜ sh /opt/course/5/find_pods_uid.sh
NAMESPACE         NAME                                      ...          AGE
kube-system       coredns-5644d7b6d9-vwm7g                  ...          68m
project-c13       c13-3cc-runner-heavy-5486d76dd4-ddvlt     ...          63m
project-hamster   web-hamster-shop-849966f479-278vp         ...          63m
project-c13       c13-3cc-web-646b6c8756-qsg4b              ...          63m

题库6

任务权重：8%

使用上下文：kubectl config use-context k8s-c1-H

创建一个名为 的新持久卷。它应该具有2Gi的容量，访问模式读写一次，hostPath和未定义存储类名。safari-pv/Volumes/Data

接下来，在命名空间中创建一个名为 的新 PersistentVolumeClaim。它应该请求 2Gi 存储，访问模式 ReadWriteOnce，并且不应该定义 storageClassName。PVC 应正确绑定到 PV。project-tigersafari-pvc

最后，在命名空间中创建一个新的部署，该部署将该卷装载在 。该部署的 Pod 应为映像 。safariproject-tiger/tmp/safari-datahttpd:2.4.41-alpine

Use context: kubectl config use-context k8s-c1-H

 

Create a new PersistentVolume named . It should have a capacity of 2Gi, accessMode ReadWriteOnce, hostPath and no storageClassName defined.safari-pv/Volumes/Data

Next create a new PersistentVolumeClaim in Namespace named . It should request 2Gi storage, accessMode ReadWriteOnce and should not define a storageClassName. The PVC should bound to the PV correctly.project-tigersafari-pvc

Finally create a new Deployment in Namespace which mounts that volume at . The Pods of that Deployment should be of image .safariproject-tiger/tmp/safari-datahttpd:2.4.41-alpine

题解

答

vim 6_pv.yaml

从 https://kubernetes.io/docs 中找到一个例子 并对其进行更改：

# 6_pv.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
 name: safari-pv
spec:
 capacity:
  storage: 2Gi
 accessModes:
  - ReadWriteOnce
 hostPath:
  path: "/Volumes/Data"

然后创建它：

k -f 6_pv.yaml create

接下来是 PersistentVolumeClaim：

vim 6_pvc.yaml

从 https://kubernetes.io/docs 中找到一个例子 并对其进行更改：

# 6_pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: safari-pvc
  namespace: project-tiger
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
     storage: 2Gi

然后创建：

k -f 6_pvc.yaml create

并检查两者的状态是否为“绑定”：

➜ k -n project-tiger get pv,pvc
NAME                         CAPACITY  ... STATUS   CLAIM                    ...
persistentvolume/safari-pv   2Gi       ... Bound    project-tiger/safari-pvc ...

NAME                               STATUS   VOLUME      CAPACITY ...
persistentvolumeclaim/safari-pvc   Bound    safari-pv   2Gi      ...

接下来，我们创建一个部署并挂载该卷：

k -n project-tiger create deploy safari \
  --image=httpd:2.4.41-alpine $do > 6_dep.yaml

vim 6_dep.yaml

更改 yaml 以装载卷：

# 6_dep.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: safari
  name: safari
  namespace: project-tiger
spec:
  replicas: 1
  selector:
    matchLabels:
      app: safari
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: safari
    spec:
      volumes:                                      # add
      - name: data                                  # add
        persistentVolumeClaim:                      # add
          claimName: safari-pvc                     # add
      containers:
      - image: httpd:2.4.41-alpine
        name: container
        volumeMounts:                               # add
        - name: data                                # add
          mountPath: /tmp/safari-data               # add
k -f 6_dep.yaml create

我们可以确认它是否正确安装：

➜ k -n project-tiger describe pod safari-5cbf46d6d-mjhsb  | grep -A2 Mounts:   
    Mounts:
      /tmp/safari-data from data (rw) # there it is
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-n2sjj (ro)

题库7

Use context: kubectl config use-context k8s-c1-H

 

The metrics-server has been installed in the cluster. Your college would like to know the kubectl commands to:

show Nodes resource usage
show Pods and their containers resource usage
Please write the commands into and ./opt/course/7/node.sh/opt/course/7/pod.sh

任务权重：1%

使用上下文：kubectl config use-context k8s-c1-H

指标服务器已安装在群集中。您的大学想知道 kubectl 命令：

显示节点资源使用情况
显示 Pod 及其容器的资源使用情况
请将命令写入 和 。/opt/course/7/node.sh/opt/course/7/pod.sh

题解

答：

我们在这里需要使用的命令是 top：

➜ k top -h
Display Resource (CPU/Memory/Storage) usage.

 The top command allows you to see the resource consumption for nodes or pods.

 This command requires Metrics Server to be correctly configured and working on the server.

Available Commands:
  node        Display Resource (CPU/Memory/Storage) usage of nodes
  pod         Display Resource (CPU/Memory/Storage) usage of pods

我们看到指标服务器提供有关资源使用情况的信息：

➜ k top node
NAME               CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%   
cluster1-controlplane1   178m         8%     1091Mi          57%       
cluster1-node1   66m          6%     834Mi           44%       
cluster1-node2   91m          9%     791Mi           41% 

我们创建第一个文件：

# /opt/course/7/node.sh
kubectl top node

对于第二个文件，我们可能需要再次检查文档：

➜ k top pod -h
Display Resource (CPU/Memory/Storage) usage of pods.
...
Namespace in current context is ignored even if specified with --namespace.
      --containers=false: If present, print usage of containers within a pod.
      --no-headers=false: If present, print output without headers.
...

有了这个，我们可以完成这个任务：

# /opt/course/7/pod.sh
kubectl top pod --containers=true

题库8

Use context: kubectl config use-context k8s-c1-H

 

Ssh into the controlplane node with . Check how the controlplane components kubelet, kube-apiserver, kube-scheduler, kube-controller-manager and etcd are started/installed on the controlplane node. Also find out the name of the DNS application and how it's started/installed on the controlplane node.ssh cluster1-controlplane1

Write your findings into file . The file should be structured like:/opt/course/8/controlplane-components.txt

任务权重：2%

使用上下文：kubectl config use-context k8s-c1-H

使用 Ssh 进入控制平面节点。检查控制平面组件 kubelet、kube-apiserver、kube-scheduler、kube-controller-manager 和 etcd 是如何在控制平面节点上启动/安装的。还要了解 DNS 应用程序的名称以及它在控制平面节点上的启动/安装方式。ssh cluster1-controlplane1

将您的发现写入文件 。该文件的结构应如下所示：/opt/course/8/controlplane-components.txt

# /opt/course/8/controlplane-components.txt
kubelet: [TYPE]
kube-apiserver: [TYPE]
kube-scheduler: [TYPE]
kube-controller-manager: [TYPE]
etcd: [TYPE]
dns: [TYPE] [NAME]
选项包括：、、、[TYPE]not-installedprocessstatic-podpod

题解

答：

我们可以从查找所请求组件的进程开始，尤其是一开始的 kubelet：

➜ ssh cluster1-controlplane1

root@cluster1-controlplane1:~# ps aux | grep kubelet # shows kubelet process

我们可以看到哪些组件是通过 systemd 控制的，查看 目录：/etc/systemd/system

➜ root@cluster1-controlplane1:~# find /etc/systemd/system/ | grep kube
/etc/systemd/system/kubelet.service.d
/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
/etc/systemd/system/multi-user.target.wants/kubelet.service

➜ root@cluster1-controlplane1:~# find /etc/systemd/system/ | grep etcd

这表明 kubelet 是通过 systemd 控制的，但没有其他名为 kube 和 etcd 的服务。似乎这个集群是使用 kubeadm 设置的，所以我们检查默认的清单目录：

➜ root@cluster1-controlplane1:~# find /etc/kubernetes/manifests/
/etc/kubernetes/manifests/
/etc/kubernetes/manifests/kube-controller-manager.yaml
/etc/kubernetes/manifests/etcd.yaml
/etc/kubernetes/manifests/kube-apiserver.yaml
/etc/kubernetes/manifests/kube-scheduler.yaml

（kubelet 也可以有一个不同的清单目录，通过 它的 systemd 启动配置中的参数指定）--pod-manifest-path

这意味着主要的 4 个控制平面服务被设置为静态 Pod。实际上，让我们检查在 控制平面节点上的命名空间中 运行的所有 Pod：kube-system

➜ root@cluster1-controlplane1:~# kubectl -n kube-system get pod -o wide | grep controlplane1
coredns-5644d7b6d9-c4f68                            1/1     Running            ...   cluster1-controlplane1
coredns-5644d7b6d9-t84sc                            1/1     Running            ...   cluster1-controlplane1
etcd-cluster1-controlplane1                         1/1     Running            ...   cluster1-controlplane1
kube-apiserver-cluster1-controlplane1               1/1     Running            ...   cluster1-controlplane1
kube-controller-manager-cluster1-controlplane1      1/1     Running            ...   cluster1-controlplane1
kube-proxy-q955p                                    1/1     Running            ...   cluster1-controlplane1
kube-scheduler-cluster1-controlplane1               1/1     Running            ...   cluster1-controlplane1
weave-net-mwj47                                     2/2     Running            ...   cluster1-controlplane1

在那里，我们看到 5 个静态 pod，带有 as 后缀。-cluster1-controlplane1

我们还看到 dns 应用程序似乎是 coredns，但它是如何控制的？

➜ root@cluster1-controlplane1$ kubectl -n kube-system get ds
NAME         DESIRED   CURRENT   ...   NODE SELECTOR            AGE
kube-proxy   3         3         ...   kubernetes.io/os=linux   155m
weave-net    3         3         ...   <none>                   155m

➜ root@cluster1-controlplane1$ kubectl -n kube-system get deploy
NAME      READY   UP-TO-DATE   AVAILABLE   AGE
coredns   2/2     2            2           155m

似乎 coredns 是通过部署控制的。我们将我们的发现合并到请求的文件中：

# /opt/course/8/controlplane-components.txt
kubelet: process
kube-apiserver: static-pod
kube-scheduler: static-pod
kube-controller-manager: static-pod
etcd: static-pod
dns: pod coredns

您应该能够轻松调查正在运行的集群，了解如何设置集群及其服务的不同方法，并能够进行故障排除和查找错误源。

题库9

Use context: kubectl config use-context k8s-c2-AC

 

Ssh into the controlplane node with ssh cluster2-controlplane1. Temporarily stop the kube-scheduler, this means in a way that you can start it again afterwards.

Create a single Pod named manual-schedule of image httpd:2.4-alpine, confirm it's created but not scheduled on any node.

Now you're the scheduler and have all its power, manually schedule that Pod on node cluster2-controlplane1. Make sure it's running.

Start the kube-scheduler again and confirm it's running correctly by creating a second Pod named manual-schedule2 of image httpd:2.4-alpine and check if it's running on cluster2-node1.

任务权重：5%

使用上下文：kubectl config use-context k8s-c2-AC

使用 Ssh 进入控制平面节点。暂时停止 kube-scheduler，这意味着您可以在之后再次启动它。ssh cluster2-controlplane1

创建一个以 image 命名的 Pod，确认它已创建但未在任何节点上调度。manual-schedulehttpd:2.4-alpine

现在你是调度程序并拥有它的所有功能，在节点集群 2-controlplane1 上手动调度该 Pod。确保它正在运行。

再次启动 kube-scheduler，并通过创建第二个名为 image 的 Pod 来确认它是否正常运行，并检查它是否在 cluster2-node1 上运行。manual-schedule2httpd:2.4-alpine

题解

答：

停止调度程序

首先，我们找到控制平面节点：

➜ k get node
NAME                     STATUS   ROLES           AGE   VERSION
cluster2-controlplane1   Ready    control-plane   26h   v1.26.0
cluster2-node1           Ready    <none>          26h   v1.26.0

然后我们连接并检查调度程序是否正在运行：

➜ ssh cluster2-controlplane1

➜ root@cluster2-controlplane1:~# kubectl -n kube-system get pod | grep schedule
kube-scheduler-cluster2-controlplane1            1/1     Running   0          6s

终止调度程序（暂时）：

➜ root@cluster2-controlplane1:~# cd /etc/kubernetes/manifests/

➜ root@cluster2-controlplane1:~# mv kube-scheduler.yaml ..

并且应该停止：

➜ root@cluster2-controlplane1:~# kubectl -n kube-system get pod | grep schedule

➜ root@cluster2-controlplane1:~# 

创建容器

现在我们创建 Pod：

k run manual-schedule --image=httpd:2.4-alpine

并确认它没有分配节点：

➜ k get pod manual-schedule -o wide
NAME              READY   STATUS    ...   NODE     NOMINATED NODE
manual-schedule   0/1     Pending   ...   <none>   <none>        

手动调度 Pod

现在让我们玩调度程序：

k get pod manual-schedule -o yaml > 9.yaml
# 9.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2020-09-04T15:51:02Z"
  labels:
    run: manual-schedule
  managedFields:
...
    manager: kubectl-run
    operation: Update
    time: "2020-09-04T15:51:02Z"
  name: manual-schedule
  namespace: default
  resourceVersion: "3515"
  selfLink: /api/v1/namespaces/default/pods/manual-schedule
  uid: 8e9d2532-4779-4e63-b5af-feb82c74a935
spec:
  nodeName: cluster2-controlplane1        # add the controlplane node name
  containers:
  - image: httpd:2.4-alpine
    imagePullPolicy: IfNotPresent
    name: manual-schedule
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-nxnc7
      readOnly: true
  dnsPolicy: ClusterFirst
...

调度器唯一要做的就是为 Pod 声明设置 nodeName 。它如何找到正确的节点进行调度，这是一个非常复杂的问题，需要考虑许多变量。

由于我们不能 或 ，在这种情况下，我们需要删除并创建或替换：kubectl apply``kubectl edit

k -f 9.yaml replace --force

它看起来如何？

➜ k get pod manual-schedule -o wide
NAME              READY   STATUS    ...   NODE            
manual-schedule   1/1     Running   ...   cluster2-controlplane1

看起来我们的 Pod 现在正在按照要求在控制平面上运行，尽管没有指定容忍度。只有调度程序在查找正确的节点名称时会考虑 tains/toleranceations/affinity。这就是为什么仍然可以手动将 Pod 直接分配给控制平面节点并跳过调度程序的原因。

再次启动调度程序

➜ ssh cluster2-controlplane1

➜ root@cluster2-controlplane1:~# cd /etc/kubernetes/manifests/

➜ root@cluster2-controlplane1:~# mv ../kube-scheduler.yaml .

检查它是否正在运行：

➜ root@cluster2-controlplane1:~# kubectl -n kube-system get pod | grep schedule
kube-scheduler-cluster2-controlplane1            1/1     Running   0          16s

安排第二个测试 Pod：

k run manual-schedule2 --image=httpd:2.4-alpine
➜ k get pod -o wide | grep schedule
manual-schedule    1/1     Running   ...   cluster2-controlplane1
manual-schedule2   1/1     Running   ...   cluster2-node1

恢复正常。

题库10

Use context: kubectl config use-context k8s-c1-H

 

Create a new ServiceAccount processor in Namespace project-hamster. Create a Role and RoleBinding, both named processor as well. These should allow the new SA to only create Secrets and ConfigMaps in that Namespace.

任务权重：6%

使用上下文：kubectl config use-context k8s-c1-H

在命名空间中创建新的服务帐户。创建角色和角色绑定，两者都已命名。这些应该允许新的 SA 仅在该命名空间中创建机密和配置映射。processorproject-hamsterprocessor

题解

答：

让我们谈谈 RBAC 资源

集群角色|角色定义一组权限，以及该权限可用的地方，在整个群集中或仅单个命名空间中。

集群角色绑定|RoleBinding 将一组权限与帐户连接起来，并定义在整个群集或单个命名空间中的应用位置。

因此，有 4 种不同的 RBAC 组合和 3 种有效的组合：

1. 角色 + 角色绑定（在单个命名空间中可用，在单个命名空间中应用）)
2. ClusterRole + ClusterRoleBinding（在群集范围内可用，在群集范围内应用）
3. 群集角色 + 角色绑定（在群集范围内可用，在单个命名空间中应用）)
4. 角色 + 群集角色绑定（**不可能：**在单个命名空间中可用，在群集范围内应用）

跳转至解决方案

我们首先创建服务帐户：

➜ k -n project-hamster create sa processor
serviceaccount/processor created

然后对于角色：

k -n project-hamster create role -h # examples

所以我们执行：

k -n project-hamster create role processor \
  --verb=create \
  --resource=secret \
  --resource=configmap

这将创建一个角色，例如：

# kubectl -n project-hamster create role processor --verb=create --resource=secret --resource=configmap
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: processor
  namespace: project-hamster
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  - configmaps
  verbs:
  - create

现在，我们将角色绑定到服务帐户：

k -n project-hamster create rolebinding -h # examples

所以我们创建它：

k -n project-hamster create rolebinding processor \
  --role processor \
  --serviceaccount project-hamster:processor

这将创建一个角色绑定，如下所示：

# kubectl -n project-hamster create rolebinding processor --role processor --serviceaccount project-hamster:processor
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: processor
  namespace: project-hamster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: processor
subjects:
- kind: ServiceAccount
  name: processor
  namespace: project-hamster

要测试我们的 RBAC 设置，我们可以使用：kubectl auth can-i

k auth can-i -h # examples

喜欢这个：

➜ k -n project-hamster auth can-i create secret \
  --as system:serviceaccount:project-hamster:processor
yes

➜ k -n project-hamster auth can-i create configmap \
  --as system:serviceaccount:project-hamster:processor
yes

➜ k -n project-hamster auth can-i create pod \
  --as system:serviceaccount:project-hamster:processor
no

➜ k -n project-hamster auth can-i delete secret \
  --as system:serviceaccount:project-hamster:processor
no

➜ k -n project-hamster auth can-i get configmap \
  --as system:serviceaccount:project-hamster:processor
no

做。

题库11

Use context: kubectl config use-context k8s-c1-H

 

Use Namespace project-tiger for the following. Create a DaemonSet named ds-important with image httpd:2.4-alpine and labels id=ds-important and uuid=18426a0b-5f59-4e10-923f-c0e078e82462. The Pods it creates should request 10 millicore cpu and 10 mebibyte memory. The Pods of that DaemonSet should run on all nodes, also controlplanes.

任务权重：4%

使用上下文：kubectl config use-context k8s-c1-H

将命名空间用于以下内容。创建一个以图像和标签命名的守护程序集，以及 .它创建的 Pod 应该请求 10 毫核 CPU 和 10 兆字节内存。该守护程序集的 Pod 应该在所有节点上运行，也应该在控制平面上运行。project-tigerds-importanthttpd:2.4-alpineid=ds-importantuuid=18426a0b-5f59-4e10-923f-c0e078e82462

题解

答：

到目前为止，我们无法 直接使用 创建守护程序集，因此我们创建一个部署并对其进行更改：kubectl

k -n project-tiger create deployment --image=httpd:2.4-alpine ds-important $do > 11.yaml

vim 11.yaml

（当然，你也可以在 Kubernetes 文档中搜索 DaemonSet 示例 yaml 并对其进行更改。

然后我们将 yaml 调整为：

# 11.yaml
apiVersion: apps/v1
kind: DaemonSet                                     # change from Deployment to Daemonset
metadata:
  creationTimestamp: null
  labels:                                           # add
    id: ds-important                                # add
    uuid: 18426a0b-5f59-4e10-923f-c0e078e82462      # add
  name: ds-important
  namespace: project-tiger                          # important
spec:
  #replicas: 1                                      # remove
  selector:
    matchLabels:
      id: ds-important                              # add
      uuid: 18426a0b-5f59-4e10-923f-c0e078e82462    # add
  #strategy: {}                                     # remove
  template:
    metadata:
      creationTimestamp: null
      labels:
        id: ds-important                            # add
        uuid: 18426a0b-5f59-4e10-923f-c0e078e82462  # add
    spec:
      containers:
      - image: httpd:2.4-alpine
        name: ds-important
        resources:
          requests:                                 # add
            cpu: 10m                                # add
            memory: 10Mi                            # add
      tolerations:                                  # add
      - effect: NoSchedule                          # add
        key: node-role.kubernetes.io/control-plane  # add
#status: {}                                         # remove

请求守护程序集在所有节点上运行，因此我们需要为此指定容忍度。

让我们确认一下：

k -f 11.yaml create
➜ k -n project-tiger get ds
NAME           DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
ds-important   3         3         3       3            3           <none>          8s
➜ k -n project-tiger get pod -l id=ds-important -o wide
NAME                      READY   STATUS          NODE
ds-important-6pvgm        1/1     Running   ...   cluster1-node1
ds-important-lh5ts        1/1     Running   ...   cluster1-controlplane1
ds-important-qhjcq        1/1     Running   ...   cluster1-node2

题库12

Use context: kubectl config use-context k8s-c1-H

 

Use Namespace project-tiger for the following. Create a Deployment named deploy-important with label id=very-important (the Pods should also have this label) and 3 replicas. It should contain two containers, the first named container1 with image nginx:1.17.6-alpine and the second one named container2 with image kubernetes/pause.

There should be only ever one Pod of that Deployment running on one worker node. We have two worker nodes: cluster1-node1 and cluster1-node2. Because the Deployment has three replicas the result should be that on both nodes one Pod is running. The third Pod won't be scheduled, unless a new worker node will be added.

In a way we kind of simulate the behaviour of a DaemonSet here, but using a Deployment and a fixed number of replicas.

任务权重：6%

使用上下文：kubectl config use-context k8s-c1-H

将命名空间用于以下内容。创建一个以标签命名的部署（也应具有此标签）和 3 个副本。它应该包含两个容器，第一个名为 container1 带有图像，第二个名为 container2 带有图像。project-tigerdeploy-importantid=very-importantPodsnginx:1.17.6-alpinekubernetes/pause

在一个工作节点上应该只有一个该部署的 Pod 运行。我们有两个工作节点：cluster1-node1 和cluster1-node2。因为部署有三个副本，所以结果应该是在两个节点上都有一个 Pod 正在运行。第三个 Pod 不会被调度，除非添加新的工作节点。

在某种程度上，我们在这里模拟了守护程序集的行为，但使用部署和固定数量的副本。

题解

答：

有两种可能的方法，一种使用，一种 使用 .podAntiAffinity``topologySpreadConstraint

豆荚反亲和力

这里的想法是，我们创建了一个“Pod 间反亲和力”，它允许我们说一个 Pod 应该只调度在一个节点上，其中另一个特定标签的 Pod （这里是相同的标签）尚未运行。

让我们从创建部署模板开始：

k -n project-tiger create deployment \
  --image=nginx:1.17.6-alpine deploy-important $do > 12.yaml

vim 12.yaml

然后将 yaml 更改为：

# 12.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    id: very-important                  # change
  name: deploy-important
  namespace: project-tiger              # important
spec:
  replicas: 3                           # change
  selector:
    matchLabels:
      id: very-important                # change
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        id: very-important              # change
    spec:
      containers:
      - image: nginx:1.17.6-alpine
        name: container1                # change
        resources: {}
      - image: kubernetes/pause         # add
        name: container2                # add
      affinity:                                             # add
        podAntiAffinity:                                    # add
          requiredDuringSchedulingIgnoredDuringExecution:   # add
          - labelSelector:                                  # add
              matchExpressions:                             # add
              - key: id                                     # add
                operator: In                                # add
                values:                                     # add
                - very-important                            # add
            topologyKey: kubernetes.io/hostname             # add
status: {}

指定一个拓扑键，这是一个预填充的 Kubernetes 标签，你可以通过描述一个节点来找到它。

拓扑扩展约束

我们可以用. 最好尝试并同时玩两者。topologySpreadConstraints

# 12.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    id: very-important                  # change
  name: deploy-important
  namespace: project-tiger              # important
spec:
  replicas: 3                           # change
  selector:
    matchLabels:
      id: very-important                # change
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        id: very-important              # change
    spec:
      containers:
      - image: nginx:1.17.6-alpine
        name: container1                # change
        resources: {}
      - image: kubernetes/pause         # add
        name: container2                # add
      topologySpreadConstraints:                 # add
      - maxSkew: 1                               # add
        topologyKey: kubernetes.io/hostname      # add
        whenUnsatisfiable: DoNotSchedule         # add
        labelSelector:                           # add
          matchLabels:                           # add
            id: very-important                   # add
status: {}

应用并运行

让我们运行它：

k -f 12.yaml create

然后我们检查部署状态，其中显示 2/3 就绪计数：

➜ k -n project-tiger get deploy -l id=very-important
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
deploy-important   2/3     3            2           2m35s

运行以下命令，我们看到每个工作节点上都有一个 Pod，一个未调度。

➜ k -n project-tiger get pod -o wide -l id=very-important
NAME                                READY   STATUS    ...   NODE             
deploy-important-58db9db6fc-9ljpw   2/2     Running   ...   cluster1-node1
deploy-important-58db9db6fc-lnxdb   0/2     Pending   ...   <none>          
deploy-important-58db9db6fc-p2rz8   2/2     Running   ...   cluster1-node2

如果我们 kubectl 描述 Pod，它将向我们展示不调度的原因是我们实现的 podAntiAffinity 规则：deploy-important-58db9db6fc-lnxdb

Warning  FailedScheduling  63s (x3 over 65s)  default-scheduler  0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/control-plane: }, that the pod didn't tolerate, 2 node(s) didn't match pod affinity/anti-affinity, 2 node(s) didn't satisfy existing pods anti-affinity rules.

或者我们的拓扑扩展约束：

Warning  FailedScheduling  16s   default-scheduler  0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/control-plane: }, that the pod didn't tolerate, 2 node(s) didn't match pod topology spread constraints.

题库13

Use context: kubectl config use-context k8s-c1-H

 

Create a Pod named multi-container-playground in Namespace default with three containers, named c1, c2 and c3. There should be a volume attached to that Pod and mounted into every container, but the volume shouldn't be persisted or shared with other Pods.

Container c1 should be of image nginx:1.17.6-alpine and have the name of the node where its Pod is running available as environment variable MY_NODE_NAME.

Container c2 should be of image busybox:1.31.1 and write the output of the date command every second in the shared volume into file date.log. You can use while true; do date >> /your/vol/path/date.log; sleep 1; done for this.

Container c3 should be of image busybox:1.31.1 and constantly send the content of file date.log from the shared volume to stdout. You can use tail -f /your/vol/path/date.log for this.

Check the logs of container c3 to confirm correct setup.

任务权重：4%

使用上下文：kubectl config use-context k8s-c1-H

创建一个在命名空间中命名的 Pod，其中包含三个容器，分别名为 和 。应该有一个卷附加到该 Pod 并挂载到每个容器中，但该卷不应持久化或与其他 Pod 共享。multi-container-playgrounddefaultc1c2c3

容器应该是镜像的，并且具有运行其 Pod 的节点的名称，可用作环境变量MY_NODE_NAME。c1nginx:1.17.6-alpine

容器应该是映像的，并且每秒将共享卷中命令的输出写入文件。你可以为此使用。c2busybox:1.31.1datedate.logwhile true; do date >> /your/vol/path/date.log; sleep 1; done

容器应该是图像的，并不断将文件的内容从共享卷发送到标准输出。你可以为此使用。c3busybox:1.31.1date.logtail -f /your/vol/path/date.log

检查容器的日志以确认设置正确。c3

题解

答：

首先，我们创建 Pod 模板：

k run multi-container-playground --image=nginx:1.17.6-alpine $do > 13.yaml

vim 13.yaml

并添加其他容器和它们应执行的命令：

# 13.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: multi-container-playground
  name: multi-container-playground
spec:
  containers:
  - image: nginx:1.17.6-alpine
    name: c1                                                                      # change
    resources: {}
    env:                                                                          # add
    - name: MY_NODE_NAME                                                          # add
      valueFrom:                                                                  # add
        fieldRef:                                                                 # add
          fieldPath: spec.nodeName                                                # add
    volumeMounts:                                                                 # add
    - name: vol                                                                   # add
      mountPath: /vol                                                             # add
  - image: busybox:1.31.1                                                         # add
    name: c2                                                                      # add
    command: ["sh", "-c", "while true; do date >> /vol/date.log; sleep 1; done"]  # add
    volumeMounts:                                                                 # add
    - name: vol                                                                   # add
      mountPath: /vol                                                             # add
  - image: busybox:1.31.1                                                         # add
    name: c3                                                                      # add
    command: ["sh", "-c", "tail -f /vol/date.log"]                                # add
    volumeMounts:                                                                 # add
    - name: vol                                                                   # add
      mountPath: /vol                                                             # add
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  volumes:                                                                        # add
    - name: vol                                                                   # add
      emptyDir: {}                                                                # add
status: {}
k -f 13.yaml create

哦，天哪，很多要求的东西。我们检查 Pod 是否一切正常：

➜ k get pod multi-container-playground
NAME                         READY   STATUS    RESTARTS   AGE
multi-container-playground   3/3     Running   0          95s

很好，然后我们检查容器 c1 是否将请求的节点名称作为 env 变量：

➜ k exec multi-container-playground -c c1 -- env | grep MY
MY_NODE_NAME=cluster1-node2

最后我们检查日志记录：

➜ k logs multi-container-playground -c c3
Sat Dec  7 16:05:10 UTC 2077
Sat Dec  7 16:05:11 UTC 2077
Sat Dec  7 16:05:12 UTC 2077
Sat Dec  7 16:05:13 UTC 2077
Sat Dec  7 16:05:14 UTC 2077
Sat Dec  7 16:05:15 UTC 2077
Sat Dec  7 16:05:16 UTC 2077

题库14

Use context: kubectl config use-context k8s-c1-H

 

You're ask to find out following information about the cluster k8s-c1-H:

How many controlplane nodes are available?
How many worker nodes are available?
What is the Service CIDR?
Which Networking (or CNI Plugin) is configured and where is its config file?
Which suffix will static pods have that run on cluster1-node1?
Write your answers into file /opt/course/14/cluster-info, structured like this:

任务权重：2%

使用上下文：kubectl config use-context k8s-c1-H

系统会要求您了解有关群集的以下信息：k8s-c1-H

有多少个控制平面节点可用？
有多少个工作器节点可用？
什么是服务网段？
配置了哪个网络（或CNI插件），其配置文件在哪里？
静态 Pod 将具有哪个后缀在 cluster1-node1 上运行？
将您的答案写入文件，结构如下：/opt/course/14/cluster-info

# /opt/course/14/cluster-info
1: [ANSWER]
2: [ANSWER]
3: [ANSWER]
4: [ANSWER]
5: [ANSWER]

题解

答：

有多少个控制平面和工作器节点可用？

➜ k get node
NAME                    STATUS   ROLES          AGE   VERSION
cluster1-controlplane1  Ready    control-plane  27h   v1.26.0
cluster1-node1          Ready    <none>         27h   v1.26.0
cluster1-node2          Ready    <none>         27h   v1.26.0

我们看到一个控制平面和两个工作线程。

什么是服务网段？

➜ ssh cluster1-controlplane1

➜ root@cluster1-controlplane1:~# cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep range
    - --service-cluster-ip-range=10.96.0.0/12

配置了哪个网络（或CNI插件），其配置文件在哪里？

➜ root@cluster1-controlplane1:~# find /etc/cni/net.d/
/etc/cni/net.d/
/etc/cni/net.d/10-weave.conflist

➜ root@cluster1-controlplane1:~# cat /etc/cni/net.d/10-weave.conflist
{
    "cniVersion": "0.3.0",
    "name": "weave",
...

默认情况下，kubelet 会查看 以发现 CNI 插件。这在每个控制平面和工作器节点上都是相同的。/etc/cni/net.d

静态 Pod 将具有哪个后缀在 cluster1-node1 上运行？

后缀是带有前导连字符的节点主机名。它曾经在 早期的 Kubernetes 版本中。-static

结果

结果 可能如下所示：/opt/course/14/cluster-info

# /opt/course/14/cluster-info

# How many controlplane nodes are available?
1: 1

# How many worker nodes are available?
2: 2

# What is the Service CIDR?
3: 10.96.0.0/12

# Which Networking (or CNI Plugin) is configured and where is its config file?
4: Weave, /etc/cni/net.d/10-weave.conflist

# Which suffix will static pods have that run on cluster1-node1?
5: -cluster1-node1

题库15

Use context: kubectl config use-context k8s-c2-AC

 

Write a command into /opt/course/15/cluster_events.sh which shows the latest events in the whole cluster, ordered by time (metadata.creationTimestamp). Use kubectl for it.

Now kill the kube-proxy Pod running on node cluster2-node1 and write the events this caused into /opt/course/15/pod_kill.log.

Finally kill the containerd container of the kube-proxy Pod on node cluster2-node1 and write the events into /opt/course/15/container_kill.log.

Do you notice differences in the events both actions caused?

任务权重：3%

使用上下文：kubectl config use-context k8s-c2-AC

编写一个命令，在其中显示整个群集中的最新事件，按时间排序 （）。用于它。/opt/course/15/cluster_events.shmetadata.creationTimestampkubectl

现在杀死在节点集群 2-node1 上运行的 kube-proxy Pod，并将由此引起的事件写入 ./opt/course/15/pod_kill.log

最后在节点集群 2-node1 上杀死 kube-proxy Pod 的容器，并将事件写入 ./opt/course/15/container_kill.log

您是否注意到两种行为引起的事件存在差异？

题解

答：

# /opt/course/15/cluster_events.sh
kubectl get events -A --sort-by=.metadata.creationTimestamp

现在我们杀死 kube-proxy Pod：

k -n kube-system get pod -o wide | grep proxy # find pod running on cluster2-node1

k -n kube-system delete pod kube-proxy-z64cg

现在检查事件：

sh /opt/course/15/cluster_events.sh

将杀戮造成的事件写成：/opt/course/15/pod_kill.log

# /opt/course/15/pod_kill.log
kube-system   9s          Normal    Killing           pod/kube-proxy-jsv7t   ...
kube-system   3s          Normal    SuccessfulCreate  daemonset/kube-proxy   ...
kube-system   <unknown>   Normal    Scheduled         pod/kube-proxy-m52sx   ...
default       2s          Normal    Starting          node/cluster2-node1  ...
kube-system   2s          Normal    Created           pod/kube-proxy-m52sx   ...
kube-system   2s          Normal    Pulled            pod/kube-proxy-m52sx   ...
kube-system   2s          Normal    Started           pod/kube-proxy-m52sx   ...

最后，我们将尝试通过杀死属于 kube-proxy Pod 容器的容器来引发事件：

➜ ssh cluster2-node1

➜ root@cluster2-node1:~# crictl ps | grep kube-proxy
1e020b43c4423   36c4ebbc9d979   About an hour ago   Running   kube-proxy     ...

➜ root@cluster2-node1:~# crictl rm 1e020b43c4423
1e020b43c4423

➜ root@cluster2-node1:~# crictl ps | grep kube-proxy
0ae4245707910   36c4ebbc9d979   17 seconds ago      Running   kube-proxy     ...     

我们杀死了主容器（1e020b43c4423），但也注意到直接创建了一个新容器（0ae4245707910）。谢谢 Kubernetes！

现在我们看看这是否再次导致事件，并将其写入第二个文件：

sh /opt/course/15/cluster_events.sh
# /opt/course/15/container_kill.log
kube-system   13s         Normal    Created      pod/kube-proxy-m52sx    ...
kube-system   13s         Normal    Pulled       pod/kube-proxy-m52sx    ...
kube-system   13s         Normal    Started      pod/kube-proxy-m52sx    ...

比较事件，我们看到当我们删除整个 Pod 时，有更多的事情要做，因此有更多的事件。例如，游戏中的守护程序集用于重新创建丢失的 Pod。当我们手动杀死 Pod 的主容器时，Pod 仍然存在，但只需要重新创建它的容器，因此事件更少。

题库16

Use context: kubectl config use-context k8s-c1-H

 

Write the names of all namespaced Kubernetes resources (like Pod, Secret, ConfigMap...) into /opt/course/16/resources.txt.

Find the project-* Namespace with the highest number of Roles defined in it and write its name and amount of Roles into /opt/course/16/crowded-namespace.txt.

任务权重：2%

使用上下文：kubectl config use-context k8s-c1-H

将所有命名空间的 Kubernetes 资源（如 Pod、Secret、ConfigMap...）的名称写入 ./opt/course/16/resources.txt

找到其中定义的命名空间数量最多，并将其名称和角色数量写入 。project-*Roles/opt/course/16/crowded-namespace.txt

题解

答：

命名空间和命名空间资源

现在我们可以得到所有资源的列表，例如：

k api-resources    # shows all

k api-resources -h # help always good

k api-resources --namespaced -o name > /opt/course/16/resources.txt

这将导致文件：

# /opt/course/16/resources.txt
bindings
configmaps
endpoints
events
limitranges
persistentvolumeclaims
pods
podtemplates
replicationcontrollers
resourcequotas
secrets
serviceaccounts
services
controllerrevisions.apps
daemonsets.apps
deployments.apps
replicasets.apps
statefulsets.apps
localsubjectaccessreviews.authorization.k8s.io
horizontalpodautoscalers.autoscaling
cronjobs.batch
jobs.batch
leases.coordination.k8s.io
events.events.k8s.io
ingresses.extensions
ingresses.networking.k8s.io
networkpolicies.networking.k8s.io
poddisruptionbudgets.policy
rolebindings.rbac.authorization.k8s.io
roles.rbac.authorization.k8s.io

具有大多数角色的命名空间

➜ k -n project-c13 get role --no-headers | wc -l
No resources found in project-c13 namespace.
0

➜ k -n project-c14 get role --no-headers | wc -l
300

➜ k -n project-hamster get role --no-headers | wc -l
No resources found in project-hamster namespace.
0

➜ k -n project-snake get role --no-headers | wc -l
No resources found in project-snake namespace.
0

➜ k -n project-tiger get role --no-headers | wc -l
No resources found in project-tiger namespace.
0

最后，我们将名称和金额写入文件：

# /opt/course/16/crowded-namespace.txt
project-c14 with 300 resources

题库17

Use context: kubectl config use-context k8s-c1-H

 

In Namespace project-tiger create a Pod named tigers-reunite of image httpd:2.4.41-alpine with labels pod=container and container=pod. Find out on which node the Pod is scheduled. Ssh into that node and find the containerd container belonging to that Pod.

Using command crictl:

Write the ID of the container and the info.runtimeType into /opt/course/17/pod-container.txt
Write the logs of the container into /opt/course/17/pod-container.log

任务权重：3%

使用上下文：kubectl config use-context k8s-c1-H

在命名空间中，创建一个以图像命名的 Pod，并带有标签和 .找出 Pod 调度在哪个节点上。Ssh 进入该节点并找到属于该 Pod 的容器。project-tigertigers-reunitehttpd:2.4.41-alpinepod=containercontainer=pod

使用命令 ：crictl

将容器的 ID 写入info.runtimeType/opt/course/17/pod-container.txt

将容器的日志写入/opt/course/17/pod-container.log

题解

答：

首先，我们创建 Pod：

k -n project-tiger run tigers-reunite \
  --image=httpd:2.4.41-alpine \
  --labels "pod=container,container=pod"

接下来，我们找出它被调度的节点：

k -n project-tiger get pod -o wide

# or fancy:
k -n project-tiger get pod tigers-reunite -o jsonpath="{.spec.nodeName}"

然后我们 ssh 到该节点并检查容器信息：

➜ ssh cluster1-node2

➜ root@cluster1-node2:~# crictl ps | grep tigers-reunite
b01edbe6f89ed    54b0995a63052    5 seconds ago    Running        tigers-reunite ...

➜ root@cluster1-node2:~# crictl inspect b01edbe6f89ed | grep runtimeType
    "runtimeType": "io.containerd.runc.v2",

然后我们填写请求的文件（在主终端上）：

# /opt/course/17/pod-container.txt
b01edbe6f89ed io.containerd.runc.v2

最后，我们将容器日志写入第二个文件中：

ssh cluster1-node2 'crictl logs b01edbe6f89ed' &> /opt/course/17/pod-container.log

上面的 in 命令重定向标准输出和标准错误。&>

您也可以简单地在 节点上运行并手动复制内容，如果不是很多。该文件应如下所示：crictl logs

# /opt/course/17/pod-container.log
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.44.0.37. Set the 'ServerName' directive globally to suppress this message
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.44.0.37. Set the 'ServerName' directive globally to suppress this message
[Mon Sep 13 13:32:18.555280 2021] [mpm_event:notice] [pid 1:tid 139929534545224] AH00489: Apache/2.4.41 (Unix) configured -- resuming normal operations
[Mon Sep 13 13:32:18.555610 2021] [core:notice] [pid 1:tid 139929534545224] AH00094: Command line: 'httpd -D FOREGROUND'

题库18

Use context: kubectl config use-context k8s-c3-CCC

 

There seems to be an issue with the kubelet not running on cluster3-node1. Fix it and confirm that cluster has node cluster3-node1 available in Ready state afterwards. You should be able to schedule a Pod on cluster3-node1 afterwards.

Write the reason of the issue into /opt/course/18/reason.txt.

任务权重：8%

使用上下文：kubectl config use-context k8s-c3-CCC

似乎有一个问题 kubelet 无法运行.修复此问题，并在之后确认群集的节点处于“就绪”状态。之后你应该能够安排一个 Pod。cluster3-node1cluster3-node1cluster3-node1

将问题的原因写到 中。/opt/course/18/reason.txt

题解

答：

此类任务的过程应该是检查 kubelet 是否正在运行，如果没有启动它，则检查其日志并纠正错误（如果有）。

检查其他集群是否已经定义并运行了一些组件总是有帮助的，因此您可以复制和使用现有的配置文件。尽管在这种情况下可能不需要。

检查节点状态：

➜ k get node
NAME                     STATUS     ROLES           AGE   VERSION
cluster3-controlplane1   Ready      control-plane   14d   v1.26.0
cluster3-node1           NotReady   <none>          14d   v1.26.0

首先，我们检查 kubelet 是否正在运行：

➜ ssh cluster3-node1

➜ root@cluster3-node1:~# ps aux | grep kubelet
root     29294  0.0  0.2  14856  1016 pts/0    S+   11:30   0:00 grep --color=auto kubelet

不，所以我们检查它是否使用 systemd 作为服务进行配置：

➜ root@cluster3-node1:~# service kubelet status
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: inactive (dead) since Sun 2019-12-08 11:30:06 UTC; 50min 52s ago
...

是的，它配置为 config at 的服务，但我们看到它处于非活动状态。让我们尝试启动它：/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

➜ root@cluster3-node1:~# service kubelet start

➜ root@cluster3-node1:~# service kubelet status
● kubelet.service - kubelet: The Kubernetes Node Agent
   Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/kubelet.service.d
           └─10-kubeadm.conf
   Active: activating (auto-restart) (Result: exit-code) since Thu 2020-04-30 22:03:10 UTC; 3s ago
     Docs: https://kubernetes.io/docs/home/
  Process: 5989 ExecStart=/usr/local/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=203/EXEC)
 Main PID: 5989 (code=exited, status=203/EXEC)

Apr 30 22:03:10 cluster3-node1 systemd[5989]: kubelet.service: Failed at step EXEC spawning /usr/local/bin/kubelet: No such file or directory
Apr 30 22:03:10 cluster3-node1 systemd[1]: kubelet.service: Main process exited, code=exited, status=203/EXEC
Apr 30 22:03:10 cluster3-node1 systemd[1]: kubelet.service: Failed with result 'exit-code'.

我们看到它正在尝试 使用其服务配置文件中定义的一些参数来执行。查找错误并获取更多日志的一个好方法是手动运行命令（通常也使用其参数）。/usr/local/bin/kubelet

➜ root@cluster3-node1:~# /usr/local/bin/kubelet
-bash: /usr/local/bin/kubelet: No such file or directory

➜ root@cluster3-node1:~# whereis kubelet
kubelet: /usr/bin/kubelet

另一种方法是查看服务的扩展日志记录，例如使用 .journalctl -u kubelet

好吧，我们有它，指定了错误的路径。更正文件中的路径并运行：/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

vim /etc/systemd/system/kubelet.service.d/10-kubeadm.conf # fix

systemctl daemon-reload && systemctl restart kubelet

systemctl status kubelet  # should now show running

此外，该节点应该可用于 api 服务器，不过要给它一点时间：

➜ k get node
NAME                     STATUS   ROLES           AGE   VERSION
cluster3-controlplane1   Ready    control-plane   14d   v1.26.0
cluster3-node1           Ready    <none>          14d   v1.26.0

最后，我们将原因写入文件：

# /opt/course/18/reason.txt
wrong path to kubelet binary specified in service config

题库19

NOTE: This task can only be solved if questions 18 or 20 have been successfully implemented and the k8s-c3-CCC cluster has a functioning worker node

 

Use context: kubectl config use-context k8s-c3-CCC

 

Do the following in a new Namespace secret. Create a Pod named secret-pod of image busybox:1.31.1 which should keep running for some time.

There is an existing Secret located at /opt/course/19/secret1.yaml, create it in the Namespace secret and mount it readonly into the Pod at /tmp/secret1.

Create a new Secret in Namespace secret called secret2 which should contain user=user1 and pass=1234. These entries should be available inside the Pod's container as environment variables APP_USER and APP_PASS.

Confirm everything is working.

任务权重：3%

注意：仅当问题 18 或 20 已成功实现并且 k8s-c3-CCC 集群具有正常运行的工作节点时，才能解决此任务

使用上下文：kubectl config use-context k8s-c3-CCC

在新的命名空间中执行以下操作。创建一个以 image 命名的 Pod，它应该会持续运行一段时间。secretsecret-podbusybox:1.31.1

有一个现有的密钥位于 ，在命名空间中创建它并将其只读地挂载到 Pod 中。/opt/course/19/secret1.yamlsecret/tmp/secret1

在名为 的命名空间中创建一个新的密钥，该密钥应包含 和 。这些条目应该在 Pod 的容器中作为环境变量APP_USER和APP_PASS提供。secretsecret2user=user1pass=1234

确认一切正常。

题解

答

首先，我们在其中创建命名空间和请求的机密：

k create ns secret

cp /opt/course/19/secret1.yaml 19_secret1.yaml

vim 19_secret1.yaml

我们需要调整 该密钥的命名空间：

# 19_secret1.yaml
apiVersion: v1
data:
  halt: IyEgL2Jpbi9zaAo...
kind: Secret
metadata:
  creationTimestamp: null
  name: secret1
  namespace: secret           # change
k -f 19_secret1.yaml create

接下来，我们创建第二个密钥：

k -n secret create secret generic secret2 --from-literal=user=user1 --from-literal=pass=1234

现在我们创建 Pod 模板：

k -n secret run secret-pod --image=busybox:1.31.1 $do -- sh -c "sleep 5d" > 19.yaml

vim 19.yaml

然后进行必要的更改：

# 19.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: secret-pod
  name: secret-pod
  namespace: secret                       # add
spec:
  containers:
  - args:
    - sh
    - -c
    - sleep 1d
    image: busybox:1.31.1
    name: secret-pod
    resources: {}
    env:                                  # add
    - name: APP_USER                      # add
      valueFrom:                          # add
        secretKeyRef:                     # add
          name: secret2                   # add
          key: user                       # add
    - name: APP_PASS                      # add
      valueFrom:                          # add
        secretKeyRef:                     # add
          name: secret2                   # add
          key: pass                       # add
    volumeMounts:                         # add
    - name: secret1                       # add
      mountPath: /tmp/secret1             # add
      readOnly: true                      # add
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  volumes:                                # add
  - name: secret1                         # add
    secret:                               # add
      secretName: secret1                 # add
status: {}

在当前的 K8s 版本中可能不需要指定 ， 因为它无论如何都是默认设置。readOnly: true

并执行：

k -f 19.yaml create

最后，我们检查一切是否正确：

➜ k -n secret exec secret-pod -- env | grep APP
APP_PASS=1234
APP_USER=user1
➜ k -n secret exec secret-pod -- find /tmp/secret1
/tmp/secret1
/tmp/secret1/..data
/tmp/secret1/halt
/tmp/secret1/..2019_12_08_12_15_39.463036797
/tmp/secret1/..2019_12_08_12_15_39.463036797/halt
➜ k -n secret exec secret-pod -- cat /tmp/secret1/halt
#! /bin/sh
### BEGIN INIT INFO
# Provides:          halt
# Required-Start:
# Required-Stop:
# Default-Start:
# Default-Stop:      0
# Short-Description: Execute the halt command.
# Description:
...

题库20

Use context: kubectl config use-context k8s-c3-CCC

 

Your coworker said node cluster3-node2 is running an older Kubernetes version and is not even part of the cluster. Update Kubernetes on that node to the exact version that's running on cluster3-controlplane1. Then add this node to the cluster. Use kubeadm for this.

任务权重：10%

使用上下文：kubectl config use-context k8s-c3-CCC

你的同事说节点运行的是较旧的 Kubernetes 版本，甚至不是集群的一部分。将该节点上的 Kubernetes 更新为在 上运行的确切版本。然后将此节点添加到群集。为此使用 kubeadm。cluster3-node2cluster3-controlplane1

题解

答：

将 Kubernetes 升级到 cluster3-controlplane1 版本

在文档中搜索 kubeadm 升级： https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade

➜ k get node
NAME                     STATUS   ROLES           AGE   VERSION
cluster3-controlplane1   Ready    control-plane   22h   v1.26.0
cluster3-node1           Ready    <none>          22h   v1.26.0

控制平面节点似乎正在运行 Kubernetes 1.26.0，并且还不是集群的一部分。cluster3-node2

➜ ssh cluster3-node2

➜ root@cluster3-node2:~# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.0", GitCommit:"b46a3f887ca979b1a5d14fd39cb1af43e7e5d12d", GitTreeState:"clean", BuildDate:"2022-12-08T19:57:06Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}

➜ root@cluster3-node2:~# kubectl version --short
Client Version: v1.25.5
Kustomize Version: v4.5.7

➜ root@cluster3-node2:~# kubelet --version
Kubernetes v1.25.5

这里 kubeadm 已经安装在想要的版本中，所以我们不需要安装它。因此，我们可以运行：

➜ root@cluster3-node2:~# kubeadm upgrade node
couldn't create a Kubernetes client from file "/etc/kubernetes/kubelet.conf": failed to load admin kubeconfig: open /etc/kubernetes/kubelet.conf: no such file or directory
To see the stack trace of this error execute with --v=5 or higher

这通常是升级节点的正确命令。但是这个错误意味着这个节点甚至从未初始化过，所以这里没有什么可更新的。稍后将使用 完成此操作。现在我们可以继续使用 kubelet 和 kubectl：kubeadm join

➜ root@cluster3-node2:~# apt update
...
Fetched 5,775 kB in 2s (2,313 kB/s)                               
Reading package lists... Done
Building dependency tree       
Reading state information... Done
90 packages can be upgraded. Run 'apt list --upgradable' to see them.

➜ root@cluster3-node2:~# apt show kubectl -a | grep 1.26
Version: 1.26.0-00

➜ root@cluster3-node2:~# apt install kubectl=1.26.0-00 kubelet=1.26.0-00
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be upgraded:
  kubectl kubelet
2 upgraded, 0 newly installed, 0 to remove and 135 not upgraded.
Need to get 30.5 MB of archives.
After this operation, 9,996 kB of additional disk space will be used.
Get:1 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubectl amd64 1.26.0-00 [10.1 MB]
Get:2 https://packages.cloud.google.com/apt kubernetes-xenial/main amd64 kubelet amd64 1.26.0-00 [20.5 MB]
Fetched 30.5 MB in 1s (29.7 MB/s)  
(Reading database ... 112508 files and directories currently installed.)
Preparing to unpack .../kubectl_1.26.0-00_amd64.deb ...
Unpacking kubectl (1.26.0-00) over (1.25.5-00) ...
Preparing to unpack .../kubelet_1.26.0-00_amd64.deb ...
Unpacking kubelet (1.26.0-00) over (1.25.5-00) ...
Setting up kubectl (1.26.0-00) ...
Setting up kubelet (1.26.0-00) ...

➜ root@cluster3-node2:~# kubelet --version
Kubernetes v1.26.0

现在我们了解了 kubeadm、kubectl 和 kubelet。重新启动 kubelet：

➜ root@cluster3-node2:~# service kubelet restart

➜ root@cluster3-node2:~# service kubelet status
● kubelet.service - kubelet: The Kubernetes Node Agent
     Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/kubelet.service.d
             └─10-kubeadm.conf
     Active: activating (auto-restart) (Result: exit-code) since Wed 2022-12-21 16:29:26 UTC; 5s ago
       Docs: https://kubernetes.io/docs/home/
    Process: 32111 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=1/FAILURE)
   Main PID: 32111 (code=exited, status=1/FAILURE)

Dec 21 16:29:26 cluster3-node2 systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
Dec 21 16:29:26 cluster3-node2 systemd[1]: kubelet.service: Failed with result 'exit-code'.

发生这些错误是因为我们仍然需要运行 才能将节点加入群集。让我们在下一步中执行此操作。kubeadm join

将群集 3-节点 2 添加到群集

首先，我们登录到控制平面 1 并生成一个新的 TLS 引导令牌，同时打印出 join 命令：

➜ ssh cluster3-controlplane1

➜ root@cluster3-controlplane1:~# kubeadm token create --print-join-command
kubeadm join 192.168.100.31:6443 --token rbhrjh.4o93r31o18an6dll --discovery-token-ca-cert-hash sha256:d94524f9ab1eed84417414c7def5c1608f84dbf04437d9f5f73eb6255dafdb18

➜ root@cluster3-controlplane1:~# kubeadm token list
TOKEN                     TTL         EXPIRES                ...
44dz0t.2lgmone0i1o5z9fe   <forever>   <never>
4u477f.nmpq48xmpjt6weje   1h          2022-12-21T18:14:30Z
rbhrjh.4o93r31o18an6dll   23h         2022-12-22T16:29:58Z

我们看到我们的令牌到期了 23 小时，我们可以通过传递 ttl 参数来调整它。

接下来，我们再次连接到 并简单地执行 join 命令：cluster3-node2

➜ ssh cluster3-node2

➜ root@cluster3-node2:~# kubeadm join 192.168.100.31:6443 --token rbhrjh.4o93r31o18an6dll --discovery-token-ca-cert-hash sha256:d94524f9ab1eed84417414c7def5c1608f84dbf04437d9f5f73eb6255dafdb18
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.


➜ root@cluster3-node2:~# service kubelet status
● kubelet.service - kubelet: The Kubernetes Node Agent
     Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
    Drop-In: /etc/systemd/system/kubelet.service.d
             └─10-kubeadm.conf
     Active: active (running) since Wed 2022-12-21 16:32:19 UTC; 1min 4s ago
       Docs: https://kubernetes.io/docs/home/
   Main PID: 32510 (kubelet)
      Tasks: 11 (limit: 462)
     Memory: 55.2M
     CGroup: /system.slice/kubelet.service
             └─32510 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runti>

如果您有问题，您可能需要 运行 .kubeadm join``kubeadm reset

这对我们来说看起来很棒。最后我们回到主终端并检查节点状态：

➜ k get node
NAME                     STATUS     ROLES           AGE   VERSION
cluster3-controlplane1   Ready      control-plane   22h   v1.26.0
cluster3-node1           Ready      <none>          22h   v1.26.0
cluster3-node2           NotReady   <none>          22h   v1.26.0

给它一点时间，直到节点准备就绪。

➜ k get node
NAME                     STATUS   ROLES           AGE   VERSION
cluster3-controlplane1   Ready    control-plane   22h   v1.26.0
cluster3-node1           Ready    <none>          22h   v1.26.0
cluster3-node2           Ready    <none>          22h   v1.26.0

我们看到 现在可用且是最新的。cluster3-node2

题库21

Use context: kubectl config use-context k8s-c3-CCC

 

Create a Static Pod named my-static-pod in Namespace default on cluster3-controlplane1. It should be of image nginx:1.16-alpine and have resource requests for 10m CPU and 20Mi memory.

Then create a NodePort Service named static-pod-service which exposes that static Pod on port 80 and check if it has Endpoints and if it's reachable through the cluster3-controlplane1 internal IP address. You can connect to the internal node IPs from your main terminal.

任务权重：2%

使用上下文：kubectl config use-context k8s-c3-CCC

在群集 3-控制平面 1 上的命名空间中创建一个命名。它应该是映像的，并且具有对 CPU 和内存的资源请求。Static Podmy-static-poddefaultnginx:1.16-alpine10m20Mi

然后创建一个名为的 NodePort 服务，该服务在端口 80 上公开该静态 Pod，并检查它是否有端点以及是否可以通过内部 IP 地址访问它。您可以从主终端连接到内部节点 IP。static-pod-servicecluster3-controlplane1

题解

答：

➜ ssh cluster3-controlplane1

➜ root@cluster1-controlplane1:~# cd /etc/kubernetes/manifests/

➜ root@cluster1-controlplane1:~# kubectl run my-static-pod \
    --image=nginx:1.16-alpine \
    -o yaml --dry-run=client > my-static-pod.yaml

然后编辑 以 添加请求的资源请求：my-static-pod.yaml

# /etc/kubernetes/manifests/my-static-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: my-static-pod
  name: my-static-pod
spec:
  containers:
  - image: nginx:1.16-alpine
    name: my-static-pod
    resources:
      requests:
        cpu: 10m
        memory: 20Mi
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

并确保它正在运行：

➜ k get pod -A | grep my-static
NAMESPACE     NAME                                   READY   STATUS   ...   AGE
default       my-static-pod-cluster3-controlplane1   1/1     Running  ...   22s

现在我们公开静态 Pod：

k expose pod my-static-pod-cluster3-controlplane1 \
  --name static-pod-service \
  --type=NodePort \
  --port 80

这将生成如下服务：

# kubectl expose pod my-static-pod-cluster3-controlplane1 --name static-pod-service --type=NodePort --port 80
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    run: my-static-pod
  name: static-pod-service
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: my-static-pod
  type: NodePort
status:
  loadBalancer: {}

然后运行并测试：

➜ k get svc,ep -l run=my-static-pod
NAME                         TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/static-pod-service   NodePort   10.99.168.252   <none>        80:30352/TCP   30s

NAME                           ENDPOINTS      AGE
endpoints/static-pod-service   10.32.0.4:80   30s

看起来不错。

题库22

Use context: kubectl config use-context k8s-c2-AC

 

Check how long the kube-apiserver server certificate is valid on cluster2-controlplane1. Do this with openssl or cfssl. Write the exipiration date into /opt/course/22/expiration.

Also run the correct kubeadm command to list the expiration dates and confirm both methods show the same date.

Write the correct kubeadm command that would renew the apiserver server certificate into /opt/course/22/kubeadm-renew-certs.sh.

任务权重：2%

使用上下文：kubectl config use-context k8s-c2-AC

检查 kube-apiserver 服务器证书在 上的有效期。使用 openssl 或 cfssl。将过期日期写入 。cluster2-controlplane1/opt/course/22/expiration

还要运行正确的命令以列出到期日期，并确认两种方法显示相同的日期。kubeadm

将续订 apiserver 服务器证书的正确命令写入 。kubeadm/opt/course/22/kubeadm-renew-certs.sh

题解

答：

首先，让我们找到该证书：

➜ ssh cluster2-controlplane1

➜ root@cluster2-controlplane1:~# find /etc/kubernetes/pki | grep apiserver
/etc/kubernetes/pki/apiserver.crt
/etc/kubernetes/pki/apiserver-etcd-client.crt
/etc/kubernetes/pki/apiserver-etcd-client.key
/etc/kubernetes/pki/apiserver-kubelet-client.crt
/etc/kubernetes/pki/apiserver.key
/etc/kubernetes/pki/apiserver-kubelet-client.key

接下来我们使用openssl找出到期日期：

➜ root@cluster2-controlplane1:~# openssl x509  -noout -text -in /etc/kubernetes/pki/apiserver.crt | grep Validity -A2
        Validity
            Not Before: Dec 20 18:05:20 2022 GMT
            Not After : Dec 20 18:05:20 2023 GMT

我们有它，所以我们把它写在主终端上的所需位置：

# /opt/course/22/expiration
Dec 20 18:05:20 2023 GMT

我们也使用 kubeadm 的功能来获取到期时间：

➜ root@cluster2-controlplane1:~# kubeadm certs check-expiration | grep apiserver
apiserver                Jan 14, 2022 18:49 UTC   363d        ca               no      
apiserver-etcd-client    Jan 14, 2022 18:49 UTC   363d        etcd-ca          no      
apiserver-kubelet-client Jan 14, 2022 18:49 UTC   363d        ca               no 

看起来不错。最后，我们编写命令，将所有证书续订到请求的位置：

# /opt/course/22/kubeadm-renew-certs.sh
kubeadm certs renew apiserver

题库23

Use context: kubectl config use-context k8s-c2-AC

 

Node cluster2-node1 has been added to the cluster using kubeadm and TLS bootstrapping.

Find the "Issuer" and "Extended Key Usage" values of the cluster2-node1:

kubelet client certificate, the one used for outgoing connections to the kube-apiserver.
kubelet server certificate, the one used for incoming connections from the kube-apiserver.
Write the information into file /opt/course/23/certificate-info.txt.

Compare the "Issuer" and "Extended Key Usage" fields of both certificates and make sense of these.

任务权重：2%

使用上下文：kubectl config use-context k8s-c2-AC

节点群集 2-节点 1 已使用 和 TLS 引导添加到群集中。kubeadm

查找 cluster2-node1 的“颁发者”和“扩展密钥用法”值：

kubelet 客户端证书，用于与 kube-apiserver 的传出连接。
kubelet 服务器证书，用于从 kube-apiserver 传入连接的证书。
将信息写入文件 。/opt/course/23/certificate-info.txt

比较两个证书的“颁发者”和“扩展密钥用法”字段并理解这些字段。

题解

答：

要找到正确的 kubelet 证书目录，我们可以查找 kubelet 参数的默认值 。对于在 Kubernetes 文档中搜索“kubelet”，这将导致：https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet。我们可以检查是否已使用 或 在 中配置了另一个证书目录。--cert-dir``ps aux``/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

首先我们检查 kubelet 客户端证书：

➜ ssh cluster2-node1

➜ root@cluster2-node1:~# openssl x509  -noout -text -in /var/lib/kubelet/pki/kubelet-client-current.pem | grep Issuer
        Issuer: CN = kubernetes
        
➜ root@cluster2-node1:~# openssl x509  -noout -text -in /var/lib/kubelet/pki/kubelet-client-current.pem | grep "Extended Key Usage" -A1
            X509v3 Extended Key Usage: 
                TLS Web Client Authentication

接下来我们检查 kubelet 服务器证书：

➜ root@cluster2-node1:~# openssl x509  -noout -text -in /var/lib/kubelet/pki/kubelet.crt | grep Issuer
          Issuer: CN = cluster2-node1-ca@1588186506

➜ root@cluster2-node1:~# openssl x509  -noout -text -in /var/lib/kubelet/pki/kubelet.crt | grep "Extended Key Usage" -A1
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication

我们看到服务器证书是在工作节点本身上生成的，客户端证书是由 Kubernetes API 颁发的。“扩展密钥用法”还显示它是用于客户端还是服务器身份验证。

更多关于这个： https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping

题库24

Use context: kubectl config use-context k8s-c1-H

 

There was a security incident where an intruder was able to access the whole cluster from a single hacked backend Pod.

To prevent this create a NetworkPolicy called np-backend in Namespace project-snake. It should allow the backend-* Pods only to:

connect to db1-* Pods on port 1111
connect to db2-* Pods on port 2222
Use the app label of Pods in your policy.

After implementation, connections from backend-* Pods to vault-* Pods on port 3333 should for example no longer work.

任务权重：9%

使用上下文：kubectl config use-context k8s-c1-H

发生了一起安全事件，入侵者能够从单个被黑客入侵的后端 Pod 访问整个集群。

为了防止这种情况，请创建一个在命名空间中调用的网络策略。它应该只允许 Pod 执行以下操作：np-backendproject-snakebackend-*

连接到端口 1111 上的 Podsdb1-*
连接到端口 2222 上的 Podsdb2-*
在策略中使用 Pod 标签。app

例如，实现后，端口 3333 上从 Pod 到 Pod 的连接应该不再有效。backend-*vault-*

题解

答：

首先，我们看一下现有的 Pod 及其标签：

➜ k -n project-snake get pod
NAME        READY   STATUS    RESTARTS   AGE
backend-0   1/1     Running   0          8s
db1-0       1/1     Running   0          8s
db2-0       1/1     Running   0          10s
vault-0     1/1     Running   0          10s

➜ k -n project-snake get pod -L app
NAME        READY   STATUS    RESTARTS   AGE     APP
backend-0   1/1     Running   0          3m15s   backend
db1-0       1/1     Running   0          3m15s   db1
db2-0       1/1     Running   0          3m17s   db2
vault-0     1/1     Running   0          3m17s   vault

我们测试了当前的连接情况，发现没有任何限制：

➜ k -n project-snake get pod -o wide
NAME        READY   STATUS    RESTARTS   AGE     IP          ...
backend-0   1/1     Running   0          4m14s   10.44.0.24  ...
db1-0       1/1     Running   0          4m14s   10.44.0.25  ...
db2-0       1/1     Running   0          4m16s   10.44.0.23  ...
vault-0     1/1     Running   0          4m16s   10.44.0.22  ...

➜ k -n project-snake exec backend-0 -- curl -s 10.44.0.25:1111
database one

➜ k -n project-snake exec backend-0 -- curl -s 10.44.0.23:2222
database two

➜ k -n project-snake exec backend-0 -- curl -s 10.44.0.22:3333
vault secret storage

现在我们通过 从 k8s 文档中复制和追踪一个示例来创建 NP：

vim 24_np.yaml
# 24_np.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: np-backend
  namespace: project-snake
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
    - Egress                    # policy is only about Egress
  egress:
    -                           # first rule
      to:                           # first condition "to"
      - podSelector:
          matchLabels:
            app: db1
      ports:                        # second condition "port"
      - protocol: TCP
        port: 1111
    -                           # second rule
      to:                           # first condition "to"
      - podSelector:
          matchLabels:
            app: db2
      ports:                        # second condition "port"
      - protocol: TCP
        port: 2222

上面的NP有两个规则，每个规则有两个条件，可以理解为：

allow outgoing traffic if:
  (destination pod has label app=db1 AND port is 1111)
  OR
  (destination pod has label app=db2 AND port is 2222)

错误的例子

现在让我们看一个错误的例子：

# WRONG
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: np-backend
  namespace: project-snake
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
    - Egress
  egress:
    -                           # first rule
      to:                           # first condition "to"
      - podSelector:                    # first "to" possibility
          matchLabels:
            app: db1
      - podSelector:                    # second "to" possibility
          matchLabels:
            app: db2
      ports:                        # second condition "ports"
      - protocol: TCP                   # first "ports" possibility
        port: 1111
      - protocol: TCP                   # second "ports" possibility
        port: 2222

上面的NP有一个规则，有两个条件，每个规则有两个条件条目，可以读作：

allow outgoing traffic if:
  (destination pod has label app=db1 OR destination pod has label app=db2)
  AND
  (destination port is 1111 OR destination port is 2222)

使用此 NP，Pod 仍然可以 连接到 端口 1111 上的 Pod，例如应该禁止的端口。backend-*``db2-*

创建网络策略

我们创建正确的NP：

k -f 24_np.yaml create

并再次测试：

➜ k -n project-snake exec backend-0 -- curl -s 10.44.0.25:1111
database one

➜ k -n project-snake exec backend-0 -- curl -s 10.44.0.23:2222
database two

➜ k -n project-snake exec backend-0 -- curl -s 10.44.0.22:3333
^C

在NP上使用 也有助于了解k8s如何解释政策。kubectl describe

太好了，看起来更安全。任务完成。

题库25

Use context: kubectl config use-context k8s-c3-CCC

 

Make a backup of etcd running on cluster3-controlplane1 and save it on the controlplane node at /tmp/etcd-backup.db.

Then create a Pod of your kind in the cluster.

Finally restore the backup, confirm the cluster is still working and that the created Pod is no longer with us.

任务权重：8%

使用上下文：kubectl config use-context k8s-c3-CCC

备份在集群 3-控制平面 1 上运行的 etcd，并将其保存在控制平面节点上。/tmp/etcd-backup.db

然后在集群中创建同类 Pod。

最后恢复备份，确认集群仍在工作，并且创建的 Pod 不再与我们在一起。

题解

答：

蚀刻备份

首先，我们登录控制平面并尝试创建 etcd 的快照：

➜ ssh cluster3-controlplane1

➜ root@cluster3-controlplane1:~# ETCDCTL_API=3 etcdctl snapshot save /tmp/etcd-backup.db
Error:  rpc error: code = Unavailable desc = transport is closing

但它失败了，因为我们需要验证自己。有关必要的信息，我们可以检查 etc 清单：

➜ root@cluster3-controlplane1:~# vim /etc/kubernetes/manifests/etcd.yaml

我们只检查 必要的信息，我们不会更改它。etcd.yaml

# /etc/kubernetes/manifests/etcd.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: etcd
    tier: control-plane
  name: etcd
  namespace: kube-system
spec:
  containers:
  - command:
    - etcd
    - --advertise-client-urls=https://192.168.100.31:2379
    - --cert-file=/etc/kubernetes/pki/etcd/server.crt                           # use
    - --client-cert-auth=true
    - --data-dir=/var/lib/etcd
    - --initial-advertise-peer-urls=https://192.168.100.31:2380
    - --initial-cluster=cluster3-controlplane1=https://192.168.100.31:2380
    - --key-file=/etc/kubernetes/pki/etcd/server.key                            # use
    - --listen-client-urls=https://127.0.0.1:2379,https://192.168.100.31:2379   # use
    - --listen-metrics-urls=http://127.0.0.1:2381
    - --listen-peer-urls=https://192.168.100.31:2380
    - --name=cluster3-controlplane1
    - --peer-cert-file=/etc/kubernetes/pki/etcd/peer.crt
    - --peer-client-cert-auth=true
    - --peer-key-file=/etc/kubernetes/pki/etcd/peer.key
    - --peer-trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt                    # use
    - --snapshot-count=10000
    - --trusted-ca-file=/etc/kubernetes/pki/etcd/ca.crt
    image: k8s.gcr.io/etcd:3.3.15-0
    imagePullPolicy: IfNotPresent
    livenessProbe:
      failureThreshold: 8
      httpGet:
        host: 127.0.0.1
        path: /health
        port: 2381
        scheme: HTTP
      initialDelaySeconds: 15
      timeoutSeconds: 15
    name: etcd
    resources: {}
    volumeMounts:
    - mountPath: /var/lib/etcd
      name: etcd-data
    - mountPath: /etc/kubernetes/pki/etcd
      name: etcd-certs
  hostNetwork: true
  priorityClassName: system-cluster-critical
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki/etcd
      type: DirectoryOrCreate
    name: etcd-certs
  - hostPath:
      path: /var/lib/etcd                                                     # important
      type: DirectoryOrCreate
    name: etcd-data
status: {}

但我们也知道 API 服务器正在连接到 etcd，因此我们可以检查其清单是如何配置的：

➜ root@cluster3-controlplane1:~# cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep etcd
    - --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
    - --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
    - --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
    - --etcd-servers=https://127.0.0.1:2379

我们使用身份验证信息并将其传递给 etcdctl：

➜ root@cluster3-controlplane1:~# ETCDCTL_API=3 etcdctl snapshot save /tmp/etcd-backup.db \
--cacert /etc/kubernetes/pki/etcd/ca.crt \
--cert /etc/kubernetes/pki/etcd/server.crt \
--key /etc/kubernetes/pki/etcd/server.key

Snapshot saved at /tmp/etcd-backup.db

**注意：**请勿使用 ，因为它可能会更改快照文件并使其无效snapshot status

蚀刻恢复

现在在集群中创建一个 Pod 并等待它运行：

➜ root@cluster3-controlplane1:~# kubectl run test --image=nginx
pod/test created

➜ root@cluster3-controlplane1:~# kubectl get pod -l run=test -w
NAME   READY   STATUS    RESTARTS   AGE
test   1/1     Running   0          60s

**注意：**如果您没有解决问题 18 或 20，并且 cluster3 没有就绪的工作节点，则创建的 Pod 可能会保持“挂起”状态。对于此任务，这仍然可以。

接下来，我们停止所有控制平面组件：

root@cluster3-controlplane1:~# cd /etc/kubernetes/manifests/

root@cluster3-controlplane1:/etc/kubernetes/manifests# mv * ..

root@cluster3-controlplane1:/etc/kubernetes/manifests# watch crictl ps

现在我们将快照还原到特定目录中：

➜ root@cluster3-controlplane1:~# ETCDCTL_API=3 etcdctl snapshot restore /tmp/etcd-backup.db \
--data-dir /var/lib/etcd-backup \
--cacert /etc/kubernetes/pki/etcd/ca.crt \
--cert /etc/kubernetes/pki/etcd/server.crt \
--key /etc/kubernetes/pki/etcd/server.key

2020-09-04 16:50:19.650804 I | mvcc: restore compact to 9935
2020-09-04 16:50:19.659095 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32

我们可以指定另一个主机来使用 进行备份，但这里我们只使用默认值：。 etcdctl --endpoints http://IP``http://127.0.0.1:2379,http://127.0.0.1:4001

恢复的文件位于 新文件夹 ，现在我们必须告诉 etcd 使用该目录：/var/lib/etcd-backup

➜ root@cluster3-controlplane1:~# vim /etc/kubernetes/etcd.yaml
# /etc/kubernetes/etcd.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: etcd
    tier: control-plane
  name: etcd
  namespace: kube-system
spec:
...
    - mountPath: /etc/kubernetes/pki/etcd
      name: etcd-certs
  hostNetwork: true
  priorityClassName: system-cluster-critical
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki/etcd
      type: DirectoryOrCreate
    name: etcd-certs
  - hostPath:
      path: /var/lib/etcd-backup                # change
      type: DirectoryOrCreate
    name: etcd-data
status: {}

现在，我们将所有控制平面 yaml 再次移动到清单目录中。给它一些时间（最多几分钟）让 etcd 重新启动并再次访问 API 服务器：

root@cluster3-controlplane1:/etc/kubernetes/manifests# mv ../*.yaml .

root@cluster3-controlplane1:/etc/kubernetes/manifests# watch crictl ps

然后我们再次检查 Pod ：

➜ root@cluster3-controlplane1:~# kubectl get pod -l run=test
No resources found in default namespace.

太棒了，备份和还原在我们的 pod 消失时工作。

@cluster3-controlplane1:~# cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep etcd
- --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt
- --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt
- --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key
- --etcd-servers=https://127.0.0.1:2379

我们使用身份验证信息并将其传递给 etcdctl：

➜ root@cluster3-controlplane1:~# ETCDCTL_API=3 etcdctl snapshot save /tmp/etcd-backup.db
–cacert /etc/kubernetes/pki/etcd/ca.crt
–cert /etc/kubernetes/pki/etcd/server.crt
–key /etc/kubernetes/pki/etcd/server.key

Snapshot saved at /tmp/etcd-backup.db

 

> **注意：**请勿使用 ，因为它可能会更改快照文件并使其无效`snapshot status`

 

###### 蚀刻恢复

现在在集群中创建一个 *Pod* 并等待它运行：

➜ root@cluster3-controlplane1:~# kubectl run test --image=nginx
pod/test created

➜ root@cluster3-controlplane1:~# kubectl get pod -l run=test -w
NAME READY STATUS RESTARTS AGE
test 1/1 Running 0 60s

 

> **注意：**如果您没有解决问题 18 或 20，并且 cluster3 没有就绪的工作节点，则创建的 Pod 可能会保持“挂起”状态。对于此任务，这仍然可以。

 

接下来，我们停止所有控制平面组件：

root@cluster3-controlplane1:~# cd /etc/kubernetes/manifests/

root@cluster3-controlplane1:/etc/kubernetes/manifests# mv * …

root@cluster3-controlplane1:/etc/kubernetes/manifests# watch crictl ps

现在我们将快照还原到特定目录中：

➜ root@cluster3-controlplane1:~# ETCDCTL_API=3 etcdctl snapshot restore /tmp/etcd-backup.db
–data-dir /var/lib/etcd-backup
–cacert /etc/kubernetes/pki/etcd/ca.crt
–cert /etc/kubernetes/pki/etcd/server.crt
–key /etc/kubernetes/pki/etcd/server.key

2020-09-04 16:50:19.650804 I | mvcc: restore compact to 9935
2020-09-04 16:50:19.659095 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32

我们可以指定另一个主机来使用 进行备份，但这里我们只使用默认值：。 `etcdctl --endpoints http://IP``http://127.0.0.1:2379,http://127.0.0.1:4001`

恢复的文件位于 新文件夹 ，现在我们必须告诉 etcd 使用该目录：`/var/lib/etcd-backup`

➜ root@cluster3-controlplane1:~# vim /etc/kubernetes/etcd.yaml

/etc/kubernetes/etcd.yaml

apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
component: etcd
tier: control-plane
name: etcd
namespace: kube-system
spec:
…
- mountPath: /etc/kubernetes/pki/etcd
name: etcd-certs
hostNetwork: true
priorityClassName: system-cluster-critical
volumes:

• hostPath:
  path: /etc/kubernetes/pki/etcd
  type: DirectoryOrCreate
  name: etcd-certs
• hostPath:
  path: /var/lib/etcd-backup # change
  type: DirectoryOrCreate
  name: etcd-data
  status: {}

现在，我们将所有控制平面 yaml 再次移动到清单目录中。给它一些时间（最多几分钟）让 etcd 重新启动并再次访问 API 服务器：

root@cluster3-controlplane1:/etc/kubernetes/manifests# mv …/*.yaml .

root@cluster3-controlplane1:/etc/kubernetes/manifests# watch crictl ps

然后我们再次检查 *Pod* ：

➜ root@cluster3-controlplane1:~# kubectl get pod -l run=test
No resources found in default namespace.

太棒了，备份和还原在我们的 pod 消失时工作。