k8s常见的资源对象使用

news2024/11/24 17:24:49

目录

一、kubernetes内置资源对象

1.1、kubernetes内置资源对象介绍

1.2、kubernetes资源对象操作命令

二、job与cronjob计划任务

2.1、job计划任务

2.2、cronjob计划任务

三、RC/RS副本控制器

3.1、RC副本控制器

3.2、RS副本控制器

3.3、RS更新pod

四、Deployment副本控制器

4.1、Deployment副本控制器

五、Kubernetes之Service

5.1、Kubernetes Service介绍

5.2、service类型

六、Kubernetes之configmap

七、Kubernetes之Secret

7.1、Secret简介

7.2、Secret简介类型

7.3、Secret类型-Opaque格式

7.4、Secret类型-kubernetes.io/tls-为nginx提供证书

7.5、Secret-kubernetes.io/dockerconfigjson类型

一、kubernetes内置资源对象

1.1、kubernetes内置资源对象介绍

1.2、kubernetes资源对象操作命令

官网介绍：https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/deployment/

二、job与cronjob计划任务

2.1、job计划任务

job属于一次性任务，常用于环境初始化例如mysql/elasticsearch。

root@easzlab-deploy:~/jiege-k8s/pod-test# cat 1.job.yaml apiVersion: batch/v1
kind: Job
metadata:
  name: job-mysql-init
spec:
  template:
    spec:
      containers:
      - name: job-mysql-init-container
        image: centos:7.9.2009
        command: ["/bin/sh"]
        args: ["-c", "echo data init job at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
        volumeMounts:
        - mountPath: /cache
          name: cache-volume
      volumes:
      - name: cache-volume
        hostPath:
          path: /tmp/jobdata
      restartPolicy: Never
root@easzlab-deploy:~/pod-test# kubectl apply -f 1.job.yaml 
job.batch/job-mysql-init created
root@easzlab-deploy:~/pod-test# kubectl get pod -A
NAMESPACE              NAME                                              READY   STATUS              RESTARTS       AGE
default                job-mysql-init-n29g9                              0/1     ContainerCreating   0              14s
kube-system            calico-kube-controllers-5c8bb696bb-fxbmr          1/1     Running             1 (3d7h ago)   7d18h
kube-system            calico-node-2qtfm                                 1/1     Running             1 (3d7h ago)   7d18h
kube-system            calico-node-8l78t                                 1/1     Running             1 (3d7h ago)   7d18h
kube-system            calico-node-9b75m                                 1/1     Running             1 (3d7h ago)   7d18h
kube-system            calico-node-k75jh                                 1/1     Running             1 (3d7h ago)   7d18h
kube-system            calico-node-kmbhs                                 1/1     Running             1 (3d7h ago)   7d18h
kube-system            calico-node-lxfk9                                 1/1     Running             1 (3d7h ago)   7d18h
kube-system            coredns-69548bdd5f-6df7j                          1/1     Running             1 (3d7h ago)   7d6h
kube-system            coredns-69548bdd5f-nl5qc                          1/1     Running             1 (3d7h ago)   7d6h
kubernetes-dashboard   dashboard-metrics-scraper-8c47d4b5d-2d275         1/1     Running             1 (3d7h ago)   7d6h
kubernetes-dashboard   kubernetes-dashboard-5676d8b865-6l8n8             1/1     Running             1 (3d7h ago)   7d6h
linux70                linux70-tomcat-app1-deployment-5d666575cc-kbjhk   1/1     Running             1 (3d7h ago)   5d7h
myserver               linux70-nginx-deployment-55dc5fdcf9-58ll2         1/1     Running             0              20h
myserver               linux70-nginx-deployment-55dc5fdcf9-6xcjk         1/1     Running             0              20h
myserver               linux70-nginx-deployment-55dc5fdcf9-cxg5m         1/1     Running             0              20h
myserver               linux70-nginx-deployment-55dc5fdcf9-gv2gk         1/1     Running             0              20h
velero-system          velero-858b9459f9-5mxxx                           1/1     Running             0              21h
root@easzlab-deploy:~/pod-test#

2.2、cronjob计划任务

cronjob属于周期性任务，cronjob广泛用于数据库计划备份场景。

root@easzlab-deploy:~/jiege-k8s/pod-test# cat 2.cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: cronjob-mysql-databackup
spec:
  schedule: "*/2 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: cronjob-mysql-databackup-pod
            image: centos:7.9.2009
            command: ["/bin/sh"]
            args: ["-c", "echo mysql databackup cronjob at `date +%Y-%m-%d_%H-%M-%S` >> /cache/data.log"]
            volumeMounts:
            - mountPath: /cache
              name: cache-volume
          volumes:
          - name: cache-volume
            hostPath:
              path: /tmp/cronjobdata
          restartPolicy: OnFailure
root@easzlab-deploy:~/pod-test# kubectl apply -f 2.cronjob.yaml
root@easzlab-deploy:~/pod-test# 
root@easzlab-deploy:~/pod-test# kubectl get pod -A -owide
NAMESPACE              NAME                                              READY   STATUS      RESTARTS       AGE     IP              NODE            NOMINATED NODE   READINESS GATES
default                cronjob-mysql-databackup-27661544-wntbb           0/1     Completed   0              4m3s    10.200.2.13     172.16.88.159   <none>           <none>
default                cronjob-mysql-databackup-27661546-lbf2t           0/1     Completed   0              2m3s    10.200.2.14     172.16.88.159   <none>           <none>
default                cronjob-mysql-databackup-27661548-8p9j6           0/1     Completed   0              3s      10.200.2.15     172.16.88.159   <none>           <none>
kube-system            calico-kube-controllers-5c8bb696bb-fxbmr          1/1     Running     1 (3d7h ago)   7d18h   172.16.88.159   172.16.88.159   <none>           <none>
kube-system            calico-node-2qtfm                                 1/1     Running     1 (3d7h ago)   7d18h   172.16.88.158   172.16.88.158   <none>           <none>
kube-system            calico-node-8l78t                                 1/1     Running     1 (3d7h ago)   7d18h   172.16.88.154   172.16.88.154   <none>           <none>
kube-system            calico-node-9b75m                                 1/1     Running     1 (3d7h ago)   7d18h   172.16.88.156   172.16.88.156   <none>           <none>
kube-system            calico-node-k75jh                                 1/1     Running     1 (3d7h ago)   7d18h   172.16.88.157   172.16.88.157   <none>           <none>
kube-system            calico-node-kmbhs                                 1/1     Running     1 (3d7h ago)   7d18h   172.16.88.159   172.16.88.159   <none>           <none>
kube-system            calico-node-lxfk9                                 1/1     Running     1 (3d7h ago)   7d18h   172.16.88.155   172.16.88.155   <none>           <none>
kube-system            coredns-69548bdd5f-6df7j                          1/1     Running     1 (3d7h ago)   7d6h    10.200.2.6      172.16.88.159   <none>           <none>
kube-system            coredns-69548bdd5f-nl5qc                          1/1     Running     1 (3d7h ago)   7d6h    10.200.40.199   172.16.88.157   <none>           <none>
kubernetes-dashboard   dashboard-metrics-scraper-8c47d4b5d-2d275         1/1     Running     1 (3d7h ago)   7d6h    10.200.40.197   172.16.88.157   <none>           <none>
kubernetes-dashboard   kubernetes-dashboard-5676d8b865-6l8n8             1/1     Running     1 (3d7h ago)   7d6h    10.200.40.198   172.16.88.157   <none>           <none>
linux70                linux70-tomcat-app1-deployment-5d666575cc-kbjhk   1/1     Running     1 (3d7h ago)   5d7h    10.200.233.67   172.16.88.158   <none>           <none>
myserver               linux70-nginx-deployment-55dc5fdcf9-58ll2         1/1     Running     0              21h     10.200.2.10     172.16.88.159   <none>           <none>
myserver               linux70-nginx-deployment-55dc5fdcf9-6xcjk         1/1     Running     0              21h     10.200.2.9      172.16.88.159   <none>           <none>
myserver               linux70-nginx-deployment-55dc5fdcf9-cxg5m         1/1     Running     0              21h     10.200.2.11     172.16.88.159   <none>           <none>
myserver               linux70-nginx-deployment-55dc5fdcf9-gv2gk         1/1     Running     0              21h     10.200.233.69   172.16.88.158   <none>           <none>
velero-system          velero-858b9459f9-5mxxx                           1/1     Running     0              21h     10.200.40.202   172.16.88.157   <none>           <none>
root@easzlab-deploy:~/pod-test#

三、RC/RS副本控制器

3.1、RC副本控制器

Replication Controller： 副本控制器（ selector = !=） #第一代pod副本控制器
https://kubernetes.io/zh/docs/concepts/workloads/controllers/replicationcontroller/
https://kubernetes.io/zh/docs/concepts/overview/working-with-objects/labels/

root@easzlab-deploy:~/jiege-k8s/pod-test# cat 1.rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: ng-rc
spec:
  replicas: 2
  selector:
     app: ng-rc-80
  template:
    metadata:
      labels:
        app: ng-rc-80
    spec:
      containers:
      - name: ng-rc-80
        image: nginx
        ports:
        - containerPort: 80
root@easzlab-deploy:~/pod-test# kubectl apply -f 1.rc.yaml 
replicationcontroller/ng-rc created
root@easzlab-deploy:~/pod-test# 
root@easzlab-deploy:~/pod-test# kubectl get pod -A
NAMESPACE              NAME                                              READY   STATUS    RESTARTS        AGE
default                ng-rc-528fl                                       1/1     Running   0               2m8s
default                ng-rc-d6zqx                                       1/1     Running   0               2m8s
kube-system            calico-kube-controllers-5c8bb696bb-fxbmr          1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-2qtfm                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-8l78t                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-9b75m                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-k75jh                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-kmbhs                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-lxfk9                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            coredns-69548bdd5f-6df7j                          1/1     Running   1 (3d10h ago)   7d9h
kube-system            coredns-69548bdd5f-nl5qc                          1/1     Running   1 (3d10h ago)   7d9h
kubernetes-dashboard   dashboard-metrics-scraper-8c47d4b5d-2d275         1/1     Running   1 (3d10h ago)   7d9h
kubernetes-dashboard   kubernetes-dashboard-5676d8b865-6l8n8             1/1     Running   1 (3d10h ago)   7d9h
linux70                linux70-tomcat-app1-deployment-5d666575cc-kbjhk   1/1     Running   1 (3d10h ago)   5d9h
myserver               linux70-nginx-deployment-55dc5fdcf9-58ll2         1/1     Running   0               23h
myserver               linux70-nginx-deployment-55dc5fdcf9-6xcjk         1/1     Running   0               23h
myserver               linux70-nginx-deployment-55dc5fdcf9-cxg5m         1/1     Running   0               23h
myserver               linux70-nginx-deployment-55dc5fdcf9-gv2gk         1/1     Running   0               23h
velero-system          velero-858b9459f9-5mxxx                           1/1     Running   0               24h
root@easzlab-deploy:~/pod-test#

3.2、RS副本控制器

ReplicaSet：副本控制器，和副本控制器的区别是：对选择器的支持（ selector 还支持in notin） #第二代pod副本控制器
https://kubernetes.io/zh/docs/concepts/workloads/controllers/replicaset/

root@easzlab-deploy:~/jiege-k8s/pod-test# cat 2.rs.yaml 
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: frontend
spec:
  replicas: 2
  selector:
    matchExpressions:
      - {key: app, operator: In, values: [ng-rs-80,ng-rs-81]}
  template:
    metadata:
      labels:
        app: ng-rs-80
    spec:
      containers:
      - name: ng-rs-80
        image: nginx
        ports:
        - containerPort: 80
root@easzlab-deploy:~/pod-test# kubectl apply -f 2.rs.yaml 
replicaset.apps/frontend created
root@easzlab-deploy:~/pod-test# kubectl get pod -A
NAMESPACE              NAME                                              READY   STATUS    RESTARTS        AGE
default                frontend-jl67s                                    1/1     Running   0               97s
default                frontend-w7rb5                                    1/1     Running   0               97s
kube-system            calico-kube-controllers-5c8bb696bb-fxbmr          1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-2qtfm                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-8l78t                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-9b75m                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-k75jh                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-kmbhs                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            calico-node-lxfk9                                 1/1     Running   1 (3d10h ago)   7d21h
kube-system            coredns-69548bdd5f-6df7j                          1/1     Running   1 (3d10h ago)   7d10h
kube-system            coredns-69548bdd5f-nl5qc                          1/1     Running   1 (3d10h ago)   7d10h
kubernetes-dashboard   dashboard-metrics-scraper-8c47d4b5d-2d275         1/1     Running   1 (3d10h ago)   7d10h
kubernetes-dashboard   kubernetes-dashboard-5676d8b865-6l8n8             1/1     Running   1 (3d10h ago)   7d10h
linux70                linux70-tomcat-app1-deployment-5d666575cc-kbjhk   1/1     Running   1 (3d10h ago)   5d10h
myserver               linux70-nginx-deployment-55dc5fdcf9-58ll2         1/1     Running   0               24h
myserver               linux70-nginx-deployment-55dc5fdcf9-6xcjk         1/1     Running   0               24h
myserver               linux70-nginx-deployment-55dc5fdcf9-cxg5m         1/1     Running   0               24h
myserver               linux70-nginx-deployment-55dc5fdcf9-gv2gk         1/1     Running   0               24h
velero-system          velero-858b9459f9-5mxxx                           1/1     Running   0               24h
root@easzlab-deploy:~/pod-test#

3.3、RS更新pod

如需要手动指定镜像进行更新

kubectl set image replicaset/fronted ng-rs-80=nginx:1.18.2

四、Deployment副本控制器

4.1、Deployment副本控制器

Deployment 为 Pod 和 ReplicaSet 提供声明式的更新能力，Deployment比rs更高一级的控制器，除了有rs的功能之外，还有滚动升级、回滚、策略清理、金丝雀部署等等。

官网文档：https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/deployment/

root@easzlab-deploy:~/jiege-k8s/pod-test# cat 1.deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 3 #设置副本数
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
root@easzlab-deploy:~/jiege-k8s/pod-test# kubectl apply -f 1.deployment.yaml 
deployment.apps/nginx-deployment created
root@easzlab-deploy:~/jiege-k8s/pod-test# kubectl get pod
NAME                                READY   STATUS    RESTARTS      AGE
mysql-77d55bfdd8-cbtcz              1/1     Running   2 (16h ago)   39h
nginx-deployment-6595874d85-hm5gx   1/1     Running   0             19m
nginx-deployment-6595874d85-wdwx9   1/1     Running   0             19m
nginx-deployment-6595874d85-z8dsf   1/1     Running   0             19m
root@easzlab-deploy:~/jiege-k8s/pod-test#

五、Kubernetes之Service

5.1、Kubernetes Service介绍

由于pod重建之后ip就变了， 因此pod之间使用pod的IP直接访问会出现无法访问的问题， 而service则解耦了服务和应用， service的实现方式就是通过label标签动态匹配后端endpoint。
kube-proxy监听着k8s-apiserver，一旦service资源发生变化（调k8sapi修改service信息） ， kubeproxy就会生成对应的负载调度的调整， 这样就保证service的最新状态。
kube-proxy有三种调度模型

  • userspace： k8s1.1之前
  • iptables： 1.2-k8s1.11之前
  • ipvs： k8s 1.11之后， 如果没有开启ipvs， 则自动降级为iptables

5.2、service类型

  • ClusterIP： 用于内部服务基于service name的访问。
  • NodePort： 用于kubernetes集群以外的服务主动访问运行在kubernetes集群内部的服务。
  • LoadBalancer： 用于公有云环境的服务暴露。
  • ExternalName： 用于将k8s集群外部的服务映射至k8s集群内部访问， 从而让集群内部的pod能够通过固定的service name访问集群外部的服务， 有时候也用于将不同namespace之间的pod通过ExternalName进行访问。

应用案例

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case2# cat 1-deploy_node.yml 
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    #matchLabels: #rs or deployment
    #  app: ng-deploy3-80
    matchExpressions:
      - {key: app, operator: In, values: [ng-deploy-80,ng-rs-81]}
  template:
    metadata:
      labels:
        app: ng-deploy-80
    spec:
      containers:
      - name: ng-deploy-80
        image: nginx:1.16.1
        ports:
        - containerPort: 80
      #nodeSelector:
      #  env: group1
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case2# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case2#
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case2# cat 2-svc_service.yml 
apiVersion: v1
kind: Service
metadata:
  name: ng-deploy-80 
spec:
  ports:
  - name: http
    port: 88
    targetPort: 80
    protocol: TCP
  type: ClusterIP
  selector:
    app: ng-deploy-80
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case2# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case2# cat 3-svc_NodePort.yml 
apiVersion: v1
kind: Service
metadata:
  name: ng-deploy-80 
spec:
  ports:
  - name: http
    port: 90
    targetPort: 80
    nodePort: 30012
    protocol: TCP
  type: NodePort
  selector:
    app: ng-deploy-80
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case2#

六、Kubernetes之configmap

Configmap配置信息和镜像解耦, 实现方式为将配置信息放到configmap对象中， 然后在pod的中作为Volume挂载到pod中， 从而实现导入配置的目的。

使用场景：

  • 通过Configmap给pod定义全局环境变量
  • 通过Configmap给pod传递命令行参数， 如mysql -u -p中的账户名密码可以通过Configmap传递。
  • 通过Configmap给pod中的容器服务提供配置文件， 配置文件以挂载到容器的形式使用。

注意事项：

  • Configmap需要在pod使用它之前创建。
  • pod只能使用位于同一个namespace的Configmap， 及Configmap不能夸namespace使用。
  • 通常用于非安全加密的配置场景。
  • Configmap通常是小于1MB的配置。

应用案例

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# cat deploy_configmap.yml

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
 default: |
    server {
       listen       80;
       server_name  www.mysite.com;
       index        index.html;

       location / {
           root /data/nginx/html;
           if (!-e $request_filename) {
               rewrite ^/(.*) /index.html last;
           }
       }
    }

---
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ng-deploy-80
  template:
    metadata:
      labels:
        app: ng-deploy-80
    spec:
      containers:
      - name: ng-deploy-8080
        image: tomcat
        ports:
        - containerPort: 8080
        volumeMounts:
        - name: nginx-config
          mountPath:  /data
      - name: ng-deploy-80
        image: nginx 
        ports:
        - containerPort: 80
        volumeMounts:
        - mountPath: /data/nginx/html
          name: nginx-static-dir
        - name: nginx-config
          mountPath:  /etc/nginx/conf.d
      volumes:
      - name: nginx-static-dir
        hostPath:
          path: /data/nginx/linux70
      - name: nginx-config
        configMap:
          name: nginx-config
          items:
             - key: default
               path: mysite.conf

---
apiVersion: v1
kind: Service
metadata:
  name: ng-deploy-80
spec:
  ports:
  - name: http
    port: 81
    targetPort: 80
    nodePort: 30019
    protocol: TCP
  type: NodePort
  selector:
    app: ng-deploy-80

安装并验证

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl get pod
NAME                     READY   STATUS    RESTARTS      AGE
mysql-77d55bfdd8-cbtcz   1/1     Running   2 (18h ago)   41h
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl get configmap
NAME                 DATA   AGE
istio-ca-root-cert   1      40h
kube-root-ca.crt     1      47h
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl apply -f deploy_configmap.yml 
configmap/nginx-config created
deployment.apps/nginx-deployment created
service/ng-deploy-80 created
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl get pod -owide
NAME                                READY   STATUS    RESTARTS      AGE   IP               NODE            NOMINATED NODE   READINESS GATES
mysql-77d55bfdd8-cbtcz              1/1     Running   2 (18h ago)   41h   10.200.104.212   172.16.88.163   <none>           <none>
nginx-deployment-5699c4696d-gr4gm   2/2     Running   0             27s   10.200.104.216   172.16.88.163   <none>           <none>
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl get configmap
NAME                 DATA   AGE
istio-ca-root-cert   1      40h
kube-root-ca.crt     1      47h
nginx-config         1      32s
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl get configmap nginx-config -oyaml
apiVersion: v1
data:
  default: |
    server {
       listen       80;
       server_name  www.mysite.com;
       index        index.html;

       location / {
           root /data/nginx/html;
           if (!-e $request_filename) {
               rewrite ^/(.*) /index.html last;
           }
       }
    }
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"default":"server {\n   listen       80;\n   server_name  www.mysite.com;\n   index        index.html;\n\n   location / {\n       root /data/nginx/html;\n       if (!-e $request_filename) {\n           rewrite ^/(.*) /index.html last;\n       }\n   }\n}\n"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"nginx-config","namespace":"default"}}
  creationTimestamp: "2022-10-20T08:29:50Z"
  name: nginx-config
  namespace: default
  resourceVersion: "388823"
  uid: 1a04f3c2-bc33-4ddc-ac0a-f726c9fa33f6
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# 
root@easzlab-deploy:~# kubectl get svc
NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
kubernetes      ClusterIP   10.100.0.1       <none>        443/TCP          47h
mysql-service   NodePort    10.100.125.186   <none>        3306:33306/TCP   41h
ng-deploy-80    NodePort    10.100.80.101    <none>        81:30019/TCP     2m16s
root@easzlab-deploy:~#

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# cat deploy_configmapenv.yml #带value值

apiVersion: v1
kind: ConfigMap

metadata:
  name: nginx-config
data:
  username: user1

---
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ng-deploy-80
  template:
    metadata:
      labels:
        app: ng-deploy-80
    spec:
      containers:
      - name: ng-deploy-80
        image: nginx 
        env:
        - name: "magedu"
          value: "n70"
        - name: MY_USERNAME
          valueFrom:
            configMapKeyRef:
              name: nginx-config
              key: username
        ports:
        - containerPort: 80

安装并验证

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl apply -f deploy_configmapenv.yml 
configmap/nginx-config configured
deployment.apps/nginx-deployment configured
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6# kubectl get configmap  -oyaml
apiVersion: v1
items:
- apiVersion: v1
  data:
    root-cert.pem: |
      -----BEGIN CERTIFICATE-----
      MIIC/DCCAeSgAwIBAgIQOeHImLiidfxNM+2MuCKFMDANBgkqhkiG9w0BAQsFADAY
      MRYwFAYDVQQKEw1jbHVzdGVyLmxvY2FsMB4XDTIyMTAxODE2MjIzN1oXDTMyMTAx
      NTE2MjIzN1owGDEWMBQGA1UEChMNY2x1c3Rlci5sb2NhbDCCASIwDQYJKoZIhvcN
      AQEBBQADggEPADCCAQoCggEBALJL3P9+3f3SnYE8fFuitxosDPobOAkTy4kuGIMq
      68SzumFalYz5LjlBQpTfo0Hv/OXWWctiJuUm/oJs4jVLhruALQ1JjV5EK82iiwQo
      KypBaUHL1ql5AHBMKmmwqLSo/yd/zNqmU/iwasVN7G/ykAfqaapEvFbnJJhJT0Dz
      0amhRs/oPB1umgfwmiRYrCTZu9iKihBaYjbkmJ6o4/oUCw1Pse1PZLt4MkctTSiZ
      WXvtTF9YyQCqSAe62mVQkmYRBjf4x7QkmfZnvCnHvhJ86RfTOcIMYK8l5xgiaZyG
      1EUrOfMgJ/DQFdC7DKzIbbktTJ2YvA33VTb9gpIQKrCAHhECAwEAAaNCMEAwDgYD
      VR0PAQH/BAQDAgIEMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFA2bWsIMmCNm
      cgQJFjZrUwtYWf0gMA0GCSqGSIb3DQEBCwUAA4IBAQCIVbuVBrRigwzrF08/v2yc
      qhjunL/QrLh6nzRmfHlKn4dNlKMczReMc0yrxcl6V6rdzXpDpVb663Q36hhmmvwe
      WwmnJMZUUsFrYiTt1KYQg9o0dNcRFzYx/W9Dpi9YPwmS2Xqqc94rUDIkBMIOGnc9
      H99gvMOJbfK5BnzXko3A+dCVwUngdmxQpRePjzWSDhU1pWkyZp+hKxZff/1ieFqF
      Joh3bHInmEsWqZRWRhkmzwwjnlvVy3h90TKUizidYfXPz4xgXf/FVp++0mp09U4T
      tnFjivOFyXH/jwpRbZJq8uXsV+joxMEYy/JPbgywYoynvwejcEHksact/3FTQLd5
      -----END CERTIFICATE-----
  kind: ConfigMap
  metadata:
    creationTimestamp: "2022-10-18T16:22:39Z"
    labels:
      istio.io/config: "true"
    name: istio-ca-root-cert
    namespace: default
    resourceVersion: "65285"
    uid: 76575e18-c8b2-4dd9-b1d7-ffef0f43c640
- apiVersion: v1
  data:
    ca.crt: |
      -----BEGIN CERTIFICATE-----
      MIIDlDCCAnygAwIBAgIUXgL7CLqvFf9DxZvFt+UAzbLlYMUwDQYJKoZIhvcNAQEL
      BQAwYTELMAkGA1UEBhMCQ04xETAPBgNVBAgTCEhhbmdaaG91MQswCQYDVQQHEwJY
      UzEMMAoGA1UEChMDazhzMQ8wDQYDVQQLEwZTeXN0ZW0xEzARBgNVBAMTCmt1YmVy
      bmV0ZXMwIBcNMjIxMDEzMTIyMTAwWhgPMjEyMjA5MTkxMjIxMDBaMGExCzAJBgNV
      BAYTAkNOMREwDwYDVQQIEwhIYW5nWmhvdTELMAkGA1UEBxMCWFMxDDAKBgNVBAoT
      A2s4czEPMA0GA1UECxMGU3lzdGVtMRMwEQYDVQQDEwprdWJlcm5ldGVzMIIBIjAN
      BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApw+3h+j5I/2exVVSvxL/j70XZ5ep
      XW5tclKag7Qf/x5oZe8O1yMxZXiPKgzqGGS68morpG5vD2hVPEsqICOhHiFl2AD3
      ZgMCDWMGeOyk6zGgDbnTUsFO7R/v7kNTnBV6BqgKKlG9NqTtrDSPLoeakTB2qBtV
      Wjhv+YrXXsMVcEaiuEQ4wLD87Kmy8r7xRtEttELKHwdI8iS4Caq+qxtm/EosyTiT
      bQbUB4mkGZ6sFFwKSKaLUGz8Nq1yHkJYbI77YDhUBnaNEQBemPmEfkBeHCajbzx1
      CKPIairrAZNaoMPK9stuK+YLk9Z/gLUYrZe2S8S+k6DPlvuj327bLwKWCwIDAQAB
      o0IwQDAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQU
      XUwALoYNGxfIG/8BrPlezZd3uaQwDQYJKoZIhvcNAQELBQADggEBAIhIDLiS0R1M
      bq2RZMQROrEzKs02CclxYjwcr8hrXm/YlB6a8bHG2v3HASi+7QZ89+agz/Oeo+Cp
      6abDTiXHolUkUuyddd14KBwanC7ubwDBsqxr4iteNz5H4ml1uxaZ8G94uVyBgC2U
      qjkWGtXbw6RuY+YTuqYzX3S621U+hwLWN1cXmRcydDZwnMuI+rCwEKLXqLESDMbG
      jiQ1sbLI12oQa07fe+rffnGAWe7P2fMAu/MQxm9Mm8+pX+2WgKauDwpG/v2oZxAO
      iQqICEaYBecgLRBTj868LHVli1CnqUDVjJt59vD2/LZ8I5WnqnGFfONluYSgFiFQ
      m/7XupOph3k=
      -----END CERTIFICATE-----
  kind: ConfigMap
  metadata:
    annotations:
      kubernetes.io/description: Contains a CA bundle that can be used to verify the
        kube-apiserver when using internal endpoints such as the internal service
        IP or kubernetes.default.svc. No other usage is guaranteed across distributions
        of Kubernetes clusters.
    creationTimestamp: "2022-10-18T09:07:42Z"
    name: kube-root-ca.crt
    namespace: default
    resourceVersion: "271"
    uid: f63b2e93-d94f-4c2c-831f-49863f82e3e5
- apiVersion: v1
  data:
    username: user1
  kind: ConfigMap
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"v1","data":{"username":"user1"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"nginx-config","namespace":"default"}}
    creationTimestamp: "2022-10-20T08:37:17Z"
    name: nginx-config
    namespace: default
    resourceVersion: "390419"
    uid: 0136af36-4a7f-407a-a61c-bea7ef19497c
kind: List
metadata:
  resourceVersion: ""
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case6#

七、Kubernetes之Secret

7.1、Secret简介

  • Secret 的功能类似于 ConfigMap给pod提供额外的配置信息，但是Secret是一种包含少量敏感信息例如密码、 令牌或密钥的对象。
  • Secret 的名称必须是合法的 DNS 子域名。
  • 每个Secret的大小最多为1MiB， 主要是为了避免用户创建非常大的Secret进而导致API服务器和kubelet内存耗尽， 不过创建很多小的Secret也可能耗尽内存， 可以使用资源配额来约束每个名字空间中Secret的个数。
  • 在通过yaml文件创建secret时， 可以设置data或stringData字段，data和stringData字段都是可选的， data字段中所有键值都必须是base64编码的字符串， 如果不希望执行这种 base64字符串的转换操作， 也可以选择设置stringData字段， 其中可以使用任何非加密的字符串作为其取值。

Pod 可以用三种方式的任意一种来使用 Secret：

  • 作为挂载到一个或多个容器上的卷 中的文件(crt文件、 key文件)。
  • 作为容器的环境变量。
  • 由 kubelet 在为 Pod 拉取镜像时使用(与镜像仓库的认证)。

7.2、Secret简介类型

Kubernetes默认支持多种不同类型的secret， 用于一不同的使用场景， 不同类型的secret的配置参数也不一样。

7.3、Secret类型-Opaque格式

Opaque格式-data类型数据-事先使用base64加密

#echo admin |base64
#echo 123456 |base64

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# echo admin |base64
YWRtaW4K
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# echo 123456 |base64
MTIzNDU2Cg==
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# cat 1-secret-Opaque-data.yaml 
apiVersion: v1
kind: Secret
metadata:
  name: mysecret-data
  namespace: myserver
type: Opaque
data:
  user: YWRtaW4K
  password: MTIzNDU2Cg==
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# kubectl apply -f 1-secret-Opaque-data.yaml 
secret/mysecret-data created
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# kubectl get secrets mysecret-data -n myserver -o yaml
apiVersion: v1
data:
  password: MTIzNDU2Cg==
  user: YWRtaW4K
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"password":"MTIzNDU2Cg==","user":"YWRtaW4K"},"kind":"Secret","metadata":{"annotations":{},"name":"mysecret-data","namespace":"myserver"},"type":"Opaque"}
  creationTimestamp: "2022-10-20T09:03:33Z"
  name: mysecret-data
  namespace: myserver
  resourceVersion: "394995"
  uid: b0788df4-0195-429f-bda5-eafb5d51bd6a
type: Opaque
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8#

Opaque格式stringData类型数据-不用事先加密-上传到k8s会加密

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# cat 2-secret-Opaque-stringData.yaml 
apiVersion: v1
kind: Secret
metadata:
  name: mysecret-stringdata
  namespace: myserver
type: Opaque
stringData:
  superuser: 'admin'
  password: '123456'
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# kubectl apply -f 2-secret-Opaque-stringData.yaml 
secret/mysecret-stringdata created
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8# kubectl get secrets mysecret-stringdata -n myserver -o yaml
apiVersion: v1
data:
  password: MTIzNDU2
  superuser: YWRtaW4=
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Secret","metadata":{"annotations":{},"name":"mysecret-stringdata","namespace":"myserver"},"stringData":{"password":"123456","superuser":"admin"},"type":"Opaque"}
  creationTimestamp: "2022-10-20T09:07:15Z"
  name: mysecret-stringdata
  namespace: myserver
  resourceVersion: "395636"
  uid: 4134fe69-389d-47d0-b870-f83dd34fa537
type: Opaque
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case8#

7.4、Secret类型-kubernetes.io/tls-为nginx提供证书

自签名证书：

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# mkdir certs
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# ls
4-secret-tls.yaml  certs
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# cd certs/
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# ls
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 3560 -nodes -subj '/CN=www.ca.com'
Generating a RSA private key
..............................................++++
....................................................................++++
writing new private key to 'ca.key'
-----
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=www.mysite.com'
Generating a RSA private key
.......................................................................................................................................................................................++++
................................................++++
writing new private key to 'server.key'
-----
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# openssl x509 -req -sha256 -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
Signature ok
subject=CN = www.mysite.com
Getting CA Private Key
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# ll -h 
total 28K
drwxr-xr-x 2 root root 4.0K Oct 20 20:09 ./
drwxr-xr-x 3 root root 4.0K Oct 20 20:06 ../
-rw-r--r-- 1 root root 1.8K Oct 20 20:08 ca.crt
-rw------- 1 root root 3.2K Oct 20 20:08 ca.key
-rw-r--r-- 1 root root 1.7K Oct 20 20:09 server.crt
-rw-r--r-- 1 root root 1.6K Oct 20 20:09 server.csr
-rw------- 1 root root 3.2K Oct 20 20:09 server.key
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# kubectl create secret tls myserver-tls-key --cert=./server.crt --key=./server.key -n myserver
secret/myserver-tls-key created
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9/certs#

创建web服务nginx并使用证书：

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# cat 4-secret-tls.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
  namespace: myserver
data:
  default: |
     server {
        listen  80;
        listen  443 ssl;
        server_name www.mysite.com;
        ssl_certificate /etc/nginx/conf.d/certs/tls.crt;
        ssl_certificate_key /etc/nginx/conf.d/certs/tls.key;

        location / {
            root /usr/share/nginx/html;
            index index.html;
            if ($scheme = http){
               rewrite / https://www.mysite.com permanent;
            }

            if (!-e $request_filename){
                rewrite ^/(.*) /index.html last;
            }
         }
      }

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myserver-myapp-frontend-deployment
  namespace: myserver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myserver-myapp-frontend
  template:
    metadata:
      labels:
        app: myserver-myapp-frontend
    spec:
      containers:
      - name: myserver-myapp-frontend
        image: nginx:1.20.2-alpine
        ports:
          - containerPort: 80
        volumeMounts:
          - name: nginx-config
            mountPath: /etc/nginx/conf.d/myserver
          - name: myserver-tls-key
            mountPath: /etc/nginx/conf.d/certs
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-config
          items:
             - key: default
               path: mysite.conf
      - name: myserver-tls-key
        secret:
          secretName: myserver-tls-key

---
apiVersion: v1
kind: Service
metadata:
  name: myserver-myapp-frontend
  namespace: myserver
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    targetPort: 80
    nodePort: 30018
    protocol: TCP
  - name: https
    port: 443
    targetPort: 443
    nodePort: 30019
    protocol: TCP
  selector:
    app: myserver-myapp-frontend
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl apply -f 4-secret-tls.yaml 
configmap/nginx-config created
deployment.apps/myserver-myapp-frontend-deployment created
service/myserver-myapp-frontend created
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9#

 验证nginx pod信息

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl get pod -n myserver
NAME                                                  READY   STATUS    RESTARTS   AGE
myserver-myapp-frontend-deployment-7694cb4fcb-j9hcq   1/1     Running   0          54m
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl get secret -n myserver
NAME               TYPE                DATA   AGE
myserver-tls-key   kubernetes.io/tls   2      71m
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl get secret -n myserver -oyaml
apiVersion: v1
items:
- apiVersion: v1
  data:
    tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVvakNDQW9vQ0FRRXdEUVlKS29aSWh2Y05BUUVMQlFBd0ZURVRNQkVHQTFVRUF3d0tkM2QzTG1OaExtTnYKYlRBZUZ3MHlNakV3TWpBeE1qQTVNakJhRncwek1qRXdNVGN4TWpBNU1qQmFNQmt4RnpBVkJnTlZCQU1NRG5kMwpkeTV0ZVhOcGRHVXVZMjl0TUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUF5RE9VCmt5UHJMbC9adFRLMk1ZOWxQWU8zQXVXRnExcEJFZG9PT0R2dndtZ3FabUV1VjVNeXNTdnJLcmJBeW1pQnd1d1gKd2JYZU1MZGlxOW1GK0NzTWFkR21tUW9aY0VXNW54MGZZSzNLbVdIY2hrc0JITnlKNnhXV2ZCQk0yRDlzSnM1cQpXVWJnbUFDdGNINW1iSXo5TlV3MGwwZ2pTa3oyNDM2RVNPNjdPOFo5WEVEVGlFUTN5YnM0RTV2azNiVGJ3ZzBYCmthK2Q5Z3RCQTFmQmZFOGFEZkRweWhkZTZ5L0YxTjBlWmladFlNdUp1QTIzYkVxcWR5d1hJemJCUDBjTHdyUEIKUG5vTXY5OUdTWTVzZEZZMkRrdHMxVHdUUjRqdHVWdExTTTY1MXNXeVpVckVUcm53U3RNcXJRT1Q0QUdpcXc1MQpHZTZoQXJxZ0Y1VS96U3BiL29nQjZ1T0IvWThiNDBvazFNNGRKWXRETVhvZFFtYnNtOU9pa3VOc2lRWTVIZUlnCmVpVUdnVVRoRFRGYlB6SW5acWo3TDM5bmMvUlREeWxicGRsRVRNamVTS0o5MHBpMERNM1VOVkxzTEg0WitrWnEKZ282N0hneHFCVlBYb1dTTm54UFFheEI5TkFnTUl3aVVXZ0NoM2pHVlpwMTE5VGpsQXRqYTV0OHZVeFhCWDdUMApkSDVCYUZjTTRGTmwzYmYzMmJRck9vcWlkREFMQ2JrcVZmNjNxNUVsUERNV2p0UGdhd0JtUlZ0V0Vlb2hIV0t3CitKTFVod1o4UUNPYjhBcTYwVHk3bFdvc0JyUlZZRWgwOTJjbFU4clJmOFF1VC9venl5N3BkWHlYVzNYUGZDbHAKVXo1NGF5eVdYOVZXa3pxZHRIby8xRXRtUHpCV1FzcnhSeVNjOTg4Q0F3RUFBVEFOQmdrcWhraUc5dzBCQVFzRgpBQU9DQWdFQXRFUkdPbENCQnFmRTZzODdGcmFReXc5Q0pocUs4TndrajFVa3YwMzFsaFIwakhKSUE1TU1lYlFNCkU2YnBPdm5mS1pOWk82OXJtaVV6S3pJd2IrZjV5THZaWHJzVzdYUU5VQitiRlFJRG14cVp1dGJNbHR5YjdrcWIKaTA1eVpqUWpJNHNSYVFrUnRpZ1JBdkFWRFk4RWl3WSsvb1BFUjg4N0QrbXk2ZlZJZFhFTzNSQUloT1FhNWF1bQphV1o3bVBjL2xkd1ZoNFVicG0yTGZCNDhvb3BvS05pZ1hsZWloNWg5VWc2Mms3NHFLdVB2cnVvdVBvWWtoWGlXCmFuQzBtTXFWalk5bFMzSi9CdXpKdFpwUExlcllLdjJIQ1RiSklYdmhCazNQU3MwbmZlUFoyUDkrNHMzQXhMcSsKVGxhMHlwcXZueWtHMXRyd0g4dXMzZEM1ZEg3ZWNzWC9SRm8wM2NXbUNGZUM2Yzk4YjhiVDBXd0x6Ym1HWUl6bgpWaVk3UTRVSTNya2wrdjBySC91aDZ5OVFkU1FRZS9vaERCeEJtelQxWXVqdDMyMUpiSTIzMklrTEFQSUpWbDBnClo5SktFaCtSRko0djVRK1N6U1BSZyt5ZWJCQjExZUVvc1l3N2lQd2J4a0U1UVpaUzBLY1N4TDB3UGF3R3NXK0MKYkFmZUFpMVhFdG51MFlVMzlOTHkrOWluRFBEcjlyM3Y2N1d0UkxFeldWSzcwS05RUTY1R3VTOUsxbHFMdzdqUApwWE1RclpuQnpDelVDb0VmYkFIN0tmSXd3OGgxWFZhTnJFeXZDWnJRRlNiTjJTUzFwYjZidXFSVmtjald2U2Y3Clp0ejUzYXBDQkFXd3JxUGFSbW1VVHd0RnpZZUIvVmVsNEdJVzlEbkIwRTF1NWdXaHpaQT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUpRd0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQ1Mwd2dna3BBZ0VBQW9JQ0FRRElNNVNUSStzdVg5bTEKTXJZeGoyVTlnN2NDNVlXcldrRVIyZzQ0TysvQ2FDcG1ZUzVYa3pLeEsrc3F0c0RLYUlIQzdCZkJ0ZDR3dDJLcgoyWVg0S3d4cDBhYVpDaGx3UmJtZkhSOWdyY3FaWWR5R1N3RWMzSW5yRlpaOEVFellQMndtem1wWlJ1Q1lBSzF3CmZtWnNqUDAxVERTWFNDTktUUGJqZm9SSTdyczd4bjFjUU5PSVJEZkp1emdUbStUZHROdkNEUmVScjUzMkMwRUQKVjhGOFR4b044T25LRjE3ckw4WFUzUjVtSm0xZ3k0bTREYmRzU3FwM0xCY2pOc0UvUnd2Q3M4RStlZ3kvMzBaSgpqbXgwVmpZT1MyelZQQk5IaU8yNVcwdEl6cm5XeGJKbFNzUk91ZkJLMHlxdEE1UGdBYUtyRG5VWjdxRUN1cUFYCmxUL05LbHYraUFIcTQ0SDlqeHZqU2lUVXpoMGxpME14ZWgxQ1p1eWIwNktTNDJ5SkJqa2Q0aUI2SlFhQlJPRU4KTVZzL01pZG1xUHN2ZjJkejlGTVBLVnVsMlVSTXlONUlvbjNTbUxRTXpkUTFVdXdzZmhuNlJtcUNqcnNlREdvRgpVOWVoWkkyZkU5QnJFSDAwQ0F3akNKUmFBS0hlTVpWbW5YWDFPT1VDMk5ybTN5OVRGY0ZmdFBSMGZrRm9Wd3pnClUyWGR0L2ZadENzNmlxSjBNQXNKdVNwVi9yZXJrU1U4TXhhTzArQnJBR1pGVzFZUjZpRWRZckQ0a3RTSEJueEEKSTV2d0NyclJQTHVWYWl3R3RGVmdTSFQzWnlWVHl0Ri94QzVQK2pQTEx1bDFmSmRiZGM5OEtXbFRQbmhyTEpaZgoxVmFUT3AyMGVqL1VTMlkvTUZaQ3l2RkhKSnozendJREFRQUJBb0lDQVFEQ21WaW0rYmdGdU1ldW1KOStad3NhCmt5aFdXWEhuMEhBRmdUWm5OT05semNqQkFWK0JZcVJZa1A4aTRzZGROOTVCOFNsYWNvU0tTQWRTVWJzbU1mbjcKOWZ5Qkw4N3dVZVlQSXNpNE9kWC81NTdxcm9kalhYOTJFZUxYcnlSeTRwc20wV2VRWmhPenpKektCeU5hQ21XcAo0K3dPek9ENHZQMFN2b3lwTTl5dFNzL1oxMjJHUEFFYVJyQklaelU4eUNzQVlhZHlSZ2s5KzB4emlsNlpqVzRlCjlQamJKb0p1QzE2NS9VRXFPOW4veDNpVGZrbTNxcEF1REo1azdUbEVYN092eXZoZzJWUUJRVzlaMm1YVFkyVmgKMmJEdFNGclpJdUVvVmZSRXppVFgvZ3pjNXFNUWZ5NXlIUGFUZkRIR0FQRDBZcll5d2NDaUhYTzEySzVPcUFrSQpGV0FIUnZYQTNmMHo3TGtWNDQ1OXg4aFE4WDZFSHV5Ykx6REY5dGprT1ZUUGJ4Q0poS1FMRWZlVStQbi83ZjB5CkVteXpmODRNWU9BWHNpbk1TakVsaUZnWHFrYVFLNXdUZDhaN1R4STcxSjJ1UXBMV2VyTXlBb3BKc0FDYWJjZFcKcEVXUEJhdDZHZ0FnM1NGQUE1SUFGZk9BMFdWbUxuL3UzUjdMekM2dklucUtSYW1qZUlKY0paeWkranlEVzNrQgpzWTd1ZTRMZGYyNC9IMVlCeUhISmpsSnZRWjBWYnVuVWMvZ0c2UFUzNTZ0OUhlaGwwM1lxZXVXUE5ySW9maktECjBlQWFIc3NzY2laeXdKOUFqNUZsTEJIVS9xeUhWL1RjZTQxNEQxN2NuMit3azloUmJITUo2RFh5WFdORFFWZXAKbHBKaHUyS1hoQmNZZVFrb3pEZkJRUUtDQVFFQTQyKzhsdE83eXRld3Y1OHVwWkdBa3BJKy84NElqWWZtaGhoYgpHMlRXU0FpQmxseWxTU1YzejNtd2hzRE4yTWY1QXU4L3d2ZnpxZ3g5OHhhUW9pQjBsbnBDTzJ6OXFaSkE5YVc1CnpTQkdQS095YkhBNkpZQ0ZZSjBrUHNiVlRzY2IzTVhvSUEveHg2Wkg0QWtreGtPYTBMQmpJMW11anNVRlI3akUKMmhweUVUenZPRlNXaUNpSnA2RzFDeXBzRWozdzVQR2dGb0lCdFpwblVCM1ZDVXViNEhLVTkzT3pvaXhad05mVQpTaGdYbHZqOW5OWkdpN1NJRzJxY2xvOTI5ZlRESnV6bnUvdjBndlJnRytwbUxPSHZjM09EMzBOTzA4alhQbkNjCnJzU2sxTHVCQkNyektzUjl0RHNHTEtyWW9McFlCMUxYcTdJQXNGRFU5aUN1UlBwaW9RS0NBUUVBNFZnM25tdDkKNUFWL3NnaUx5aGtBeU5BSVUvTWFvR1RiUVgzL0hyUnlsWlRQL2d0ZWZ6WDB5Rm1lVWlRUTJHYlp1MUY0c1pFaAo2WmZWaGlOMkJsSFkyNkkwblpjQkw5Q3hDM0NNZTY3eUNmdUhDekNNN0Q4R3JieHhLV2duSWxHT1hrcFhyMzdYClg2aDBKSzV3VjlJaVlLYXVaZ2xUWm1vT2g0aTl5M252Um5ESkFrREIwMzlNYjBVUjVaaTAwOElrbUw3bDBsU2MKL0lJenBGajJTeHIvUWUySVBLYkpTWDBjWW44am5yamVZUzBjczNaKzJNMVRDVTRZVU5rTnVMVFV2ZFBPWnBNRApaUmx1MWRLbElmZDMrb0lZWkhhNmxLVDlDeitlYmdQS0Jxb0tsa1hJM0RNTldGWTZhSlF4Y0N6RkkwZStKWmVVCld4Uk96WU94Wk5PMGJ3S0NBUUVBbVVTaWZhNGdmcmpPRnNScVJnK0E2c1Y5aVJ2S3JiNG92ck5KS25QUTUrZzcKbEIzSkVUc2J1NGpSU201Q0NsWHczR1pvdkxZbDBiSHJhdGNKRHdqNktMSXBVaXpINE85N3NVOUdvQktnNHBxYQpVZk5yYS94cFpjdGdNcUlCKzcyNGJCWStzT1N0MWhLYm0wSHVNMkk1d1dzczFCVEt5dEhCRmkxUkUzNEE0dGNDCml4Nk45eUlDYWlKU2hEekphWjJ1YWtyZXpHdytSS2pSK0s2eDh6cXR5QnJQZ3RiSTlvQVcyQnRhcDdnR3Bhb1UKRnc1YnFpZzJGT3ZLckxmdnZoNTlLUTA3dVhZNHQ4dUJ2UzVBUHZ6ZlJobFJoREt5dTR3OGFZcXdQQ0t1eGVHNgpOeG5PbDBLbFI4RUREelR2R1ptYVd3MGI1RXZucE9wRUtiMnFVemU5SVFLQ0FRQmhzeHU2SmFTWlBnRVZNVHRhClRlalhKOHJVaXV3YWFsL2RUMEZURUswMVNTVzhZVFdCTmVXQkQ4bmlseHh1bG5rRUM5aW1NK1JlSUtSRTJnOEwKd21TaEpQeG03dGRtNGJaQTNYVXJFcmlCdDNuZlVoZG5QaFFwTXpCazRYRkdJZEgxODRsODN5T0ZwOFZqT2ZZZgpQVTRHVlgzN1kwT3pmWHY3SzBBT2ZqbE5jd3pUV3p3dDlGMHhTT0x2aG51djY5WnVHeVlOUVA0blJGUWJoeTZSCmRZMENDbmdzdzZzMW4zYTFCYVp0NUgwVjZMY3UzOHN6T0NJdVFKdXVRY3ovTGZlbXJiUXBLTWdxQnhMVXhkVXUKbXRwNzAvZTdadmFTQjg1bUdCa2FYYTR6b1htaG1YUHlkSGZ1dXNQc0g0UW52R0ZrWUhDQ1grdkVhVk9aS3VXNApiMGtsQW9JQkFDNmVZdlhzYUVITW5CUUQ5cDdXaGVNZitmOEtHMlAwcFFEYnFmM0x4bGtMWlIxV3l1SnMyOSttCkgrQm15OEM5blJDcGpzT0VJT3pCTW9seFdlN3F2aUhDeGsreG9SdkNFVlZvNklMd3gyQU0xV3MvTnJBTEE5Q0QKd1QyTjBQdkdnR01jZmIwS3RMeGtJbXVDaW1nSEdnak5hWkJhNjYxeHpWNVh1cnZNTndHaUw1R2lwMlA1R1pUUwpQSEdkamg5SFVTQUtibkNqcG9CL2Z5MHBiNk9YRkJHT1JvNVkvcFV4cHYrQ3JHdEQreHkrS2UzcWd6UjIrdkQxCmNnNmU2Vk1jWHVGUk45YUl5UHdpZHZJL2hwTFdNNGtiZjNlOFJ6ZGRjWUlBQjlwZ2E3dDFyWmVJVFJtNUVqMlIKd1BZRTg3b3hRWVdTNmorUjBSWWNIb2pIK0lPZWhaMD0KLS0tLS1FTkQgUFJJVkFURSBLRVktLS0tLQo=
  kind: Secret
  metadata:
    creationTimestamp: "2022-10-20T12:09:46Z"
    name: myserver-tls-key
    namespace: myserver
    resourceVersion: "427221"
    uid: cef4b425-8572-44f2-9097-5a1040c9bd03
  type: kubernetes.io/tls
kind: List
metadata:
  resourceVersion: ""
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9#

此时发现pod没有监听443端口

解决办法

root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl exec -it -n myserver myserver-myapp-frontend-deployment-7694cb4fcb-l449j sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # 
/etc/nginx/conf.d/myserver # vi /etc/nginx/nginx.conf 
/etc/nginx/conf.d/myserver # cat /etc/nginx/nginx.conf 

user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/conf.d/myserver/*.conf;
}
/etc/nginx/conf.d/myserver # ls /etc/nginx/conf.d/myserver/*.conf
/etc/nginx/conf.d/myserver/mysite.conf
/etc/nginx/conf.d/myserver # 
/etc/nginx/conf.d/myserver # nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
/etc/nginx/conf.d/myserver # nginx -s reload
2022/10/20 13:56:20 [notice] 52#52: signal process started
/etc/nginx/conf.d/myserver # netstat -tnlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      1/nginx: master pro
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1/nginx: master pro
tcp        0      0 :::80                   :::*                    LISTEN      1/nginx: master pro
/etc/nginx/conf.d/myserver #

配置负载均衡转发请求到nodeport

root@easzlab-haproxy-keepalive-01:~# vi /etc/haproxy/haproxy.cfg

listen  myserer-nginx-80
        bind 172.16.88.200:80
        mode tcp
        server easzlab-k8s-master-01 172.16.88.157:30018 check inter 2000 fall 3 rise 5
        server easzlab-k8s-master-02 172.16.88.158:30018 check inter 2000 fall 3 rise 5
        server easzlab-k8s-master-03 172.16.88.159:30018 check inter 2000 fall 3 rise 5

listen  myserer-nginx-443
        bind 172.16.88.200:443
        mode tcp
        server easzlab-k8s-master-01 172.16.88.157:30019 check inter 2000 fall 3 rise 5
        server easzlab-k8s-master-02 172.16.88.158:30019 check inter 2000 fall 3 rise 5
        server easzlab-k8s-master-03 172.16.88.159:30019 check inter 2000 fall 3 rise 5

root@easzlab-haproxy-keepalive-01:~# systemctl restart haproxy

 配置hosts 解析

通过curl命令查看证书来源

root@easzlab-haproxy-keepalive-01:~# curl -lvk https://www.mysite.com
*   Trying 172.16.88.200:443...
* TCP_NODELAY set
* Connected to www.mysite.com (172.16.88.200) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=www.mysite.com
*  start date: Oct 20 12:09:20 2022 GMT
*  expire date: Oct 17 12:09:20 2032 GMT
*  issuer: CN=www.ca.com
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> GET / HTTP/1.1
> Host: www.mysite.com
> User-Agent: curl/7.68.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.20.2
< Date: Thu, 20 Oct 2022 14:06:47 GMT
< Content-Type: text/html
< Content-Length: 612
< Last-Modified: Tue, 16 Nov 2021 15:04:23 GMT
< Connection: keep-alive
< ETag: "6193c877-264"
< Accept-Ranges: bytes
< 
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
* Connection #0 to host www.mysite.com left intact
root@easzlab-haproxy-keepalive-01:~#
root@easzlab-haproxy-keepalive-01:~# curl -vvi https://www.mysite.com
*   Trying 172.16.88.200:443...
* TCP_NODELAY set
* Connected to www.mysite.com (172.16.88.200) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
root@easzlab-haproxy-keepalive-01:~#

7.5、Secret-kubernetes.io/dockerconfigjson类型

存储docker registry的认证信息， 在下载镜像的时候使用， 这样每一个node节点就可以不登录也可以下载私有级别的镜像了。

root@easzlab-deploy:~# docker login --username=c******2 registry.cn-shenzhen.aliyuncs.com
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
root@easzlab-deploy:~# cat /root/.docker/config.json 
{
    "auths": {
        "harbor.magedu.net": {
            "auth": "YWRtaW46SGFyYm9yMTIzNDU="
        },
        "registry.cn-shenzhen.aliyuncs.com": {
            "auth": "Y*********************=="  #此处这里显示脱敏
        }
    }
}
root@easzlab-deploy:~#
root@easzlab-deploy:~# kubectl create secret generic aliyun-registry-image-pull-key \
> --from-file=.dockerconfigjson=/root/.docker/config.json \
> --type=kubernetes.io/dockerconfigjson \
> -n myserver   #将本地登录阿里云私有仓库信息存储起来，共享给k8s集群节点使用
secret/aliyun-registry-image-pull-key created
root@easzlab-deploy:~# 
root@easzlab-deploy:~# kubectl get secret -n myserver
NAME                             TYPE                             DATA   AGE
aliyun-registry-image-pull-key   kubernetes.io/dockerconfigjson   1      9m24s
myserver-tls-key                 kubernetes.io/tls                2      150m
root@easzlab-deploy:~#
root@easzlab-deploy:~# kubectl get secret -n myserver aliyun-registry-image-pull-key -oyaml
apiVersion: v1
data:
  .dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3IubWFnZWR1Lm5ldCI6IHsKCQkJImF1dGgiOiAiWVdSdGFXNDZTR0Z5WW05eU1USXpORFU9IgoJCX0sCgkJInJlZ2lzdHJ5LmNuLXNoZW56aGVuLmFsaXl1bmNzLmNvbSI6IHsKCQkJImF1d*************************n0=
kind: Secret
metadata:
  creationTimestamp: "2022-10-20T14:30:23Z"
  name: aliyun-registry-image-pull-key
  namespace: myserver
  resourceVersion: "451590"
  uid: f084175a-6260-4435-acfb-bcec9095e5a6
type: kubernetes.io/dockerconfigjson
root@easzlab-deploy:~#
root@easzlab-deploy:~# cd jiege-k8s/pod-test/case-yaml/case9/
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# vi 6-secret-imagePull.yaml
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# cat 6-secret-imagePull.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myserver-myapp-frontend-deployment-2
  namespace: myserver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myserver-myapp-frontend-2
  template:
    metadata:
      labels:
        app: myserver-myapp-frontend-2
    spec:
      containers:
      - name: myserver-myapp-frontend-2
        image: registry.cn-shenzhen.aliyuncs.com/cyh01/nginx:1.22.0 #指向阿里云公有私仓镜像
        ports:
          - containerPort: 80
      imagePullSecrets:
        - name: aliyun-registry-image-pull-key

---
apiVersion: v1
kind: Service
metadata:
  name: myserver-myapp-frontend-2
  namespace: myserver
spec:
  ports:
  - name: http
    port: 80
    targetPort: 80
    nodePort: 30033
    protocol: TCP
  type: NodePort
  selector:
    app: myserver-myapp-frontend-2
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl apply -f 6-secret-imagePull.yaml 
deployment.apps/myserver-myapp-frontend-deployment-2 created
service/myserver-myapp-frontend-2 created
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# 
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl get pod -n myserver -owide
NAME                                                   READY   STATUS    RESTARTS   AGE   IP               NODE            NOMINATED NODE   READINESS GATES
myserver-myapp-frontend-deployment-2-6d96b76bb-bgmzf   1/1     Running   0          30s   10.200.104.226   172.16.88.163   <none>           <none>
myserver-myapp-frontend-deployment-6f48755cbd-k2dbs    1/1     Running   0          28m   10.200.105.158   172.16.88.164   <none>           <none>
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# 
#验证pod信息
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9# kubectl describe pod -n myserver myserver-myapp-frontend-deployment-2-6d96b76bb-bgmzf 
Name:         myserver-myapp-frontend-deployment-2-6d96b76bb-bgmzf
Namespace:    myserver
Priority:     0
Node:         172.16.88.163/172.16.88.163
Start Time:   Thu, 20 Oct 2022 23:01:25 +0800
Labels:       app=myserver-myapp-frontend-2
              pod-template-hash=6d96b76bb
Annotations:  <none>
Status:       Running
IP:           10.200.104.226
IPs:
  IP:           10.200.104.226
Controlled By:  ReplicaSet/myserver-myapp-frontend-deployment-2-6d96b76bb
Containers:
  myserver-myapp-frontend-2:
    Container ID:   containerd://20d2061b0eaa8e21748fed2559ba0fe35e7271730097809f210e50d650ad20f9
    Image:          registry.cn-shenzhen.aliyuncs.com/cyh01/nginx:1.22.0
    Image ID:       registry.cn-shenzhen.aliyuncs.com/cyh01/nginx@sha256:b3a676a9145dc005062d5e79b92d90574fb3bf2396f4913dc1732f9065f55c4b
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 20 Oct 2022 23:01:27 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-j7wtn (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-j7wtn:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  105s  default-scheduler  Successfully assigned myserver/myserver-myapp-frontend-deployment-2-6d96b76bb-bgmzf to 172.16.88.163
  Normal  Pulled     103s  kubelet            Container image "registry.cn-shenzhen.aliyuncs.com/cyh01/nginx:1.22.0" already present on machine
  Normal  Created    103s  kubelet            Created container myserver-myapp-frontend-2
  Normal  Started    103s  kubelet            Started container myserver-myapp-frontend-2
root@easzlab-deploy:~/jiege-k8s/pod-test/case-yaml/case9#

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/785255.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

【雕爷学编程】Arduino动手做（85）---LCD1602液晶屏模块7

【雕爷学编程】Arduino动手做（85）---LCD1602液晶屏模块7

37款传感器与执行器的提法&#xff0c;在网络上广泛流传&#xff0c;其实Arduino能够兼容的传感器模块肯定是不止这37种的。鉴于本人手头积累了一些传感器和执行器模块&#xff0c;依照实践出真知&#xff08;一定要动手做&#xff09;的理念&#xff0c;以学习和交流为目的&am…
阅读更多...
js路由跳转时放弃正在pending的请求

js路由跳转时放弃正在pending的请求

在单页面应用中通常会对请求进行catch处理&#xff0c;如果用户打开a页面后页面发出了一个请求去获取aaa&#xff0c;但是由于某种原因请求一直在pending。此时用户又进入了b页面&#xff0c;在浏览时a页面的请求失败了&#xff0c;然后页面弹出提示&#xff1a;“数据aaa请求失…
阅读更多...
pearcmd.php文件包含妙用

pearcmd.php文件包含妙用

文章目录 pearcmd.php文件包含妙用利用条件原理利用config-createinstalldownload pearcmd关键词被ban参考 pearcmd.php文件包含妙用 利用条件 php.ini中register_argc_argvOn开启安装pecl/pear pecl是PHP中用于管理扩展而使用的命令行工具&#xff0c;而pear是pecl依赖的类…
阅读更多...
【stable diffusion】保姆级入门课程05-Stable diffusion（SD）图生图-涂鸦重绘的用法

【stable diffusion】保姆级入门课程05-Stable diffusion（SD）图生图-涂鸦重绘的用法

1.什么是涂鸦重绘 涂鸦重绘又称手涂蒙版。 简单来说&#xff0c;局部重绘手涂蒙版 就是涂鸦局部重绘的结合体&#xff0c;这个功能的出现是为了解决用户不想改变整张图片的情况下&#xff0c;对多个元素进行修改。 功能支持&#xff1a; 1.支持蒙版功能 2.笔刷决定绘制的元素…
阅读更多...
如何在win10环境下配置强化学习gym库（使用vscode）

如何在win10环境下配置强化学习gym库（使用vscode）

我是通过anacondavscode完成的gym库的使用&#xff0c;只是把案例跑起来了&#xff0c;具体步骤如下&#xff1a; 1、安装anaconda,参考链接&#xff1a;https://www.jianshu.com/p/2f3be7781451 我其实就是生安装的&#xff0c;也没有去配置环境啥的&#xff0c;就是下载安…
阅读更多...
贪心算法重点内容

贪心算法重点内容

贪心算法重点内容 4.1部分背包 按照单位重量的价值排序 4.2最小生成树 两种算法 4.3单源最短路径 4.4哈夫曼树
阅读更多...
Vector3.Dot

Vector3.Dot

如图&#xff1a;cube在cube&#xff08;2&#xff09;的前方、 public class DotTest : MonoBehaviour {public GameObject a;public GameObject b;// Start is called before the first frame updatevoid Start(){var vVector3.Dot(a.transform.forward,b.transform.position…
阅读更多...
丑数 II 力扣（动态规划）JAVA

丑数 II 力扣（动态规划）JAVA

给你一个整数 n &#xff0c;请你找出并返回第 n 个 丑数 。 丑数 就是只包含质因数 2、3 和/或 5 的正整数。 示例 1&#xff1a; 输入&#xff1a;n 10 输出&#xff1a;12 解释&#xff1a;[1, 2, 3, 4, 5, 6, 8, 9, 10, 12] 是由前 10 个丑数组成的序列。 示例 2&#xff…
阅读更多...
MySQL笔记——MySQL数据库介绍以及在Linux里面安装MySQL数据库，对MySQL数据库的简单操作，MySQL的外接应用程序使用说明

MySQL笔记——MySQL数据库介绍以及在Linux里面安装MySQL数据库，对MySQL数据库的简单操作，MySQL的外接应用程序使用说明

文章目录 前言 一 MySQL数据库 1.1 什么是数据库&#xff1f; 1.2 RDBMS 术语 1.3 DBMS解释 1.4 数据库分类 1.5 MySQL现状 二 在Linux/UNIX 上安装 MySQL 2.1 安装 MySQL 2.2 验证 MySQL 安装 2.3 使用 MySQL Client(Mysql客户端) 执行简单的SQL命令 2.4 Mysql安装…
阅读更多...
153、仿真-基于51单片机四相步进电机正反转控制系统设计(程序+Proteus仿真+参考论文+流程图+配套资料等)

153、仿真-基于51单片机四相步进电机正反转控制系统设计(程序+Proteus仿真+参考论文+流程图+配套资料等)

毕设帮助、开题指导、技术解答(有偿)见文未 目录 一、设计功能 二、Proteus仿真图​ 三、程序源码 资料包括&#xff1a; 需要完整的资料可以点击下面的名片加下我&#xff0c;找我要资源压缩包的百度网盘下载地址及提取码。 方案选择 单片机的选择 方案一&#xff1a;ST…
阅读更多...
代码随想录算法训练营一刷总结

代码随想录算法训练营一刷总结

第一次系统地刷leetcode 经过两个多月终于一刷完结 虽然不是按部就班每天都写 但总算坚持下来了 夸夸自己&#xff01;&#xff01; 非常感谢卡哥 题目的解析都写的很清楚易懂 才让我能够坚持这么一题一题做下来 没有卡住不前 训练营的学习氛围让我感受到很多小伙伴在和我一起…
阅读更多...
服务器数据库中了Locked勒索病毒，企业应该如何正确处理并采取后续防护措施

服务器数据库中了Locked勒索病毒，企业应该如何正确处理并采取后续防护措施

网络技术的发展极大地方便了人们的工作生活&#xff0c;但同样带来了一定的网络安全威胁&#xff0c;其中较为危险的威胁就是勒索病毒攻击&#xff0c;勒索病毒不仅会给我们的计算机系统带来破坏&#xff0c;还会加密我们的重要文件数据来敲诈勒索&#xff0c;只有用户支付的赎…
阅读更多...
灾备基础学习

灾备基础学习

灾备 灾备&#xff1a;灾备是容灾和备份的简称&#xff0c;它是利用科学的技术手段和方法&#xff0c;提前建立系统化的数据应急方式&#xff0c;以应对灾难的发生。其内容包括&#xff1a;数据备份、系统备份、业务连续规划、人员架构、通信保障、危机公关、灾难恢复规划、灾…
阅读更多...
ES6基础知识五：你是怎么理解ES6新增Set、Map两种数据结构的？

ES6基础知识五：你是怎么理解ES6新增Set、Map两种数据结构的？

如果要用一句来描述&#xff0c;我们可以说 Set是一种叫做集合的数据结构&#xff0c;Map是一种叫做字典的数据结构 什么是集合&#xff1f;什么又是字典&#xff1f; 集合 是由一堆无序的、相关联的&#xff0c;且不重复的内存结构【数学中称为元素】组成的组合 字典 是…
阅读更多...
unity的asmdef报错GUID（这个报错记得重现一下，报错信息作为标题记录一下）

unity的asmdef报错GUID（这个报错记得重现一下，报错信息作为标题记录一下）

这应该是unity给这个程序集分配的guid unity基于2020开发的科骏插件1.0.8&#xff0c;放到了2017.首先出现的这个报错。 解决方法是将这个asmdef删除。asmdef简单来说是 库的描述信息文件。相关参考 初步理解Unity的asmdef 删掉就没分库直接所有源码在工程。简单来说就是编译…
阅读更多...
深度学习：Pytorch最全面学习率调整策略lr_scheduler

深度学习：Pytorch最全面学习率调整策略lr_scheduler

深度学习&#xff1a;Pytorch最全面学习率调整策略lr_scheduler lr_scheduler.LambdaLRlr_scheduler.MultiplicativeLRlr_scheduler.StepLRlr_scheduler.MultiStepLRlr_scheduler.ConstantLRlr_scheduler.LinearLRlr_scheduler.ExponentialLRlr_scheduler.PolynomialLRlr_sched…
阅读更多...
Nginx 301 https跳转后出现跨域和混合内容问题 —— 筑梦之路

Nginx 301 https跳转后出现跨域和混合内容问题 —— 筑梦之路

问题 在浏览器地址栏敲入url访问静态资源目录时&#xff0c;发现默认跳转到了http协议的地址 如上图所示&#xff0c;客户端https请求先到达API网关&#xff0c;然后网关将请求通过http协议转发到静态资源服务器。 调出浏览器发现客户端发送的https请求收到了一个301状态码的响…
阅读更多...
WPF实现DiagramChart

WPF实现DiagramChart

1、文件架构 2、FlowChartStencils.xaml <ResourceDictionary xmlns"http://schemas.microsoft.com/winfx/2006/xaml/presentation"xmlns:x"http://schemas.microsoft.com/winfx/2006/xaml"xmlns:s"clr-namespace:DiagramDesigner"xmlns:c&…
阅读更多...
数据结构【数组、串、广义表】

数据结构【数组、串、广义表】

第四章 数组、串、广义表 一、数组 1.概念&#xff1a;线性表是通过数组实现的&#xff0c;数组是线性表的推广&#xff0c;数组只有存取元素和修改元素的操作&#xff08;除了初始化和销毁&#xff09;&#xff1b; 2.数组的存储结构&#xff1a;一个数组的所有元素在内存中占…
阅读更多...
第八章：list类

第八章：list类

系列文章目录 文章目录 系列文章目录前言list的介绍及使用list的介绍list的使用list的构造函数list的迭代器list的容量list的成员访问list的增删改查 list与vector的对比总结 前言 list是STL的一种链表类&#xff0c;可以在常数范围内在任意位置进行插入和删除的序列式容器。 …
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023