K8s-Ingress Nginx-Day 08

1. 什么是Ingress

官方文档：https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/#what-is-ingress

Ingress 是 kubernetes API 中的标准资源类型之一，主要是k8s官方在维护。

2. Ingress的作用

Ingress 提供从集群外部到集群内服务的 HTTP 和 HTTPS 路由。流量路由由 Ingress 资源所定义的规则来控制。
ingress 实现的功能是在应用层对客户端请求的 host 名称或请求的 URL 路径把请求转发到指定的 service 资源的规则，即用
于将 kubernetes 集群外部的请求资源转发之集群内部的 service，再被 service 转发之 pod
处理客户端的请求。
说简单点就是另一种对外暴露服务的方式。
Ingress本身其实就是nginx二次开发的一个东西。

3. Ingress-controller

为了让 Ingress 资源工作，集群必须有一个正在运行的 Ingress 控制器。
Ingress 资源需要指定监听地址、请求的 host 和 URL 等配置，然后根据这些规则的匹配机制将客户端的请求进行转发，这种能够为 ingress 配置资源监听并转发流量的组件称为ingress 控制器(ingress controller)，ingress controller 是 kubernetes 的一个附件，类似于dashboard 或者 flannel 一样，需要单独部署。

4. 其他类型的Ingres控制器

https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress-controllers/#additional-controllers

5. 部署Ingress Nginx

官方文档：https://kubernetes.github.io/ingress-nginx/deploy/
官方文档：https://github.com/kubernetes/ingress-nginx/blob/main/docs/deploy/index.md
可以把官方提供的这个yaml下载下面，改改部署也可以
在这里插入图片描述

5.1 部署注意事项

官方文档：https://github.com/kubernetes/ingress-nginx?tab=readme-ov-file#supported-versions-table
部署Ingress-Nginx的时候一定要注意和k8s版本兼容性的问题
在这里插入图片描述

5.2 部署方式

Ingress-nginx有2种部署方式，一个是deployment，一个是demonset，区别：
（1）deployment方式部署
默认只有一个ingress控制器，在被调度到的节点上监听默认的80和443端口，如果有请求是从其他节点过来的，需要先经过路由转发才能到我ingress控制器所在的这个节点，就增加了一个转发的性能开销。

（2）demonset方式部署
在每个k8s节点上都部署一个ingress控制器，同时也在每个节点上都监听了80和443端口，该方式性能更好。

5.3 部署Ingress Nginx v1.3.0

因为我本地也无法从外网拉取docker镜像，所以改用了改好的1.3.0，我的集群是1.24.0的，刚好1.3.0能用
在这里插入图片描述

在这里插入图片描述

5.3.1 yaml配置

5.3.1.1 deployment

[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# cat 1.ingress-nginx-controller-v1.3.0_deployment.yaml
apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resourceNames:
  - ingress-controller-leader
  resources:
  - configmaps
  verbs:
  - get
  - update
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
- apiGroups:
  - coordination.k8s.io
  resourceNames:
  - ingress-controller-leader
  resources:
  - leases
  verbs:
  - get
  - update
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
    nodePort: 50080
  - appProtocol: http #kubernetes v1.20 stable,appProtocol字段提供了一种为每个Service端口指定应用协议的方式,此字段的取值会被映射到对应的Endpoints
    name: prometheus-metrics-port
    port: 10254
    protocol: TCP
    targetPort: 10254 #ingress-nginx-controller内置的指标数据采集端口
    nodePort: 50254
#  - name: metrics-port
#    port: 10254
#    targetPort: 10254
#    nodePort: 50254
#    protocol: TCP
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
    nodePort: 50443
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --election-id=ingress-controller-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/ingress-nginx-controller:v1.3.0
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          runAsUser: 101
        volumeMounts:
        - mountPath: /usr/local/certificates/
          name: webhook-cert
          readOnly: true
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
      - name: webhook-cert
        secret:
          secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.3.0
      name: ingress-nginx-admission-create
    spec:
      containers:
      - args:
        - create
        - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
        - --namespace=$(POD_NAMESPACE)
        - --secret-name=ingress-nginx-admission
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
        imagePullPolicy: IfNotPresent
        name: create
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.3.0
      name: ingress-nginx-admission-patch
    spec:
      containers:
      - args:
        - patch
        - --webhook-name=ingress-nginx-admission
        - --namespace=$(POD_NAMESPACE)
        - --patch-mutating=false
        - --secret-name=ingress-nginx-admission
        - --patch-failure-policy=Fail
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
        imagePullPolicy: IfNotPresent
        name: patch
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None

5.3.1.2 demonset

[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# cat 2.ingress-nginx-controller-v1.3.0_daemonset.yaml
apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resourceNames:
  - ingress-controller-leader
  resources:
  - configmaps
  verbs:
  - get
  - update
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
- apiGroups:
  - coordination.k8s.io
  resourceNames:
  - ingress-controller-leader
  resources:
  - leases
  verbs:
  - get
  - update
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
    nodePort: 30080
  - appProtocol: http #kubernetes v1.20 stable,appProtocol字段提供了一种为每个Service端口指定应用协议的方式,此字段的取值会被映射到对应的Endpoints
    name: prometheus-metrics-port
    port: 10254
    protocol: TCP
    targetPort: 10254 #ingress-nginx-controller内置的指标数据采集端口
    nodePort: 30254
#  - name: metrics-port
#    port: 10254
#    targetPort: 10254
#    nodePort: 50254
#    protocol: TCP
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
    nodePort: 30443
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
    spec:
      #hostNetwork: true #使用宿主机网络。使用宿主机网络一定要修改dns策略为ClusterFirstWithHostNet，否则ingress控制器的默认dns策略ClusterFirst会因为pod使用了hostnetwork而退回default模式，从而直接使用宿主机dns配置。
      #hostPID: true #使用宿主机Pid
      containers:
      - args:
        - /nginx-ingress-controller
        - --election-id=ingress-controller-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/ingress-nginx-controller:v1.3.0
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          runAsUser: 101
        volumeMounts:
        - mountPath: /usr/local/certificates/
          name: webhook-cert
          readOnly: true
      dnsPolicy: ClusterFirst
      tolerations:
      - key: "key1"
        operator: "Equal"
        value: "value1"
        effect: "NoSchedule"
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
      - name: webhook-cert
        secret:
          secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.3.0
      name: ingress-nginx-admission-create
    spec:
      containers:
      - args:
        - create
        - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
        - --namespace=$(POD_NAMESPACE)
        - --secret-name=ingress-nginx-admission
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
        imagePullPolicy: IfNotPresent
        name: create
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.3.0
      name: ingress-nginx-admission-patch
    spec:
      containers:
      - args:
        - patch
        - --webhook-name=ingress-nginx-admission
        - --namespace=$(POD_NAMESPACE)
        - --patch-mutating=false
        - --secret-name=ingress-nginx-admission
        - --patch-failure-policy=Fail
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: registry.cn-hangzhou.aliyuncs.com/zhangshijie/kube-webhook-certgen:v1.3.0
        imagePullPolicy: IfNotPresent
        name: patch
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None

在这里插入图片描述

5.3.2 demonset方式部署ingress nginx 控制器

[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# kubectl apply -f 2.ingress-nginx-controller-v1.3.0_daemonset.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
daemonset.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

[root@master1 1.1-ingress-nginx-1.3.0_deploy-yaml]# kubectl get po,svc -n ingress-nginx
NAME                                       READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-bbxkb   0/1     Completed   0          26m
pod/ingress-nginx-admission-patch-z7zqf    0/1     Completed   0          26m
pod/ingress-nginx-controller-5ch6s         1/1     Running     0          26m
pod/ingress-nginx-controller-d9mtz         1/1     Running     0          26m

NAME                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                      AGE
service/ingress-nginx-controller             NodePort    10.200.238.16    <none>        80:30080/TCP,10254:30254/TCP,443:30443/TCP   26m
service/ingress-nginx-controller-admission   ClusterIP   10.200.118.113   <none>        443/TCP                                      22m

在这里插入图片描述

6. 基于域名实现单host及多host的ingress

6.1 部署测试用的web服务

[root@master1 Ingress-case-20230611-backup]# cat tomcat-app1.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    app: tomcat-app1-deployment-label
  name: tomcat-app1-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat-app1-selector
  template:
    metadata:
      labels:
        app: tomcat-app1-selector
    spec:
      containers:
      - name: tomcat-app1-container
        image: tomcat:7.0.94-alpine
        #command: ["/apps/tomcat/bin/run_tomcat.sh"]
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
        env:
        - name: "password"
          value: "123456"
        - name: "age"
          value: "18"
        resources:
          limits:
            cpu: 1
            memory: "512Mi"
          requests:
            cpu: 500m
            memory: "512Mi"
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: tomcat-app1-service-label
  name: tomcat-app1-service
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: tomcat-app1-selector

[root@master1 Ingress-case-20230611-backup]# kubectl apply -f tomcat-app1.yaml
deployment.apps/tomcat-app1-deployment created
service/tomcat-app1-service created

[root@master1 Ingress-case-20230611-backup]# cat tomcat-app2.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  labels:
    app: tomcat-app2-deployment-label
  name: tomcat-app2-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: tomcat-app2-selector
  template:
    metadata:
      labels:
        app: tomcat-app2-selector
    spec:
      containers:
      - name: tomcat-app2-container
        image: tomcat:7.0.94-alpine
        #command: ["/apps/tomcat/bin/run_tomcat.sh"]
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
          protocol: TCP
          name: http
        env:
        - name: "password"
          value: "123456"
        - name: "age"
          value: "18"
        resources:
          limits:
            cpu: 1
            memory: "512Mi"
          requests:
            cpu: 500m
            memory: "512Mi"
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: tomcat-app2-service-label
  name: tomcat-app2-service
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: tomcat-app2-selector

[root@master1 Ingress-case-20230611-backup]# kubectl apply -f tomcat-app2.yaml
deployment.apps/tomcat-app2-deployment created
service/tomcat-app2-service created

[root@master1 Ingress-case-20230611-backup]# kubectl get po |grep tomcat
tomcat-app1-deployment-749f777dbf-mz9ws   1/1     Running   0          4m10s
tomcat-app2-deployment-6999ff4c89-brmrp   1/1     Running   0          2m48s

6.2 环境初始化

6.2.1 tomcat-1

[root@master1 Ingress-case-20230611-backup]# kubectl exec -it tomcat-app1-deployment-749f777dbf-mz9ws -- /bin/bash
bash-4.4# pwd
/usr/local/tomcat/webapps
bash-4.4# mkdir app1
bash-4.4# echo 'pc web test page' > app1/index.jsp
bash-4.4# exit

6.2.2 tomcat-2

[root@master1 Ingress-case-20230611-backup]# kubectl exec -it tomcat-app2-deployment-6999ff4c89-brmrp -- /bin/bash
bash-4.4# pwd
/usr/local/tomcat/webapps
bash-4.4# mkdir app2
bash-4.4# echo 'mobile app test page' > app2/index.jsp
bash-4.4# exit

6.3 配置单域名访问的ingress规则

6.3.1 配置ingress规则

[root@master1 Ingress-case-20230611-backup]# cat 2.1.ingress_single-mobile.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mobile.test.com # 用域名作为ingress名称，方便识别
  namespace: default # 注意nginx规则要和后端pod在同一个ns下
  annotations: # 注解，类似于指令，相当于配置nginx服务的一些功能
    kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型
    nginx.ingress.kubernetes.io/use-regex: "true" ##指定后面rules定义的path可以使用正则表达式
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" ##连接超时时间,默认为5s
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600" # 设置代理服务器向后端服务器发送数据的超时时间，单位为秒，默认60s，如果超时Ingress控制器将终止连接并返回适当的错误。
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600" ##读取超时时间,默认为60s
    # 上面的三个proxy超时时间，建议设置长一点时间。
    nginx.ingress.kubernetes.io/proxy-body-size: "50m" ##客户端上传文件，最大大小，默认为20m
    #nginx.ingress.kubernetes.io/rewrite-target: / ##URL重写
    nginx.ingress.kubernetes.io/app-root: /index.html # 定义上下文中 / 必须重定向的应用程序根目录
spec:
  rules: # 路由规则
  - host: mobile.test.com # 客户端访问的host域名
    http:
      paths:
      - pathType: Prefix # 前缀匹配，且区分大小写。另外还有Exact精确匹配和ImplementationSpecific。
        path: "/" # 匹配路径，类似nginx的location
        backend: # 后端配置
          service:
            name: tomcat-app2-service # 后端pod的service名称
            port:
              number: 80 # service端口

[root@master1 Ingress-case-20230611-backup]# kubectl apply -f  2.1.ingress_single-mobile.yaml
ingress.networking.k8s.io/mobile.test.com created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME              CLASS    HOSTS             ADDRESS   PORTS   AGE
mobile.test.com   <none>   mobile.test.com             80      3s

6.3.2 访问测试

在这里插入图片描述

6.4 配置多域名访问的ingress规则

6.4.1 配置ingress规则

# 清理环境
[root@master1 Ingress-case-20230611-backup]# kubectl delete -f 2.1.ingress_single-mobile.yaml

# 配置并应用新的ingress规则
[root@master1 Ingress-case-20230611-backup]# cat 2.2.ingress_multi-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-web
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
    #nginx.ingress.kubernetes.io/rewrite-target: / ##URL重写
    nginx.ingress.kubernetes.io/app-root: /index.html
spec:
  rules:
  - host: pc.test.com # 域名一
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: tomcat-app1-service
            port:
              number: 80


  - host: mobile.test.com # 域名二
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: tomcat-app2-service
            port:

[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 2.2.ingress_multi-host.yaml
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME        CLASS    HOSTS                         ADDRESS                       PORTS   AGE
nginx-web   <none>   pc.test.com,mobile.test.com   192.168.10.11,192.168.10.12   80      15m

6.4.2 访问测试

在这里插入图片描述

6.4.3 清理环境

[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME        CLASS    HOSTS                         ADDRESS                       PORTS   AGE
nginx-web   <none>   pc.test.com,mobile.test.com   192.168.10.11,192.168.10.12   80      15m

[root@master1 Ingress-case-20230611-backup]# kubectl delete ingress nginx-web
ingress.networking.k8s.io "nginx-web" deleted

6.5 配置基于URL访问的ingress规则

就是和nginx location配置差不多的道理，通过不同的URL location匹配并转发到不同的后端service。

6.5.1 ingress规则配置

[root@master1 Ingress-case-20230611-backup]# cat 3.1.ingress-url.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-web
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-body-size: "10m"
    #nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/app-root: /index.html
spec:
  rules:
  - host: pc.test.com
    http:
      paths:
      - pathType: Prefix
        path: "/app1"
        backend:
          service:
            name: tomcat-app1-service
            port:
              number: 80

      - pathType: Prefix
        path: "/app2"
        backend:
          service:
            name: tomcat-app2-service
            port:
              number: 80

[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 3.1.ingress-url.yaml
ingress.networking.k8s.io/nginx-web created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME        CLASS    HOSTS         ADDRESS   PORTS   AGE
nginx-web   <none>   pc.test.com             80      3s

6.5.2 访问测试

在这里插入图片描述

6.5.3 清理环境

[root@master1 Ingress-case-20230611-backup]# kubectl delete -f 3.1.ingress-url.yaml
ingress.networking.k8s.io "nginx-web" deleted

7. 单域名和多域名的https证书配置

7.1 单域名https证书配置

7.1.1 自签证书

7.1.1.1 签发ca证书

[root@master1 ~]# mkdir certs
[root@master1 ~]# cd certs
[root@master1 certs]# openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 3560 -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
...........++
..............................................................................++
writing new private key to 'ca.key'
-----
[root@master1 certs]# ll
total 8
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key

7.1.1.2 签发域名证书-公钥和csr

[root@master1 certs]# openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
............................................................................................................................................................................................................++
.........................................................................................................................................++
writing new private key to 'server.key'
-----
[root@master1 certs]# ll -rt
total 16
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:38 server.key
-rw-r--r-- 1 root root 1586 Feb 19 10:38 server.csr
[root@master1 certs]#

7.1.1.3 签发域名证书-私钥

[root@master1 certs]# openssl x509 -req -sha256 -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
Signature ok
subject=/CN=pc.test.com
Getting CA Private Key
[root@master1 certs]# ll -rt
total 20
-rw-r--r-- 1 root root 3272 Feb 19 10:37 ca.key
-rw-r--r-- 1 root root 1793 Feb 19 10:37 ca.crt
-rw-r--r-- 1 root root 3272 Feb 19 10:38 server.key
-rw-r--r-- 1 root root 1586 Feb 19 10:38 server.csr
-rw-r--r-- 1 root root 1663 Feb 19 10:40 server.crt

7.1.2 上传证书到k8s集群中

[root@master1 certs]# kubectl create secret tls tls-secret-pc --cert=server.crt --key=server.key -n default
secret/tls-secret-pc created
[root@master1 certs]# kubectl get secret # 证书上传完毕后，内容就会通过base64加密
NAME            TYPE                DATA   AGE
tls-secret-pc   kubernetes.io/tls   2      5s

7.1.3 配置ingress规则

[root@master1 Ingress-case-20230611-backup]# cat 4.1.ingress-https-magedu_single-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-web
  annotations:
    kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型
    nginx.ingress.kubernetes.io/ssl-redirect: 'true' #SSL重定向，即将http请求强制重定向至https，等于nginx中的全站https
spec:
  tls:
  - hosts:
    - pc.test.com
    secretName: tls-secret-pc
  rules:
  - host: pc.test.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: tomcat-app1-service
            port:
              number: 80

[root@master1 Ingress-case-20230611-backup]# kubectl apply -f 4.1.ingress-https-magedu_single-host.yaml
ingress.networking.k8s.io/nginx-web created
[root@master1 Ingress-case-20230611-backup]# kubectl get ingress
NAME        CLASS    HOSTS         ADDRESS   PORTS     AGE
nginx-web   <none>   pc.test.com             80, 443   5s

7.1.4 访问测试

在这里插入图片描述

7.2 多域名https证书配置

这里只展示配置，其他的不再演示

7.2.1 配置ingress规则

[root@master1 Ingress-case-20230611-backup]# cat 4.2.ingress-https-magedu_multi-host.yaml
#apiVersion: networking.k8s.io/v1beta1
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-web-mobile
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:
  tls: # 就是在tls下配置不同的hosts就行了，指定不同的secret证书
  - hosts:
    - mobile.test.com
    secretName: tls-secret-mobile
  - hosts:
    - pc.test.com
    secretName: tls-secret-pc
  rules:
  - host: pc.test.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: tomcat-app1-service
            port:
              number: 80


  - host: mobile.test.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: tomcat-app2-service
            port:
              number: 80

8. ingress 证书更新案例

8.1 确认老证书的过期时间

在这里插入图片描述

8.2 签发新证书

实际工作中，使用商业证书都是不需要自己手动签发的。

[root@master1 certs-new]# openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=pc.test.com'
Generating a 4096 bit RSA private key
......................................................................................................................++
.........++
writing new private key to 'server.key'
-----
[root@master1 certs-new]# ls
server.csr  server.key
[root@master1 certs-new]# openssl x509 -req -sha256 -days 36500 -in server.csr -CA ../certs/ca.crt -CAkey ../certs/ca.key -set_serial 01 -out server.crt # 这里还是基于之前的ca签发新证书，这里的证书有效期我配置了100年 -days 36500
Signature ok
subject=/CN=pc.test.com
Getting CA Private Key
[root@master1 certs-new]# ll -rt
total 12
-rw-r--r-- 1 root root 3268 Feb 19 11:40 server.key
-rw-r--r-- 1 root root 1586 Feb 19 11:40 server.csr
-rw-r--r-- 1 root root 1667 Feb 19 11:41 server.crt
[root@master1 certs-new]#

8.3 base64加密新证书内容

8.3.1 加密公钥.crt文件内容

在这里插入图片描述

# 加密后的内容：
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUVvakNDQW9vQ0FRRXdEUVlKS29aSWh2Y05BUUVMQlFBd0ZqRVVNQklHQTFVRUF3d0xjR011ZEdWemRDNWoKYjIwd0lCY05NalF3TWpFNU1ETTBNVE0xV2hnUE1qRXlOREF4TWpZd016UXhNelZhTUJZeEZEQVNCZ05WQkFNTQpDM0JqTG5SbGMzUXVZMjl0TUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUFzRlFuClovOXlLOVZBYnZiNVQ0bm5oSTNBMXUzMEVnYzFadlZRS1Q3L29uMFpWYnVZTFVNdDc3bHJNMjRRaUtGS0JjaXIKYjFrT0xJNDAvV1lmdVBUMDVIK2J5MlFxR3Zkb2dOWXBza2E5SWsxcnM1bzNVVjltNDBNSnhSOE9IOEJoUThNLwpzZVZGMWp5SXEwb0ZLUHFFSDVkMnhZTXdMekMrT3ArNGNNZUsxU3hoNlNNbm82TXhhcU00aGJlTU5NbmFPY1BjCkVHT3FqZHVMbmtPRXd4UDFmWDZrQkpEaW5WditKYUJwWG8zdHJJaFg4RWIxNGpqL21qemFON1pObm43d3grVGcKTHl3cUhEZWdHVmoxays4V2hqOUdXNWhXL3F1ZXM1L3lDeUdDR1ZYODNxN3NvRWxjR0ovRU8yRnZPTWJQTUJRdwplZjdvTkU5WFhzdXZMZVVMUXNOZ3FvR3RicEpaTlFxLzh1QnBIbUhsNjljQmtuVkVnSlJZMC9pK3VEaGFpQ1lWCkJPZURVb3hFOFZQSWlMMWJ2KzhIZ2lWVm5LL3lmazhsU3J1RldnbzlkTkhDTWFEQ014cE5KYTA5N2xYQ1c5N1cKMGNUNEF6WGdXL3B0VlR6ZktWZUd4VCtGem9jUGlZbk1EeXEwcXVTMXNYeWpjK0xZTjZWWmRPcGRwZzVZczg3Tgp5V25EVUl1OWg4NldRbnJMZDd1ZTY1VU9EWmJjb2QzTVVoMUVNcFNwTHE3bEZwWTdZRmVONklBRWRobnRzWVhECkFzU3ZWdVNSL1hkRThZYVNFYXNhMEYrWVczZWlGQzNobEFzZDZVVG5UK0E4Z09zemhpQVorRE43NDROOVVVdEgKeWlKYXVkTlVxRXA1VjE3V1gzN25oZWJ1SzdaUlRMOWRmcThYSzk4Q0F3RUFBVEFOQmdrcWhraUc5dzBCQVFzRgpBQU9DQWdFQVVzR09qUG5Sb2NOaFRqUjNoa1dQYmNDT3AwenJDbndEY1Y4ZG11QzVGbm5PTUdJc2hZelV5ZHJUCjZsOEFmRlhibUNtZ2xvbjFxUEpPR3lMT3A5YlRRdlQ3bDY4elRSb0p5R1JxdVZCRGoxb1JCUUI3TER3ZUN4THQKU2NGR0JFdi9wWFc3NnVWbXovZTVRRmlIY0pna1F1bmsyR0JvZUUwM2lLR3ZabW9PcnM3b2VMRW1XM3prb2JxbgpiTlV1MFMraHRJcHlnZEYrM1pjL0RVTmsyaWEwS3JFVnVlMWdDL0YveldiNVR5eDBQVEhnNlIwKzRKeERMRlF0Cnh0N3IrNXVQeUg0QVlsWFNwenNnM3VvL1BmNkZtOHA0VlJDcGNCYUlaUCtPRlZzSzN2SFhyODJuNHdXOU8vRCsKNFJ1cnpXWnRGK0hmNHJVY3VpN2gwbmk5d0pUUUVUZDJuTEZxcHM0ZENuNCs1L1FtY3JVUDdkNlk0OExEYktLawp3UWd2NzB1aUp6alhRNjRKa1dMSDhWVTRRdEpGQSs1QkFraHFaRFZEWXoxMEV3QlJvRWNuWFA5MXpaUyszOTVICkZFNjB0Vkx0YmVnSktxb0pUSTJ0WThyWWNhSVBWbjNESi9KbTdLTWpXZGwxK3BKNEhVNzgzRW9sRzMyYkZiNTUKZm9LQzE2T0tkQ2x4ZTZrOCtyQWxBRnV1Z1UxZzBFVXVycjNHeERtbjBaNG5hZEFKcmYvK1p6NXZXUndPWEwwSgpGRFhUTkRVRE9qcnIxTlJ1bHVTOERROEVIZTM4N1dtM2ExMSttOUZVYUhpdHI4U21DZG5lK2RzQkswNUxVVEtoCmJEL3pOQzI3SzJIQUxmaUlrRmZZTFN4TGl1Q1c3QXpoQ3oyeHp2UVdUS0RQRGRaZ1BCMD0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQ==

8.3.2 加密私钥.key文件内容

在这里插入图片描述

8.3 更新证书

在线更新secrets内容是实时生效的，但是生产环境变更时，一定不要在线编辑，先导出旧的secrets yaml到本地，备份后，在文件中修改应用，这样就算出了问题，也可以第一时间恢复正常。
但是由于secrets创建后，无法通过apply -f 更新内容，所以先导出旧的内容备份，然后edit在线编辑实时生效。
或者备份好后，先delete 旧的secrets，再apply -f创建新的，但是这种方式没有第一种稳妥。

8.3.1 修改对应证书secrets的内容

[root@master1 certs-new]# kubectl get secrets
NAME            TYPE                DATA   AGE
tls-secret-pc   kubernetes.io/tls   2      73m
[root@master1 certs-new]# kubectl edit secrets tls-secret-pc

在这里插入图片描述

8.3.2 验证证书更新结果

在这里插入图片描述

9. 配置ingress控制器日志格式为json

参考：https://github.com/zhangshijle/ingress-files/blob/main/2.ingress-configmap.yaml

[root@master1 Ingress-case-20230611-backup]# kubectl get cm -n ingress-nginx|grep ingress
ingress-controller-leader   0      18h
ingress-nginx-controller    1      18h

# 调整内容展示
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  annotations:
    # 下面这个注解的作用记录了最后一次应用的配置信息。
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/part-of":"ingress-nginx"},"name":"nginx-configuration","namespace":"ingress-nginx"}}
data:
  use-forwarded-headers: "true" # 启用使用转发的HTTP头信息。
  compute-full-forwarded-for: "true" # 启用计算完整的转发的HTTP头中的"X-Forwarded-For"字段。
  use-proxy-protocol: 'True' # 启用使用代理协议（Proxy Protocol），但是要后端支持4层才行。
  real-ip-header: proxy_protocol # 指定真实IP的头信息名称为"proxy_protocol"。
  set-real-ip-from: 172.26.0.0/16 # 公有云的话要修改为当前kubernetes环境的当前使用的VPC的CIDR格式的网络信息（掩码），然后日志里面会显示是从哪个网段转发过来的请求。私有云不用加。
  # 日志格式
  log-format-upstream: '{"nginx_timestamp":"$time_iso8601","tcp_xff":"$proxy_protocol_addr","clientip":"$remote_addr","nginx_host":"$server_addr","host":"$http_host","request":"$request","url":"$request_uri","upstreamhost":"$upstream_addr","status":"$status","body_bytes_sent":"$body_bytes_sent","request_time":"$request_time","upstream_response_time":"$upstream_response_time","xff":"$http_x_forwarded_for","referer":"$http_referer","http_user_agent":"$http_user_agent","request_length":"$request_length","request_method":"$request_method"}'