1. 说明

1.1 关于PHP+Nginx体系的WebApp，这里将实践两种部署模式：

1.2 配置清单

2. PHP+Nginx体系的WebApp部署

2.1 单节点多容器模式A

2.1.1 准备工作

2.2.2 部署

2.2.3 访问效果

编辑

2.2 多节点单容器模式B

2.2.1 准备工作

2.2 配置虚拟主机

3. Java Springboot 虚拟主机

4. 会话Session保持设置

5. 遇到的问题

6. 参考

1. 说明

- 根据之前的k8s基础，我打算设置两种不同的虚拟主机运行在这个K8s集群上面，一个是PHP+Nginx体系的WebApp，一个是Java SpringBoot体系的WebApp。

1.1 关于PHP+Nginx体系的WebApp，这里将实践两种部署模式：

单节点多容器模式 A - 即跨容器的设计模式，目的是在单个主机上运行多个具有超亲密关系的容器(进程)，因此容器管理系统需要将其作为一个原子单位进行统一调度。Kubernetes编排系统设计的Pod概念就是该设计模式的实现之一。
多节点单容器协作模式 B - 将分布式应用(eg：nginx, php, mariadb...)的每个实例分布于多个节点，分别以单节点模式运行

1.2 配置清单

- wwwroot 是之前做pvc定义的文件目录，形如：/data0/nfs/iot-age-wwwroot-pvc-202ba85a-fd3f-4817-98ea-6764f9ec0d55

模式	主机域名	体系	指向hosts ip	项目存放路径
A	phpmyadmin.k8s-t2.com	PHP	192.168.0.106	/var/www/app

B	t1.k8s-t1.com	PHP	192.168.0.106	wwwroot/t1
B	t2.k8s-t1.com	PHP	192.168.0.106	wwwroot/t2

2. PHP+Nginx体系的WebApp部署

2.1 单节点多容器模式A

2.1.1 准备工作

以广为人知的phpmyadmin为示范app, 首先用buildkit工具打包成镜像，并上传到 hub.docker.com (这是我的镜像链接 https://hub.docker.com/repository/docker/bennybi/phpmyadmin/general) 。

buildctl build \
    --frontend=dockerfile.v0 \
    --local context=. \
    --local dockerfile=. \
    --export-cache type=inline \
    --output type=image,name=docker.io/bennybi/phpmyadmin:v1,push=true

Dockerfile

# This Dockerfile uses the latest version of the Bitnami PHP-FPM Docker image
FROM alpine:latest

# Copy app's source code to the /app directory
COPY . /app

# The application's directory will be the working directory
WORKDIR /app

2.2.2 部署

依据模式A的思想，模式A是用于快捷方便地部署单一应用，所以我把相关的设置都统一到 phpmyadmin.k8s-t2.com.yaml文件中定义, 其中规则：

项目名或domain需替换 "." 为 "-" 切换为部署名, 如phpmyadmin.k8s-t2.com => phpmyadmin-k8s-t2-com
容器中以刚才上传的镜像为initContainer，部署代码
nginx, php-fpm为容器服务，nginx调用local-php-fpm:9000 解释php运行时逻辑
打包时，需在代码预先定义与数据库服务的连接，如：my-release-mariadb-galera.default，这里没做额外的configmap

# Service
apiVersion: v1
kind: Service
metadata:
  name: phpmyadmin-k8s-t2-com
  namespace: iot-age
spec:
  type: ClusterIP
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 10800
  ports:
  - name: nginx
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: phpmyadmin-k8s-t2-com

---
# deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: phpmyadmin-k8s-t2-com
  namespace: iot-age
spec:
  replicas: 2
  selector:
    matchLabels:
      app: phpmyadmin-k8s-t2-com
  template:
    metadata:
      labels:
        app: phpmyadmin-k8s-t2-com
    spec:
      hostAliases:
      - ip: "127.0.0.1"
        hostnames:
        - "local-php-fpm"
      # 私有docker 镜像仓库
      # imagePullSecrets:
      # - name: registrykey-qlcoud-1
      # 自定义设置POD的hosts
      initContainers:
      - name: app-php
        image: "bennybi/phpmyadmin:v1"
        imagePullPolicy: Always
        # 复制php程序文件到wwwroot volume
        command: ["sh", "-c", "cp -r /app /appdata"]
        volumeMounts:
        - mountPath: /appdata
          name: wwwroot
      containers:
      #php-fpm php运行环境
      - name: php-fpm
        image: "bennybi/php-fpm:v1"
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /var/www
          name: wwwroot
      #webserver
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        ports:
        - containerPort: 80
        volumeMounts:
        - name: wwwroot
          mountPath: /var/www
        - name: nginx-conf
          mountPath: /etc/nginx/conf.d/
      #做一个emptyDir类型，名为wwwroot的volume 用于多个容器共享同一个挂载
      volumes:
      - name: wwwroot
        emptyDir: {}
      - name: nginx-conf
        configMap:
          name: phpmyadmin-k8s-t2-com.nginx-conf
          
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: phpmyadmin-k8s-t2-com.nginx-conf
  namespace: iot-age
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","data":{"default.conf":"server {\n    listen      
      80;\n    listen  [::]:80;\n    server_name  localhost;\n    \n    index
      index.php index.html;\n    error_log  /var/log/nginx/error.log;\n   
      access_log /var/log/nginx/access.log;\n    root /var/www/html;\n   
      location / {\n        try_files $uri $uri/ /index.php?$query_string;\n   
      }\n\n    #location / {\n    #    #root   /usr/share/nginx/html;\n    #   
      root   /var/www/html;\n    #    index  index.php index.html
      index.htm;\n    #}\n    \n    location ~ \\.php$ {\n        try_files $uri
      =404;\n        fastcgi_split_path_info ^(.+\\.php)(/.+)$;\n       
      fastcgi_pass   local-php-fpm:9000;\n        fastcgi_index  index.php;\n       
      include        fastcgi_params;\n        fastcgi_param SCRIPT_FILENAME
      $document_root$fastcgi_script_name;\n        fastcgi_param PATH_INFO
      $fastcgi_path_info;\n    }\n\n    error_page   500 502 503 504 
      /50x.html;\n    location = /50x.html {\n        root  
      /usr/share/nginx/html;\n   
      }\n}\n"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"phpmyadmin-k8s-t2-com.nginx-conf","namespace":"iot-age"}}
data:
  default.conf: |
    server {
        listen       80;
        listen  [::]:80;
        server_name  localhost;
        
        index index.php index.html;
        error_log  /var/log/nginx/error.log;
        access_log /var/log/nginx/access.log;
        root /var/www/app;
        location / {
            try_files $uri $uri/ /index.php?$query_string;
        }

        #location / {
        #    #root   /usr/share/nginx/html;
        #    root   /var/www/html;
        #    index  index.php index.html index.htm;
        #}
        
        location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass   local-php-fpm:9000;
            fastcgi_index  index.php;
            include        fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param PATH_INFO $fastcgi_path_info;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
    }
    
---
# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: phpmyadmin-k8s-t2-com
  namespace: iot-age
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/session-cookie-name: stickounet
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
spec:
  rules:
    - host: phpmyadmin.k8s-t2.com
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: phpmyadmin-k8s-t2-com
              port:
                number: 80
  ingressClassName: nginx

2.2.3 访问效果

2.2 多节点单容器模式B

2.2.1 准备工作

先在本地准备好两个t1,t2文件夹，里面各放置一个index.php

<?
echo "T1 Site is {$_SERVER['HTTP_HOST']} <br>";
phpinfo();

<?
echo "T2 Site is {$_SERVER['HTTP_HOST']} <br>";
phpinfo();

上传到设置表格描述的对应路径

当前容器组截图

2.2 配置虚拟主机

在KubeSphere后台，去配置-》配置字典，在nginx-conf配置项中，点编辑设置

添加数据项，t1.k8s-t1.com.conf & t2.k8s-t1.com.conf

server {
        listen       80;
        listen  [::]:80;
        server_name  t1.k8s-t1.com;
        
        index index.php index.html;
        error_log  /var/log/nginx/t1.error.log;
        access_log /var/log/nginx/t1.access.log;
        root /var/www/html/t1;

        location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|flv|mp4|ico)$ {
          expires 30d;
          access_log off;
        }

        location / {
            try_files $uri $uri/ /index.php?$query_string;
        }
        
        location ~ \.php$ {
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_pass   php:9000;
            fastcgi_index  index.php;
            include        fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param PATH_INFO $fastcgi_path_info;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
    }

完成后，需重启nginx pods, 可分别访问这两个域名验证结果

3. Java Springboot 虚拟主机

... 待补充

4. 会话Session保持设置

这是当使用phpmyadmin时，遇到错误:

Failed to set session cookie. Maybe you are using HTTP instead of HTTPS to access phpMyAdmin.

我很快意识到这是由于访问请求被分配到不同的php pods上处理，而导致的session不同步引起的，解决办法：设置 ingress, nginx service, php service会话保持（即相同client ip的访问分配到相同pod, 有过期时间）

nginx.yaml

...
spec:
  type: ClusterIP
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 10800

php.yaml

...
spec:
  type: ClusterIP
  sessionAffinity: ClientIP
  sessionAffinityConfig:
    clientIP:
      timeoutSeconds: 10800

ingress-nginx.yaml

...
metadata:
  name: ia-web-service1
  namespace: iot-age
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/affinity: cookie
    nginx.ingress.kubernetes.io/session-cookie-name: stickounet
    nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
    nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"