文章目录
- 前言
- 技术积累
- 什么是Helm
- StorageClass
- 使用的工具版本
- helm 安装 MySQL 1主2从
- 1. 添加 bitnami 的仓库
- 2. 查询 MySQL 资源
- 3. 拉取 MySQL chart 到本地
- 4. 对chart 本地 values-test.yaml 修改
- 5. 对本地 templates 模板 修改
- 6. 安装 MySQL 集群
- 7. 查看部署的 MySQL 集群
- 8. 连接 MySQL 集群 验证服务
- ingress-nginx开放主从端口
前言
K8S对于云原生部署有着至关重要的作用，几乎所有的应用和中间件都可以部署在K8S，让其帮助我们进行管理。一般情况下我们推荐数据库不使用容器化部署，但是在某些场合为了方便管理也可以采用该种方式。今天我们就用在K8S集群部署MySQL，仅当作学习学习，在生产环境谨慎使用。
技术积累
什么是Helm
Helm是K8S中包管理工具，包管理器类似于我们在 Ubuntu 中使用的apt、Centos中使用的yum一样，能快速查找、下载和安装软件包， 能够将一组K8S资源打包统一管理, 是查找、共享和使用为Kubernetes构建的软件的最佳方式。具体介绍和安装使用参见之前博文： Kubernetes包管理工具Helm简介及使用
StorageClass
由于数据库需要使用存储资源，K8S集群中我们可手动创建pv、pvc,StorageClass工具则可以自动车床件pvc。具体简介和使用参见之前博文：【实战】Kubernetes安装持久化工具NFS-StorageClass
使用的工具版本
| 软件 | 版本 |
|---|---|
| chart | 9.12.0 |
| mysql | 8.0.34 |
| kubernetes | version v1.27.2 |
| helm | version v3.12.1 |
helm 安装 MySQL 1主2从
1. 添加 bitnami 的仓库
$ helm repo add bitnami https://charts.bitnami.com/bitnami
2. 查询 MySQL 资源
$ helm repo update
[root@master k8s]# helm search repo mysql
NAME CHART VERSION APP VERSION DESCRIPTION
bitnami/mysql 9.12.1 8.0.34 MySQL is a fast, reliable, scalable, and easy t…
bitnami/phpmyadmin 12.1.0 5.2.1 phpMyAdmin is a free software tool written in P…
bitnami/mariadb 13.1.2 11.0.3 MariaDB is an open source, community-developed …
bitnami/mariadb-galera 9.1.1 11.0.3 MariaDB Galera is a multi-primary database clus…
3. 拉取 MySQL chart 到本地
$ mkdir -p /k8s/mysql && cd /k8s/mysql
#拉取 chart 到本地 /root/mysql 目录
$ helm pull bitnami/mysql --version 9.12.1
$ tar -xvf mysql-9.12.1.tgz
$ cp mysql/values.yaml ./values-test.yaml
#查看当前目录层级
[root@master mysql]# tree -L 2
.
├── mysql
│ ├── Chart.lock
│ ├── charts
│ ├── Chart.yaml
│ ├── README.md
│ ├── templates
│ ├── values.schema.json
│ └── values.yaml
├── mysql-9.12.1.tgz
└── values-test.yaml
3 directories, 7 files
4. 对chart 本地 values-test.yaml 修改
查看集群 storageclasses
[root@master mysql]# kubectl get storageclass
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
nfs-storage (default) nfs-provisioner Delete Immediate false 10d
修改配置
$ vim values-test.yaml
#镜像修改
image:
registry: registry.cn-hangzhou.aliyuncs.com
repository: senfel/mysql
tag: 8.0.34-debian-11-r31
#架构改为集群复制
architecture: replication
#root密码 主从复制密码
auth:
rootPassword: "123456root"
#主节点
primary:
configuration: |-
[mysqld]
default_authentication_plugin=caching_sha2_password ## 加密方式
persistence: ## 储存方式
storageClass: "nfs-storage"
#从节点副本数
secondary:
replicaCount: 2
#从节点
secondary:
configuration: |-
[mysqld]
default_authentication_plugin=caching_sha2_password ## 加密方式
read_only=1 ##限定普通用户只读
super_read_only=on ##限定root只读
persistence: ##储存方式
storageClass: "nfs-storage"
5. 对本地 templates 模板 修改
mysql5会有以下问题，mysql8无需处理
[root@master mysql]# kubectl api-versions
# k8s v1.27.2 版本中，statefulsets 版本为：apps/v1 而不是模板中 apps/v1beta1 的版本
# 因此需要替换 master-statefulset.yaml 和 slave-statefulset.yaml 模板
$ sed -i "s#apps/v1beta1#apps/v1#" mysql/templates/master-statefulset.yaml
$ sed -i "s#apps/v1beta1#apps/v1#" mysql/templates/slave-statefulset.yaml
未替换模板中 statefulsets 版本，直接安装服务会报错
$ helm install mysql-cluster mysql -f values-test.yaml
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: unable to recognize "": no matches for kind "StatefulSet" in version "apps/v1beta1"
6. 安装 MySQL 集群
#创建 mysql-cluster 名称空间
$ kubectl create ns mysql-cluster
#安装 MySQL 集群
$ helm install mysql-cluster -n mysql-cluster mysql -f values-test.yaml
##helm -n NAMESAPCE install SERVER_NAME FILE_NAME -f CONFIG_FILE
-n 指定 kubernetes 集群名称空间
-f 指定使用的配置文件，文件中定义的配置可以覆盖 mysql/values.yaml 文件中配置
[root@master mysql]# helm install mysql-cluster -n mysql-cluster mysql -f values-test.yaml
NAME: mysql-cluster
LAST DEPLOYED: Mon Sep 4 15:09:28 2023
NAMESPACE: mysql-cluster
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: mysql
CHART VERSION: 9.12.1
APP VERSION: 8.0.34
** Please be patient while the chart is being deployed **
Tip:
Watch the deployment status using the command: kubectl get pods -w --namespace mysql-cluster
Services:
echo Primary: mysql-cluster-primary.mysql-cluster.svc.cluster.local:3306
echo Secondary: mysql-cluster-secondary.mysql-cluster.svc.cluster.local:3306
Execute the following to get the administrator credentials:
echo Username: root
MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace mysql-cluster mysql-cluster -o jsonpath="{.data.mysql-root-password}" | base64 -d)
To connect to your database:
1. Run a pod that you can use as a client:
kubectl run mysql-cluster-client --rm --tty -i --restart='Never' --image registry.cn-hangzhou.aliyuncs.com/senfel/mysql:8.0.34-debian-11-r31 --namespace mysql-cluster --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command -- bash
2. To connect to primary service (read/write):
mysql -h mysql-cluster-primary.mysql-cluster.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"
3. To connect to secondary service (read-only):
mysql -h mysql-cluster-secondary.mysql-cluster.svc.cluster.local -uroot -p"$MYSQL_ROOT_PASSWORD"
7. 查看部署的 MySQL 集群
[root@master mysql]# kubectl get svc,pods -n mysql-cluster
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/mysql-cluster-primary ClusterIP 10.102.253.38 <none> 3306/TCP 2m2s
service/mysql-cluster-primary-headless ClusterIP None <none> 3306/TCP 2m2s
service/mysql-cluster-secondary ClusterIP 10.102.8.228 <none> 3306/TCP 2m2s
service/mysql-cluster-secondary-headless ClusterIP None <none> 3306/TCP 2m2s
NAME READY STATUS RESTARTS AGE
pod/mysql-cluster-primary-0 1/1 Running 0 2m2s
pod/mysql-cluster-secondary-0 1/1 Running 0 2m2s
pod/mysql-cluster-secondary-1 1/1 Running 0 80s
#查看 pvc
[root@master mysql]# kubectl get pvc -n mysql-cluster
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-mysql-cluster-primary-0 Bound pvc-1c32a6b6-4b22-4ce0-b45b-82c38fd0d107 8Gi RWO nfs-storage 4m12s
data-mysql-cluster-secondary-0 Bound pvc-5d263bc4-c9d4-4f3d-951c-fd578cad6e32 8Gi RWO nfs-storage 4m12s
data-mysql-cluster-secondary-1 Bound pvc-90072614-6cbb-48f2-8cdf-a8816557e46b 8Gi RWO nfs-storage 3m30s
#查看 pv
[root@master mysql]# kubectl get pv | grep mysql
pvc-1c32a6b6-4b22-4ce0-b45b-82c38fd0d107 8Gi RWO Delete Bound mysql-cluster/data-mysql-cluster-primary-0 nfs-storage 4m45s
pvc-5d263bc4-c9d4-4f3d-951c-fd578cad6e32 8Gi RWO Delete Bound mysql-cluster/data-mysql-cluster-secondary-0 nfs-storage 4m45s
pvc-90072614-6cbb-48f2-8cdf-a8816557e46b 8Gi RWO Delete Bound mysql-cluster/data-mysql-cluster-secondary-1 nfs-storage 4m3s
pvc-a5673e1b-e3b2-415f-b2bb-c649eb4cd33d 8Gi RWO Delete Bound default/data-mysql-cluster-secondary-0 nfs-storage 32m
pvc-a683be89-289c-4088-b46f-f2af4d322a1a 8Gi RWO Delete Bound default/data-mysql-cluster-primary-0 nfs-storage 32m
8. 连接 MySQL 集群 验证服务
#启动一个临时容器
$ kubectl run mysql-cluster-client --rm --tty -i --restart=‘Never’ --image registry.cn-hangzhou.aliyuncs.com/senfel/mysql:8.0.34-debian-11-r31 --namespace mysql-cluster --env MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD --command – bash
##登陆 MySQL Master节点
$ mysql -h mysql-cluster-primary.mysql-cluster.svc.cluster.local -uroot -p
Enter password: 123456root
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| my_database |
| mysql |
| performance_schema |
| sys |
+--------------------+
5 rows in set (0.01 sec)
#查看主从状态
#查看File和Position的值，在从库配置中会显示。
mysql> show master status\G
*************************** 1. row ***************************
File: mysql-bin.000003
Position: 157
Binlog_Do_DB:
Binlog_Ignore_DB:
Executed_Gtid_Set:
1 row in set (0.00 sec)
##登陆从库，查看主从同步状态
$ mysql -h mysql-cluster-secondary.mysql-cluster.svc.cluster.local -uroot -p
Enter password: 123456root
mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for source to send event
Master_Host: mysql-cluster-primary
Master_User: replicator
Master_Port: 3306
Connect_Retry: 10
Master_Log_File: mysql-bin.000003
Read_Master_Log_Pos: 157
Relay_Log_File: mysql-relay-bin.000006
Relay_Log_Pos: 373
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 157
Relay_Log_Space: 799
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 313
Master_UUID: ff169bc1-4af1-11ee-b795-f6df04eaa47a
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Replica has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
Master_public_key_path:
Get_master_public_key: 0
Network_Namespace:
1 row in set, 1 warning (0.00 sec)
至此，使用Helm在K8S集群安装MySQL主从实战演示完成。
ingress-nginx开放主从端口
ingress-nginx是K8S开放端口的工具，可以直接将端口映射到集群外部。
#修改ingress-nginx tcp配置
$ vim values.yaml
#格式 "ns":"svc:port"
tcp: {
"56379":"redis-cluster/redis-cluster:6379",
"53306": "mysql-cluster/mysql-cluster-primary:3306",
"54406": "mysql-cluster/mysql-cluster-secondary:3306"
}
#升级releas
$ helm upgrade ingress-nginx -n ingress-nginx .
#验证服务器端口是否开启
[root@node2 ~]# netstat -nplt | grep 06
tcp 0 0 0.0.0.0:53306 0.0.0.0:* LISTEN 16944/nginx: master
tcp 0 0 0.0.0.0:54406 0.0.0.0:* LISTEN 16944/nginx: master
tcp6 0 0 :::53306 :::* LISTEN 16944/nginx: master
tcp6 0 0 :::54406 :::* LISTEN 16944/nginx: master
查看最终的nginx.conf
#获取ingress-nginx pods
[root@node2 ~]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-controller-75x86 1/1 Running 0 8m51s
ingress-nginx-controller-jgzks 1/1 Running 0 8m29s
#查看最终的nginx.conf
[root@master ingress-nginx]# kubectl exec ingress-nginx-controller-75x86 -n ingress-nginx – cat /etc/nginx/nginx.conf
至此TCP端口已暴露完成，对于UDP也是一样的修改value.yaml配置即可
外网navicat连接验证
两台ingress-nginx服务物理机都开放了端口，都可以连接！！！
