centos7部署rancher2.5

一、什么是 Rancher

Rancher 是为使用容器的公司打造的容器管理平台。Rancher 简化了使用 Kubernetes 的流程，开发者可以随处运行 Kubernetes（Run Kubernetes Everywhere），满足 IT 需求规范，赋能 DevOps 团队。

Rancher 1.x 最初是为了支持多种容器编排引擎而构建的，其中包括 Rancher 自己的容器编排引擎 Cattle。但随着 Kubernetes 在市场上的兴起，Rancher 2.x 已经完全转向了 Kubernetes。Rancher 2.x 可以部署和管理在任何地方运行的 Kubernetes 集群。

说明：下文中所有的“Rancher”代指的都是 Rancher 2.x。

Rancher 可以创建来自 Kubernetes 托管服务提供商的集群，自动创建节点并安装 Kubernetes 集群，或者导入任何已经存在的 Kubernetes 集群。

Rancher 通过支持集群的身份验证和基于角色的访问控制（RBAC），使系统管理员能够从一个位置控制全部集群的访问。Rancher 可以对集群及其资源进行详细的监控和并在需要时发送告警，也可以将容器日志发送给外部日志系统，并通过应用商店与 Helm 集成。如果您具有外部 CI/CD 流水线系统，则可以将其与 Rancher 对接，如果没有，Rancher 也提供了简单易用的流水线来帮助您自动部署和升级工作负载。除此之外，Rancher 还有很多开箱即用的功能来帮助您更好的管理集群和业务应用，例如多集群应用，全局 DNS，服务网格，安全扫描，集群模版和基于 OPA 的策略管理等功能。

总而言之，Rancher 是一个全栈式的 Kubernetes 容器管理平台，也是一个可以在任何地方都能成功运行 Kubernetes 的工具。
在这里插入图片描述

二、准备环境

本次搭建共使用了4台机器，操作系统为Centos7：

ip	服务
172.20.29.95	rancher
172.20.29.1	k8s master worker0
172.20.29.2	k8s worker1
172.20.29.3	k8s worker2

以下操作每台机器都需要做。

1. 关闭防火墙

$ sudo systemctl status firewalld.service
$ sudo systemctl disable firewalld.service
$ sudo firewall-cmd --state
not running
$ sudo sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config # 禁用Security-Enhanced Linux

2. 同步时钟

$ yum install -y ntp
$ ntpdate time1.aliyun.com
$ crontab -e
0 */1 * * * ntpdate time1.aliyun.com

3. 关闭swap

k8s官方要求使用真实的物理内存，而不是swap的虚拟内存，据说k8s v1.25后不需要再关闭swap。

$ sudo swapoff -a # 临时关闭，立即生效。
$ sudo vim /etc/fstab
#/dev/mapper/centos-swap swap                    swap    defaults        0 0  # 注释掉该行，机器重启后swap仍是关闭状态，永久生效。
$ sudo free -h  # 检查swap是否关闭。
              total        used        free      shared  buff/cache   available
Mem:           7.6G        1.0G        456M        448M        6.2G        5.3G
Swap:            0B          0B          0B

4. 开启内核路由转发

如果不开启，发布应用后，无法访问应用

$ sudo vim /etc/sysctl.conf
net.ipv4.ip_forward=1
$ sudo sysctl -p  # 使之生效

5. 安装docker

参考docker官网安装文档
https://docs.docker.com/engine/install/

$ docker --version
Docker version 20.10.18, build b40c2f6

三、安装rancher

rancher推荐将其部署在一个专用的k8s集群上，以保证rancher的高可用。由于服务器资源有限，这里将rancher部署在单节点上，方便快速启动试用。
注意：rancher2.5不能在linux内核的5.15以上版本运行，查看内核版本uname -a

1. rancher主节点启动rancher服务

docker run -d --privileged -p 80:80 -p 443:443 -v /root/docker_volume/rancher:/var/lib/rancher --restart=always --name rancher-2.5.15 rancher/rancher:v2.5.15

rancher主节点启动前，服务器内存占用1G，启动后内存占用2.7G

$ docker images
REPOSITORY        TAG       IMAGE ID       CREATED       SIZE
rancher/rancher   v2.5.16   ec97e049bc70   4 weeks ago   1.12GB

$ docker ps
CONTAINER ID   IMAGE                     COMMAND           CREATED          STATUS          PORTS                                                                      NAMES
2123ec53cb28   rancher/rancher:v2.5.16   "entrypoint.sh"   12 minutes ago   Up 11 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   rancher

2. 配置rancher增加新k8s集群

等待一段时间后，服务启动，浏览器访问rancher所在节点的IP，设置账号密码
在这里插入图片描述

添加新的集群

选择添加自定义集群（等会儿就会看到用rancher提供的命令拉起一个新的K8S集群）

这里Node Port Range代表开放的端口范围，默认是30000-32767，将它改为全部端口都开放0-65535,因为如果部署的是web服务，域名指向worker(三个worker都是等效的)时，需要访问80和443
在这里插入图片描述

3. 初始话k8s集群master节点

将所有复选框勾上，然后点击复制命令到剪切板
在这里插入图片描述
在master机器上粘贴刚复制的命令，它会自动拉起k8s的master节点所需的所有服务。
k8s 主节点启动前，服务器占用内存为486M，启动后，占用内容为1.6G，启动后，镜像和容器情况如下：

$ docker images
REPOSITORY                                            TAG                    IMAGE ID       CREATED         SIZE
rancher/rancher-agent                                 v2.5.16                fd825fbb4fed   4 weeks ago     554MB
rancher/hyperkube                                     v1.20.15-rancher2      aa8bbbd04a74   2 months ago    2.01GB
rancher/nginx-ingress-controller                      nginx-1.2.1-rancher1   010d83d7f87d   3 months ago    292MB
rancher/fleet-agent                                   v0.3.9                 25ddd5cd4c11   5 months ago    155MB
rancher/rke-tools                                     v0.1.80                c1309431f38c   6 months ago    289MB
rancher/mirrored-coreos-flannel                       v0.15.1                e6ea68648f0c   10 months ago   69.5MB
rancher/mirrored-ingress-nginx-kube-webhook-certgen   v1.1.1                 c41e9fcadf5a   11 months ago   47.7MB
rancher/mirrored-pause                                3.6                    6270bb605e12   12 months ago   683kB
rancher/mirrored-metrics-server                       v0.5.0                 1c655933b9c5   16 months ago   63.5MB
rancher/mirrored-coreos-etcd                          v3.4.15-rancher1       87c4f81d8822   18 months ago   83.8MB
rancher/mirrored-calico-node                          v3.17.2                5a3c598c81d5   19 months ago   165MB
rancher/mirrored-calico-pod2daemon-flexvol            v3.17.2                e2608e41ac3d   19 months ago   21.7MB
rancher/mirrored-calico-cni                           v3.17.2                81860c306a8d   19 months ago   128MB
rancher/mirrored-calico-kube-controllers              v3.17.2                59b927df412f   19 months ago   52.1MB
rancher/mirrored-coredns-coredns                      1.8.0                  296a6d5035e2   23 months ago   42.5MB
rancher/mirrored-cluster-proportional-autoscaler      1.8.1                  4f1064cf7caf   2 years ago     40.7MB
rancher/kube-api-auth                                 v0.1.4                 96148b821282   2 years ago     37.3MB


$ docker ps    
CONTAINER ID   IMAGE                                              COMMAND                  CREATED          STATUS          PORTS     NAMES
8c0759387c77   rancher/nginx-ingress-controller                   "/usr/bin/dumb-init …"   11 minutes ago   Up 11 minutes             k8s_controller_nginx-ingress-controller-xdszb_ingress-nginx_bad16261-4cae-4ce7-a3fd-7c40abb91f33_0
3f0ab8ccbb17   rancher/mirrored-pause:3.6                         "/pause"                 11 minutes ago   Up 11 minutes             k8s_POD_nginx-ingress-controller-xdszb_ingress-nginx_bad16261-4cae-4ce7-a3fd-7c40abb91f33_0
f3e129eb02b7   rancher/mirrored-metrics-server                    "/metrics-server --c…"   13 minutes ago   Up 13 minutes             k8s_metrics-server_metrics-server-b545f4746-kj5b5_kube-system_5a41c10e-1a6e-4d01-8618-9c324ca5865b_0
616205f47028   rancher/mirrored-calico-kube-controllers           "/usr/bin/kube-contr…"   13 minutes ago   Up 13 minutes             k8s_calico-kube-controllers_calico-kube-controllers-7d5d95c8c9-fgzfv_kube-system_c0ce4392-9f64-4caa-8c56-66c342184225_0
b545662ef83a   rancher/mirrored-cluster-proportional-autoscaler   "/cluster-proportion…"   13 minutes ago   Up 13 minutes             k8s_autoscaler_coredns-autoscaler-7958578cb9-z8jgd_kube-system_29e5f5c7-87c9-4822-b717-739647e09244_0
a7d537af290d   rancher/mirrored-coredns-coredns                   "/coredns -conf /etc…"   14 minutes ago   Up 14 minutes             k8s_coredns_coredns-b85b997d-w5v77_kube-system_04b59a2f-8e64-41ab-ac67-596e35f4c528_0
12aa618d5ea2   25ddd5cd4c11                                       "fleetagent"             14 minutes ago   Up 14 minutes             k8s_fleet-agent_fleet-agent-96f6f455c-mbhk7_fleet-system_e1ccd71f-0721-4499-9272-f5a8b815113a_0
9aba34c2a6d9   rancher/mirrored-pause:3.6                         "/pause"                 14 minutes ago   Up 14 minutes             k8s_POD_fleet-agent-96f6f455c-mbhk7_fleet-system_e1ccd71f-0721-4499-9272-f5a8b815113a_0
2866cebfc019   rancher/mirrored-coreos-flannel                    "/opt/bin/flanneld -…"   14 minutes ago   Up 14 minutes             k8s_kube-flannel_canal-4t7zl_kube-system_632fbc95-c421-4102-a0ce-2095b19a27c4_0
e063f7019640   fd825fbb4fed                                       "run.sh"                 15 minutes ago   Up 15 minutes             k8s_cluster-register_cattle-cluster-agent-6f8459d7cf-stzlj_cattle-system_67d2bee2-ee1a-4fe5-b618-9d87145db2bf_0
e5019366e889   rancher/mirrored-pause:3.6                         "/pause"                 15 minutes ago   Up 15 minutes             k8s_POD_metrics-server-b545f4746-kj5b5_kube-system_5a41c10e-1a6e-4d01-8618-9c324ca5865b_1
f5b29ed4218b   rancher/mirrored-pause:3.6                         "/pause"                 15 minutes ago   Up 15 minutes             k8s_POD_cattle-cluster-agent-6f8459d7cf-stzlj_cattle-system_67d2bee2-ee1a-4fe5-b618-9d87145db2bf_1
438b7cf9d6a1   rancher/mirrored-pause:3.6                         "/pause"                 15 minutes ago   Up 15 minutes             k8s_POD_calico-kube-controllers-7d5d95c8c9-fgzfv_kube-system_c0ce4392-9f64-4caa-8c56-66c342184225_1
e6c38ee60a68   rancher/mirrored-pause:3.6                         "/pause"                 15 minutes ago   Up 15 minutes             k8s_POD_coredns-b85b997d-w5v77_kube-system_04b59a2f-8e64-41ab-ac67-596e35f4c528_1
e394d5373bf4   rancher/mirrored-pause:3.6                         "/pause"                 15 minutes ago   Up 15 minutes             k8s_POD_coredns-autoscaler-7958578cb9-z8jgd_kube-system_29e5f5c7-87c9-4822-b717-739647e09244_1
79c3bd5287b1   rancher/mirrored-calico-node                       "start_runit"            15 minutes ago   Up 15 minutes             k8s_calico-node_canal-4t7zl_kube-system_632fbc95-c421-4102-a0ce-2095b19a27c4_0
8483e41c3c99   rancher/kube-api-auth                              "/bin/sh -c 'kube-ap…"   15 minutes ago   Up 15 minutes             k8s_kube-api-auth_kube-api-auth-w86tn_cattle-system_352df28e-054f-4270-b2f1-6887276f6b32_0
ac3ce82d36f9   fd825fbb4fed                                       "run.sh"                 15 minutes ago   Up 15 minutes             k8s_agent_cattle-node-agent-lndgr_cattle-system_75a6860d-709d-430a-9b86-615e21256463_0
95960fc4f645   rancher/mirrored-pause:3.6                         "/pause"                 15 minutes ago   Up 15 minutes             k8s_POD_kube-api-auth-w86tn_cattle-system_352df28e-054f-4270-b2f1-6887276f6b32_0
a0090aa709bc   rancher/mirrored-pause:3.6                         "/pause"                 15 minutes ago   Up 15 minutes             k8s_POD_cattle-node-agent-lndgr_cattle-system_75a6860d-709d-430a-9b86-615e21256463_0
1260263ddcc3   rancher/mirrored-pause:3.6                         "/pause"                 16 minutes ago   Up 16 minutes             k8s_POD_canal-4t7zl_kube-system_632fbc95-c421-4102-a0ce-2095b19a27c4_0
8a40d18636ac   rancher/hyperkube:v1.20.15-rancher2                "/opt/rke-tools/entr…"   16 minutes ago   Up 16 minutes             kube-proxy
16eb7570fd24   rancher/hyperkube:v1.20.15-rancher2                "/opt/rke-tools/entr…"   16 minutes ago   Up 16 minutes             kubelet
d0f93025656f   rancher/hyperkube:v1.20.15-rancher2                "/opt/rke-tools/entr…"   17 minutes ago   Up 17 minutes             kube-scheduler
7010f36b1d91   rancher/hyperkube:v1.20.15-rancher2                "/opt/rke-tools/entr…"   17 minutes ago   Up 17 minutes             kube-controller-manager
458de14842f5   rancher/hyperkube:v1.20.15-rancher2                "/opt/rke-tools/entr…"   17 minutes ago   Up 17 minutes             kube-apiserver
3686395fe721   rancher/mirrored-coreos-etcd:v3.4.15-rancher1      "/usr/local/bin/etcd…"   17 minutes ago   Up 17 minutes             etcd

4. 初始化另外两台worker节点

只保留worker的复选框，复制该命令到剪切板
在这里插入图片描述
在另外两台worker机器上粘贴该命令，等待所有服务启动。
k8s 工作节点启动前占用内存，475M，启动后，占用内存1G，镜像和容器情况如下：

$ docker images
REPOSITORY                                   TAG                    IMAGE ID       CREATED         SIZE
rancher/rancher-agent                        v2.5.16                fd825fbb4fed   4 weeks ago     554MB
rancher/hyperkube                            v1.20.15-rancher2      aa8bbbd04a74   2 months ago    2.01GB
rancher/nginx-ingress-controller             nginx-1.2.1-rancher1   010d83d7f87d   3 months ago    292MB
rancher/rke-tools                            v0.1.80                c1309431f38c   6 months ago    289MB
rancher/mirrored-pause                       3.6                    6270bb605e12   12 months ago   683kB
rancher/mirrored-calico-pod2daemon-flexvol   v3.17.2                e2608e41ac3d   19 months ago   21.7MB
rancher/mirrored-calico-cni                  v3.17.2                81860c306a8d   19 months ago   128MB
rancher/mirrored-coredns-coredns             1.8.0                  296a6d5035e2   23 months ago   42.5MB

# root @ gp-sdw1 in /etc [0:59:44] 
$ docker ps
CONTAINER ID   IMAGE                                 COMMAND                  CREATED              STATUS              PORTS     NAMES
80b880c5c8c6   rancher/mirrored-coredns-coredns      "/coredns -conf /etc…"   12 seconds ago       Up 11 seconds                 k8s_coredns_coredns-b85b997d-2r96k_kube-system_067a672a-281c-49b7-961c-dabc2b3052e9_0
9b05a56ba441   rancher/mirrored-pause:3.6            "/pause"                 33 seconds ago       Up 32 seconds                 k8s_POD_coredns-b85b997d-2r96k_kube-system_067a672a-281c-49b7-961c-dabc2b3052e9_1
41dcc1514fc2   rancher/nginx-ingress-controller      "/usr/bin/dumb-init …"   58 seconds ago       Up 58 seconds                 k8s_controller_nginx-ingress-controller-c9tzj_ingress-nginx_5d9c5f14-8432-484c-92fc-cd9f30cef5d5_0
3dd8d94cc2b9   fd825fbb4fed                          "run.sh"                 About a minute ago   Up About a minute             k8s_agent_cattle-node-agent-jnhgf_cattle-system_3fcec631-932b-4ca3-b42c-d588c26d5d04_0
4af44bd41b41   rancher/mirrored-pause:3.6            "/pause"                 About a minute ago   Up About a minute             k8s_POD_cattle-node-agent-jnhgf_cattle-system_3fcec631-932b-4ca3-b42c-d588c26d5d04_0
faf7f5d91bf5   rancher/mirrored-pause:3.6            "/pause"                 About a minute ago   Up About a minute             k8s_POD_nginx-ingress-controller-c9tzj_ingress-nginx_5d9c5f14-8432-484c-92fc-cd9f30cef5d5_0
fbb0a3523468   rancher/mirrored-pause:3.6            "/pause"                 About a minute ago   Up About a minute             k8s_POD_canal-69x29_kube-system_bfb07cb9-58d9-4253-98da-db05c7c89d8b_0
9c3724814eb9   rancher/hyperkube:v1.20.15-rancher2   "/opt/rke-tools/entr…"   About a minute ago   Up About a minute             kubelet
4c7926ccb321   rancher/hyperkube:v1.20.15-rancher2   "/opt/rke-tools/entr…"   About a minute ago   Up About a minute             kube-proxy
99f8f4559ebc   rancher/rke-tools:v0.1.80             "nginx-proxy CP_HOST…"   4 minutes ago        Up 4 minutes                  nginx-proxy

四、遇到的一些问题

1. 尝试重装不同rancher版本时遇到服务启动异常

查看rancher主节点的容器日志，执行到检查注册节点的健康状态失败：
[etcd] Successfully started etcd plane… Checking etcd cluster health

注册节点的容器日志
time=“2022-09-19T15:40:33Z” level=info msg=“Waiting for node to register. Either cluster is not ready for registering, cluster is currently provisioning, or etcd, controlplane and worker node have to be registered”

这是因为重装不同版本时，只删掉了容器，没有删掉容器的挂载内容，查看注册节点的etcd容器信息，可以看到宿主机上由绑定的目录

        "Mounts": [
            {
                "Type": "bind",
                "Source": "/var/lib/etcd",
                "Destination": "/var/lib/rancher/etcd",
                "Mode": "z",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/etc/kubernetes",
                "Destination": "/etc/kubernetes",
                "Mode": "z",
                "RW": true,
                "Propagation": "rprivate"
            }
        ]