==============================================
主机环境：windows 11
k8s版本：v1.25
dashboard版本：v2.7.0
calico版本： v3.26.1
CentOS8版本：4.18.0-348.7.1.el8_5.x86_64
用到的脚本：
https://gitcode.net/sundongsdu/k8s_cluster
==============================================
1. Vagrant创建虚拟机节点
首先需要在win11主机上安装vagrant与virtualbox，此处不在赘述。
vagrant默认账号：vagrant/vagrant, root/vagrant
然后在win11的Vagrantfile文件所在目录下，执行：
vagrant up 创建虚拟机
通过vagrant status查看状态：
2. 通用配置
##################### 所有节点都需要执行的操作 开始 #################
vagrant ssh kubemaster/worker1/worker2
ssh配置：
vi /etc/ssh/sshd_config
PasswordAuthentication yes
然后执行 systemctl restart sshd.service
配置centos yum源：
设置centos8 mirror list:
sudo sed -i -e "s|mirrorlist=|#mirrorlist=|g" /etc/yum.repos.d/CentOS-*
sudo sed -i -e "s|#baseurl=http://mirror.centos.org|baseurl=http://mirrors.aliyun.com|g" /etc/yum.repos.d/CentOS-*
如果需要设置代理：
vi /etc/yum.conf
增加一行：
proxy=http://your-proxy-server:port
### 升级内核：
yum list installed|grep kernel*
yum update kernel -y
yum -y install kernel-devel kernel-headers
yum list installed|grep kernel*
删除老的内核，比如：
rpm -e kernel-4.18.0-240.1.1.el8_3 kernel-core-4.18.0-240.1.1.el8_3 kernel-modules-4.18.0-240.1.1.el8_3 kernel-tools-4.18.0-240.1.1.el8_3 kernel-tools-libs-4.18.0-240.1.1.el8_3
yum list installed|grep kernel*
安装完以后reboot
==============================================
此处，安装Guest Additions:
yum -y install libX11 libXt libXext libXmu xorg-x11-server-Xorg
yum -y install elfutils-libelf-devel gcc make perl
vbox设置->存储里面挂载VBoxGuestAdditions.iso
mkdir -p /mnt/cdrom
mount /dev/cdrom /mnt/cdrom
cd /mnt/cdrom
./VBoxLinuxAdditions.run
查看：
[root@kubemaster cdrom]# lsmod|grep vboxguest
vboxguest 409600 2 vboxsf
安装完以后reboot
设置VM共享目录
此时的共享目录为：/media/sf_share_dir/
!!! 此处执行configure-vm.sh
##################### 所有节点都需要执行的操作 结束 #################
3. master节点配置
##################### master节点需要执行的操作 开始 #################
3.1 master节点 init
k8s init之前 执行
containerd config default > /etc/containerd/config.toml
修改 /etc/containerd/config.toml
要确保 cri 没有出现在 /etc/containerd/config.toml 文件中 disabled_plugins
#disabled_plugins = ["cri"]
修改registry.k8s.io/pause:3.6，比如是通过阿里云拿到的镜像，那么修改为：
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"
然后执行systemctl restart containerd
===============================================
执行init:
kubeadm init --apiserver-advertise-address=192.168.56.101 --pod-network-cidr=10.244.0.0/16 --image-repository registry.aliyuncs.com/google_containers
执行结果：
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.56.101:6443 --token bl52oq.d2qaxxmcw1cl3wnd \
--discovery-token-ca-cert-hash sha256:75aae5d73711924153d353395365d7bbcd44646f94f7307e58d76b16aebaedc5
如果后续忘记这个join命令，可以在master上重新生成：
kubeadm token create --print-join-command
3.2 master节点安装calico
calico的安装可以参考https://www.cnblogs.com/wangguishe/p/17635391.html
=======================================
安装calico之前可以配置docker镜像加速：
vi /etc/docker/daemon.json
{
"registry-mirrors": ["https://hub-mirror.c.163.com"]
}
systemctl daemon-reload
systemctl restart docker
如果需要配置代理，可以参考https://blog.csdn.net/weixin_45894220/article/details/130085121
=======================================
calico与k8s的版本对应关系：
https://docs.tigera.io/calico/latest/getting-started/kubernetes/requirements
We test Calico v3.26 against the following Kubernetes versions.
v1.24
v1.25
v1.26
v1.27
安装calico:
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.1/manifests/calico.yaml
安装成功以后：
[root@kubemaster vagrant]# kubectl get pod --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-74cfc9ffcc-tvkkx 1/1 Running 0 6m7s
kube-system calico-node-lzb6w 1/1 Running 0 6m7s
kube-system coredns-c676cc86f-g89d4 1/1 Running 0 13h
kube-system coredns-c676cc86f-n78xv 1/1 Running 0 13h
kube-system etcd-kubemaster 1/1 Running 0 13h
kube-system kube-apiserver-kubemaster 1/1 Running 0 13h
kube-system kube-controller-manager-kubemaster 1/1 Running 1 (3h52m ago) 13h
kube-system kube-proxy-bw82t 1/1 Running 0 13h
kube-system kube-scheduler-kubemaster 1/1 Running 2 (95s ago) 13h
3.3 master节点安装dashboard
安装dashboard之前先解除master的限制：
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
k8s与dashboard的版本对应：
https://github.com/kubernetes/dashboard/releases
v2.7.0 Compatibility
| Kubernetes version | 1.22 | 1.23 | 1.24 | 1.25 |
|---|---|---|---|---|
| Compatibility | ? | ? | ? | ✓ |
✓Fully supported version range.?Due to breaking changes between Kubernetes API versions, some features might not work correctly in the Dashboard.
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
修改其中的 imagePullPolicy: IfNotPresent
kubectl apply -f recommended.yaml
kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard
This will open up a vi screen. Search for the entry ClusterIP and replace it with
NodePort (Case sensitive)
Run the following command to find out the port on which it will listen,
kubectl get svc -n kubernetes-dashboard|grep NodePort|awk '{print
$5}'|cut -f 2 -d :|cut -f 1 -d /
比如此处端口为31768
在vbox上配置端口转发：
创建用户：
参考：https://www.cnblogs.com/wangguishe/p/17582668.html
admin-user.yml
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard执行
kubectl apply -f admin-user.yml 配置secret:
admin-user-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: admin-user-secret
namespace: kubernetes-dashboard
annotations:
kubernetes.io/service-account.name: admin-user
type: kubernetes.io/service-account-token执行
kubectl apply -f admin-user-secret.yaml查看Secret：
kubectl get secret -n kubernetes-dashboard 
查看token:
kubectl get secret admin-user-secret -n kubernetes-dashboard -o jsonpath={".data.token"} | base64 -d在windows主机上访问dashboard:
https://127.0.0.1:9443
输入上述的token,
##################### master节点需要执行的操作 结束 #################
4. worker节点配置
##################### worker节点都需要执行的操作 开始 #################
找到上面master节点的join信息并在worker上执行：
kubeadm join 192.168.56.101:6443 --token bl52oq.d2qaxxmcw1cl3wnd \
--discovery-token-ca-cert-hash sha256:75aae5d73711924153d353395365d7bbcd44646f94f7307e58d76b16aebaedc5
##################### worker节点都需要执行的操作 结束 #################
worker节点加入以后，可以在master节点查看nodes:
查看pods:
如果想查看某个node上的pods，比如worker1:
kubectl get pods --all-namespaces -o wide --field-selector spec.nodeName=worker1
参考：
Multi-node Kubernetes setup on Windows – Technology evangelist
https://medium.com/@srpillai/single-node-kubernetes-on-centos-c8c3507e3e65
https://www.cnblogs.com/wangguishe/p/17582668.html
