前言：
shell脚本的功能十分强大，这一点毋庸置疑的。那么，平常的工作中总是免不了和脚本打交道，也免不了要自己编写一些脚本。
每个人都希望自己编写的脚本强壮，简单，易用，功能多，并且总是希望脚本运行后不是一片黑白，色彩单调，无趣，而是有着整齐的，让人赏心悦目的输出日志。
OK，希望总是美好的嘛，但，如何实现也是有一定的技巧的，并不是你想实现就实现的哦。
那么，如何能够快速的提高shell编程的能力呢？
俗话说：师夷长技以制夷 这句话用在shell脚本编程方面同样的正确。在遇到一些shell脚本的时候，我们不应该是只是把这个脚本跑起来，然后看看结果正确了吗？然后就OK了。
正如某些优秀的编程人员一样，有着阅读源码的好习惯，经常的阅读一些优秀的shell脚本，并且能够解析这些优秀的shell脚本如何运作的，长此以往，shell编程的能力会如同我一样优秀，对吗？
好了，废话说了一堆，现在就隆重介绍一下github上的一个脚本。
此脚本是kubernetes集群更新证书用脚本，也就是说功能比较单一：将某个kubernetes集群的所有证书（或者某个组件的证书）延期到十年，并且在更新证书的同时给所有证书（或者某个组件的证书）做一个备份。
下载地址如下：
git clone https://github.com/yuyicai/update-kube-cert.git
cd update-kubeadm-cert
chmod 755 update-kubeadm-cert.sh
一，
脚本日志格式化输出
首先，我们定义了四种颜色，并定义了三种日志输出函数，分别是error，info，warning，最后只调用了log::info函数和check_file函数。
#!/usr/bin/env bash
#!author zsk
set -o errexit
set -o pipefail
# set -o xtrace
# set output color
NC='\033[0m'
RED='\033[31m'
GREEN='\033[32m'
YELLOW='\033[33m'
BLUE='\033[34m'
log::err() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${RED}ERROR${NC}] %b\n" "$@"
}
log::info() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${GREEN}INFO] %b\n" "$@"
}
log::warning() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${YELLOW}WARNING${NC}] \033[0m%b\n" "$@"
}
check_file() {
if [[ ! -r ${1} ]]; then
log::err "can not find ${1}"
exit 1
fi
}
log::info "${GREEN}updated !!!${NC}"
那么，此脚本的输出是这样的：
输出了脚本执行时间和日志等级
OK，根据以上脚本，改造一个收集服务器信息的脚本，此脚本见我的博客：Linux运维小技巧---每日收集所有服务器信息并归档到指定服务器_晚风_END的博客-CSDN博客_linux运维 日活量怎么提取
改造后的脚本如下：
#!/bin/bash
#!author zsk
#description:system_info
NC='\033[0m'
RED='\033[31m'
GREEN='\033[32m'
YELLOW='\033[33m'
BLUE='\033[34m'
log::err() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${RED}ERROR${NC}] %b\n" "$@"
}
log::info() {
printf "[${GREEN}$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${YELLOW}INFO]${NC} %b\n" "$*"
}
log::warning() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${YELLOW}WARNING${NC}] \033[0m%b\n" "$@"
}
log::info "-------------------------------System Information----------------------------"
log::info "${GREEN}Hostname:${NC}\t\t\t"`hostname`
log::info "${GREEN}uptime:\t\t\t"`uptime | awk '{print $3,$4}' | sed 's/,//'`
log::info "${GREEN}Manufacturer:\t\t"`cat /sys/class/dmi/id/chassis_vendor`
log::info "${GREEN}Product Name:\t\t"`cat /sys/class/dmi/id/product_name`
log::info "${GREEN}Product Version:${NC}\t\t"`cat /sys/class/dmi/id/product_version`
log::info "${GREEN}Serial Number:${NC}\t\t"`cat /sys/class/dmi/id/product_serial`
lscpu | grep VMware &>/dev/null
log::info "${GREEN}Machine Type:\t\t"`if [ $? -eq 0 ]; then echo "VMware"; else echo "Physical"; fi`
log::info "${GREEN}Operating System:\t"`hostnamectl | grep "Operating System" | cut -d ' ' -f5-`
log::info "${GREEN}Version:\t\t"`cat /etc/redhat-release `
log::info "${GREEN}Kernel:\t\t\t"`uname -r`
log::info "${GREEN}Architecture:\t\t"`arch`
log::info "${GREEN}Processor Name:\t\t"`awk -F':' '/^model name/ {print $2}' /proc/cpuinfo | uniq | sed -e 's/^[ \t]*//'`
log::info "${GREEN}processor number:\t"`cat /proc/cpuinfo | grep processor | wc -l`
log::info "${GREEN}Active User:\t\t"`w | cut -d ' ' -f1 | grep -v USER | xargs -n1`
log::info "${GREEN}System Main IP:\t\t"`hostname -I`
echo ""
log::info "-------------------------------CPU/Memory Usage------------------------------"
log::info "${GREEN}Memory Usage:\t\t"`free | awk '/Mem/{printf("%.2f%"), $3/$2*100}'`
log::info "${GREEN}Swap Usage:\t\t"`free | awk '/Swap/{printf("%.2f%"), $3/$2*100}'`
log::info "${GREEN}CPU Free Space:\t\t"`top -bn 1 -i -c | grep ^% |cut -f4 -d, | awk '{print $1}'`%
log::info "${GREEN}CPU LOAD:\t\t"`top -bn 1 -i -c|grep load|awk -F',' '{print $4,$5,$6}'`%
echo ""
log::info "-------------------------------Disk Usage>20%-------------------------------"
df -Ph | sed s/%//g | awk '{ if($5 > 20) print $0;}'
echo ""
echo "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
echo "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
脚本执行后输出如下：
可以看到比原来的脚本输出更加整齐，更加的美观了。
OK，现在说回update-kubeadm-cert.sh 这个脚本，也就是github下载下来的脚本，下面将讲讲如何调试此脚本。
二，
脚本调试方法：
update-kubeadm-cert.sh 这个脚本内容比较多，400多行，结构是比较复杂的。
大体结构是由一个main函数和众多的子函数组成。
由于函数比较多，因此，变量定义的非常多，很多是以local的形式定义的。
此脚本需要外部参数，主要是四个外部参数：master，all ，check以及*（*代表任意参数） 也就是运行脚本的时候是 bash 脚本名称 master|all|check 这样的形式运行。
例如，带外部参数check运行此脚本，输出如下：
[root@node4 ~]# bash update-kube-cert-master/update-kubeadm-cert.sh check
CERTIFICATE EXPIRES
/etc/kubernetes/controller-manager.config Dec 5 13:22:20 2032 GMT
/etc/kubernetes/scheduler.config Dec 5 13:22:20 2032 GMT
/etc/kubernetes/admin.config Dec 5 13:22:20 2032 GMT
/etc/kubernetes/pki/ca.crt Nov 12 07:26:23 2122 GMT
/etc/kubernetes/pki/apiserver.crt Dec 5 13:22:20 2032 GMT
/etc/kubernetes/pki/apiserver-kubelet-client.crt Dec 5 13:22:20 2032 GMT
/etc/kubernetes/pki/front-proxy-ca.crt Nov 12 07:26:25 2122 GMT
/etc/kubernetes/pki/front-proxy-client.crt Dec 5 13:22:20 2032 GMT
/etc/kubernetes/pki/etcd/ca.crt Nov 12 07:26:26 2122 GMT
/etc/kubernetes/pki/etcd/server.crt Dec 5 13:22:18 2032 GMT
/etc/kubernetes/pki/etcd/peer.crt Dec 5 13:22:18 2032 GMT
/etc/kubernetes/pki/etcd/healthcheck-client.crt Dec 5 13:22:18 2032 GMT
/etc/kubernetes/pki/apiserver-etcd-client.crt Dec 5 13:22:19 2032 GMT 那么，外部参数是*的时候（不是指定的外部参数的时候，*等于报错提示），该脚本的输出如下：
[root@node4 ~]# bash update-kube-cert-master/update-kubeadm-cert.sh ewrwerwe
[2022-12-09T12:37:35.53+0800][ERROR] unknown, unsupported cert type: ewrwerwe, supported type: "all", "master"
Documentation: https://github.com/yuyicai/update-kube-cert
example:
'./update-kubeadm-cert.sh all' update all etcd certificates, master certificates and kubeconf
/etc/kubernetes
├── admin.conf
├── controller-manager.conf
├── scheduler.conf
├── kubelet.conf
└── pki
├── apiserver.crt
├── apiserver-etcd-client.crt
├── apiserver-kubelet-client.crt
├── front-proxy-client.crt
└── etcd
├── healthcheck-client.crt
├── peer.crt
└── server.crt
'./update-kubeadm-cert.sh master' update only master certificates and kubeconf
/etc/kubernetes
├── admin.conf
├── controller-manager.conf
├── scheduler.conf
├── kubelet.conf
└── pki
├── apiserver.crt
├── apiserver-kubelet-client.crt
└── front-proxy-client.crt
调试思路：
1，调试脚本的时候需要暴露每一个变量，因此，脚本头设置了set -x。
2，,main函数是该脚本的主要函数，也是整个脚本的入口，因此，在main函数内增加了一个sleep 60 以作断点，脚本运行的时候会等待60秒，以方便观察每一个变量。
也就是说，不管哪个shell脚本，我们在调试的时候都需要打一个合理的断点，通常的断点是利用sleep这个内置函数。断点的位置一定要合理。
cat update-kube-cert-master/update-kubeadm-cert.sh
#!/usr/bin/env bash
set -x
set -o errexit
set -o pipefail
# set -o xtrace
# set output color
NC='\033[0m'
RED='\033[31m'
GREEN='\033[32m'
YELLOW='\033[33m'
BLUE='\033[34m'
log::err() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${RED}ERROR${NC}] %b\n" "$@"
}
log::info() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][INFO] %b\n" "$@"
}
log::warning() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${YELLOW}WARNING${NC}] \033[0m%b\n" "$@"
}
check_file() {
if [[ ! -r ${1} ]]; then
log::err "can not find ${1}"
exit 1
fi
}
# get x509v3 subject alternative name from the old certificate
cert::get_subject_alt_name() {
local cert=${1}.crt
local alt_name
check_file "${cert}"
alt_name=$(openssl x509 -text -noout -in "${cert}" | grep -A1 'Alternative' | tail -n1 | sed 's/[[:space:]]*Address//g')
printf "%s\n" "${alt_name}"
}
# get subject from the old certificate
cert::get_subj() {
local cert=${1}.crt
local subj
check_file "${cert}"
subj=$(openssl x509 -text -noout -in "${cert}" | grep "Subject:" | sed 's/Subject:/\//g;s/\,/\//;s/[[:space:]]//g')
printf "%s\n" "${subj}"
}
cert::backup_file() {
local file=${1}
if [[ ! -e ${file}.old-$(date +%Y%m%d) ]]; then
cp -rp "${file}" "${file}.old-$(date +%Y%m%d)"
log::info "backup ${file} to ${file}.old-$(date +%Y%m%d)"
else
log::warning "does not backup, ${file}.old-$(date +%Y%m%d) already exists"
fi
}
# check certificate expiration
cert::check_cert_expiration() {
local cert=${1}.crt
local cert_expires
cert_expires=$(openssl x509 -text -noout -in "${cert}" | awk -F ": " '/Not After/{print$2}')
printf "%s\n" "${cert_expires}"
}
# check kubeconfig expiration
cert::check_kubeconfig_expiration() {
local config=${1}.conf
local cert
local cert_expires
cert=$(grep "client-certificate-data" "${config}" | awk '{print$2}' | base64 -d)
cert_expires=$(openssl x509 -text -noout -in <(printf "%s" "${cert}") | awk -F ": " '/Not After/{print$2}')
printf "%s\n" "${cert_expires}"
}
# check etcd certificates expiration
cert::check_etcd_certs_expiration() {
local cert
local certs
certs=(
"${ETCD_CERT_CA}"
"${ETCD_CERT_SERVER}"
"${ETCD_CERT_PEER}"
"${ETCD_CERT_HEALTHCHECK_CLIENT}"
"${ETCD_CERT_APISERVER_ETCD_CLIENT}"
)
for cert in "${certs[@]}"; do
if [[ ! -r ${cert} ]]; then
printf "%-50s%-30s\n" "${cert}.crt" "$(cert::check_cert_expiration "${cert}")"
fi
done
}
# check master certificates expiration
cert::check_master_certs_expiration() {
local certs
local kubeconfs
local cert
local conf
certs=(
"${CERT_CA}"
"${CERT_APISERVER}"
"${CERT_APISERVER_KUBELET_CLIENT}"
"${FRONT_PROXY_CA}"
"${FRONT_PROXY_CLIENT}"
)
kubeconfs=(
"${CONF_CONTROLLER_MANAGER}"
"${CONF_SCHEDULER}"
"${CONF_ADMIN}"
)
printf "%-50s%-30s\n" "CERTIFICATE" "EXPIRES"
for conf in "${kubeconfs[@]}"; do
if [[ ! -r ${conf} ]]; then
printf "%-50s%-30s\n" "${conf}.config" "$(cert::check_kubeconfig_expiration "${conf}")"
fi
done
for cert in "${certs[@]}"; do
if [[ ! -r ${cert} ]]; then
printf "%-50s%-30s\n" "${cert}.crt" "$(cert::check_cert_expiration "${cert}")"
fi
done
}
# check all certificates expiration
cert::check_all_expiration() {
cert::check_master_certs_expiration
cert::check_etcd_certs_expiration
}
# generate certificate whit client, server or peer
# Args:
# $1 (the name of certificate)
# $2 (the type of certificate, must be one of client, server, peer)
# $3 (the subject of certificates)
# $4 (the validity of certificates) (days)
# $5 (the name of ca)
# $6 (the x509v3 subject alternative name of certificate when the type of certificate is server or peer)
cert::gen_cert() {
local cert_name=${1}
local cert_type=${2}
local subj=${3}
local cert_days=${4}
local ca_name=${5}
local alt_name=${6}
local ca_cert=${ca_name}.crt
local ca_key=${ca_name}.key
local cert=${cert_name}.crt
local key=${cert_name}.key
local csr=${cert_name}.csr
local common_csr_conf='distinguished_name = dn\n[dn]\n[v3_ext]\nkeyUsage = critical, digitalSignature, keyEncipherment\n'
for file in "${ca_cert}" "${ca_key}" "${cert}" "${key}"; do
check_file "${file}"
done
case "${cert_type}" in
client)
csr_conf=$(printf "%bextendedKeyUsage = clientAuth\n" "${common_csr_conf}")
;;
server)
csr_conf=$(printf "%bextendedKeyUsage = serverAuth\nsubjectAltName = %b\n" "${common_csr_conf}" "${alt_name}")
;;
peer)
csr_conf=$(printf "%bextendedKeyUsage = serverAuth, clientAuth\nsubjectAltName = %b\n" "${common_csr_conf}" "${alt_name}")
;;
*)
log::err "unknow, unsupported certs type: ${YELLOW}${cert_type}${NC}, supported type: client, server, peer"
exit 1
;;
esac
# gen csr
openssl req -new -key "${key}" -subj "${subj}" -reqexts v3_ext \
-config <(printf "%b" "${csr_conf}") \
-out "${csr}" >/dev/null 2>&1
# gen cert
openssl x509 -in "${csr}" -req -CA "${ca_cert}" -CAkey "${ca_key}" -CAcreateserial -extensions v3_ext \
-extfile <(printf "%b" "${csr_conf}") \
-days "${cert_days}" -out "${cert}" >/dev/null 2>&1
rm -f "${csr}"
}
cert::update_kubeconf() {
local cert_name=${1}
local kubeconf_file=${cert_name}.conf
local cert=${cert_name}.crt
local key=${cert_name}.key
local subj
local cert_base64
check_file "${kubeconf_file}"
# get the key from the old kubeconf
grep "client-key-data" "${kubeconf_file}" | awk '{print$2}' | base64 -d >"${key}"
# get the old certificate from the old kubeconf
grep "client-certificate-data" "${kubeconf_file}" | awk '{print$2}' | base64 -d >"${cert}"
# get subject from the old certificate
subj=$(cert::get_subj "${cert_name}")
cert::gen_cert "${cert_name}" "client" "${subj}" "${CERT_DAYS}" "${CERT_CA}"
# get certificate base64 code
cert_base64=$(base64 -w 0 "${cert}")
# set certificate base64 code to kubeconf
sed -i 's/client-certificate-data:.*/client-certificate-data: '"${cert_base64}"'/g' "${kubeconf_file}"
rm -f "${cert}"
rm -f "${key}"
}
cert::update_etcd_cert() {
local subj
local subject_alt_name
local cert
# generate etcd server,peer certificate
# /etc/kubernetes/pki/etcd/server
# /etc/kubernetes/pki/etcd/peer
for cert in ${ETCD_CERT_SERVER} ${ETCD_CERT_PEER}; do
subj=$(cert::get_subj "${cert}")
subject_alt_name=$(cert::get_subject_alt_name "${cert}")
cert::gen_cert "${cert}" "peer" "${subj}" "${CERT_DAYS}" "${ETCD_CERT_CA}" "${subject_alt_name}"
log::info "${GREEN}updated ${BLUE}${cert}.conf${NC}"
done
# generate etcd healthcheck-client,apiserver-etcd-client certificate
# /etc/kubernetes/pki/etcd/healthcheck-client
# /etc/kubernetes/pki/apiserver-etcd-client
for cert in ${ETCD_CERT_HEALTHCHECK_CLIENT} ${ETCD_CERT_APISERVER_ETCD_CLIENT}; do
subj=$(cert::get_subj "${cert}")
cert::gen_cert "${cert}" "client" "${subj}" "${CERT_DAYS}" "${ETCD_CERT_CA}"
log::info "${GREEN}updated ${BLUE}${cert}.conf${NC}"
done
# restart etcd
docker ps | awk '/k8s_etcd/{print$1}' | xargs -r -I '{}' docker restart {} >/dev/null 2>&1 || true
log::info "restarted etcd"
}
cert::update_master_cert() {
local subj
local subject_alt_name
local conf
# generate apiserver server certificate
# /etc/kubernetes/pki/apiserver
subj=$(cert::get_subj "${CERT_APISERVER}")
subject_alt_name=$(cert::get_subject_alt_name "${CERT_APISERVER}")
cert::gen_cert "${CERT_APISERVER}" "server" "${subj}" "${CERT_DAYS}" "${CERT_CA}" "${subject_alt_name}"
log::info "${GREEN}updated ${BLUE}${CERT_APISERVER}.crt${NC}"
# generate apiserver-kubelet-client certificate
# /etc/kubernetes/pki/apiserver-kubelet-client
subj=$(cert::get_subj "${CERT_APISERVER_KUBELET_CLIENT}")
cert::gen_cert "${CERT_APISERVER_KUBELET_CLIENT}" "client" "${subj}" "${CERT_DAYS}" "${CERT_CA}"
log::info "${GREEN}updated ${BLUE}${CERT_APISERVER_KUBELET_CLIENT}.crt${NC}"
# generate kubeconf for controller-manager,scheduler and kubelet
# /etc/kubernetes/controller-manager,scheduler,admin,kubelet.conf
for conf in ${CONF_CONTROLLER_MANAGER} ${CONF_SCHEDULER} ${CONF_ADMIN} ${CONF_KUBELET}; do
if [[ ${conf##*/} == "kubelet" ]]; then
# https://github.com/kubernetes/kubeadm/issues/1753
set +e
grep kubelet-client-current.pem /etc/kubernetes/kubelet.conf >/dev/null 2>&1
kubelet_cert_auto_update=$?
set -e
if [[ "$kubelet_cert_auto_update" == "0" ]]; then
log::info "does not need to update kubelet.conf"
continue
fi
fi
# update kubeconf
cert::update_kubeconf "${conf}"
log::info "${GREEN}updated ${BLUE}${conf}.conf${NC}"
# copy admin.conf to ${HOME}/.kube/config
if [[ ${conf##*/} == "admin" ]]; then
mkdir -p "${HOME}/.kube"
local config=${HOME}/.kube/config
local config_backup
config_backup=${HOME}/.kube/config.old-$(date +%Y%m%d)
if [[ -f ${config} ]] && [[ ! -f ${config_backup} ]]; then
cp -fp "${config}" "${config_backup}"
log::info "backup ${config} to ${config_backup}"
fi
cp -fp "${conf}.conf" "${HOME}/.kube/config"
log::info "copy the admin.conf to ${HOME}/.kube/config"
fi
done
# generate front-proxy-client certificate
# /etc/kubernetes/pki/front-proxy-client
subj=$(cert::get_subj "${FRONT_PROXY_CLIENT}")
cert::gen_cert "${FRONT_PROXY_CLIENT}" "client" "${subj}" "${CERT_DAYS}" "${FRONT_PROXY_CA}"
log::info "${GREEN}updated ${BLUE}${FRONT_PROXY_CLIENT}.crt${NC}"
# restart apiserver, controller-manager, scheduler and kubelet
for item in "apiserver" "controller-manager" "scheduler"; do
docker ps | awk '/k8s_kube-'${item}'/{print$1}' | xargs -r -I '{}' docker restart {} >/dev/null 2>&1 || true
log::info "restarted ${item}"
done
systemctl restart kubelet || true
log::info "restarted kubelet"
}
main() {
local node_type=$1
CERT_DAYS=3650
KUBE_PATH=/etc/kubernetes
PKI_PATH=${KUBE_PATH}/pki
# master certificates path
# apiserver
CERT_CA=${PKI_PATH}/ca
CERT_APISERVER=${PKI_PATH}/apiserver
CERT_APISERVER_KUBELET_CLIENT=${PKI_PATH}/apiserver-kubelet-client
CONF_CONTROLLER_MANAGER=${KUBE_PATH}/controller-manager
CONF_SCHEDULER=${KUBE_PATH}/scheduler
CONF_ADMIN=${KUBE_PATH}/admin
CONF_KUBELET=${KUBE_PATH}/kubelet
# front-proxy
FRONT_PROXY_CA=${PKI_PATH}/front-proxy-ca
FRONT_PROXY_CLIENT=${PKI_PATH}/front-proxy-client
# etcd certificates path
ETCD_CERT_CA=${PKI_PATH}/etcd/ca
ETCD_CERT_SERVER=${PKI_PATH}/etcd/server
ETCD_CERT_PEER=${PKI_PATH}/etcd/peer
ETCD_CERT_HEALTHCHECK_CLIENT=${PKI_PATH}/etcd/healthcheck-client
ETCD_CERT_APISERVER_ETCD_CLIENT=${PKI_PATH}/apiserver-etcd-client
case ${node_type} in
# etcd)
# # update etcd certificates
# cert::update_etcd_cert
# ;;
master)
# check certificates expiration
cert::check_master_certs_expiration
# backup $KUBE_PATH to $KUBE_PATH.old-$(date +%Y%m%d)
cert::backup_file "${KUBE_PATH}"
# update master certificates and kubeconf
log::info "${GREEN}updating...${NC}"
cert::update_master_cert
log::info "${GREEN}done!!!${NC}"
# check certificates expiration after certificates updated
cert::check_master_certs_expiration
;;
all)
# check certificates expiration
cert::check_all_expiration
sleep 60
# backup $KUBE_PATH to $KUBE_PATH.old-$(date +%Y%m%d)
cert::backup_file "${KUBE_PATH}"
# update etcd certificates
log::info "${GREEN}updating...${NC}"
cert::update_etcd_cert
# update master certificates and kubeconf
cert::update_master_cert
log::info "${GREEN}done!!!${NC}"
# check certificates expiration after certificates updated
cert::check_all_expiration
;;
check)
# check certificates expiration
cert::check_all_expiration
;;
*)
log::err "unknown, unsupported cert type: ${node_type}, supported type: \"all\", \"master\""
printf "Documentation: https://github.com/yuyicai/update-kube-cert
example:
'\033[32m./update-kubeadm-cert.sh all\033[0m' update all etcd certificates, master certificates and kubeconf
/etc/kubernetes
├── admin.conf
├── controller-manager.conf
├── scheduler.conf
├── kubelet.conf
└── pki
├── apiserver.crt
├── apiserver-etcd-client.crt
├── apiserver-kubelet-client.crt
├── front-proxy-client.crt
└── etcd
├── healthcheck-client.crt
├── peer.crt
└── server.crt
'\033[32m./update-kubeadm-cert.sh master\033[0m' update only master certificates and kubeconf
/etc/kubernetes
├── admin.conf
├── controller-manager.conf
├── scheduler.conf
├── kubelet.conf
└── pki
├── apiserver.crt
├── apiserver-kubelet-client.crt
└── front-proxy-client.crt
"
exit 1
;;
esac
}
main "$@"
这里使用的参数是all，因此，在all参数下的第一个函数打断点sleep 60，如果是使用master参数，那么，应该在master下的第一个函数打断点。
该脚本的调试命令是：
bash update-kube-cert-master/update-kubeadm-cert.sh all
输出的日志如下：
可以看到，我们使用的参数是all，那么，如果想让该脚本更新证书不是10年而是100年，自然需要将变量CERT_DAYS=3650修改为CERT_DAYS=36500了。
[root@node4 ~]# bash update-kube-cert-master/update-kubeadm-cert.sh all
+ set -o errexit
+ set -o pipefail
+ NC='\033[0m'
+ RED='\033[31m'
+ GREEN='\033[32m'
+ YELLOW='\033[33m'
+ BLUE='\033[34m'
+ main all
+ local node_type=all
+ CERT_DAYS=3650
+ KUBE_PATH=/etc/kubernetes
+ PKI_PATH=/etc/kubernetes/pki
+ CERT_CA=/etc/kubernetes/pki/ca
+ CERT_APISERVER=/etc/kubernetes/pki/apiserver
+ CERT_APISERVER_KUBELET_CLIENT=/etc/kubernetes/pki/apiserver-kubelet-client
+ CONF_CONTROLLER_MANAGER=/etc/kubernetes/controller-manager
+ CONF_SCHEDULER=/etc/kubernetes/scheduler
+ CONF_ADMIN=/etc/kubernetes/admin
+ CONF_KUBELET=/etc/kubernetes/kubelet
+ FRONT_PROXY_CA=/etc/kubernetes/pki/front-proxy-ca
+ FRONT_PROXY_CLIENT=/etc/kubernetes/pki/front-proxy-client
+ ETCD_CERT_CA=/etc/kubernetes/pki/etcd/ca
+ ETCD_CERT_SERVER=/etc/kubernetes/pki/etcd/server
+ ETCD_CERT_PEER=/etc/kubernetes/pki/etcd/peer
+ ETCD_CERT_HEALTHCHECK_CLIENT=/etc/kubernetes/pki/etcd/healthcheck-client
+ ETCD_CERT_APISERVER_ETCD_CLIENT=/etc/kubernetes/pki/apiserver-etcd-client
+ case ${node_type} in
+ cert::check_all_expiration
+ cert::check_master_certs_expiration
+ local certs
+ local kubeconfs
+ local cert
+ local conf
+ certs=("${CERT_CA}" "${CERT_APISERVER}" "${CERT_APISERVER_KUBELET_CLIENT}" "${FRONT_PROXY_CA}" "${FRONT_PROXY_CLIENT}")
+ kubeconfs=("${CONF_CONTROLLER_MANAGER}" "${CONF_SCHEDULER}" "${CONF_ADMIN}")
+ printf '%-50s%-30s\n' CERTIFICATE EXPIRES
CERTIFICATE EXPIRES
+ for conf in '"${kubeconfs[@]}"'
+ [[ ! -r /etc/kubernetes/controller-manager ]]
++ cert::check_kubeconfig_expiration /etc/kubernetes/controller-manager
++ local config=/etc/kubernetes/controller-manager.conf
++ local cert
++ local cert_expires
+++ grep client-certificate-data /etc/kubernetes/controller-manager.conf
+++ base64 -d
+++ awk '{print$2}'
++ cert='-----BEGIN CERTIFICATE-----
MIIC6DCCAdCgAwIBAgIJANAYEYivHZr6MA0GCSqGSIb3DQEBCwUAMBUxEzARBgNV
BAMTCmt1YmVybmV0ZXMwHhcNMjIxMjA4MTMyMjIwWhcNMzIxMjA1MTMyMjIwWjAp
MScwJQYDVQQDDB5zeXN0ZW06a3ViZS1jb250cm9sbGVyLW1hbmFnZXIwggEiMA0G
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDbCNdZWfw9BjCkkCDt1DvZUwp9Xn41
9K+Ijy2kelms5KW+qRF9mTChEtjIJG1jO+ul1cx2XBgi+x9G3R2ErPe3rX9tt0wu
3MK7BZC8KRzlAK8dRSkCL+28PsHpr+mcQxYNNZnpEHyyCahaVy2IfGtb4xBCp2CS
AaEUIXJkI4pjLIKjv11NRi3oBJbUh8CyUms/5RFhXxqUfsX7u85+NcXku1h429Wz
cRSF9PgSHuPoBiiJdtrHbQyErd0mLJEZxgLz9MEc2YiNSL6ORoV8Kfxba1Z05c1W
0waNxCrHA9uqO3NHMM9dagJhS3A5vyczvRgm6dVMrvDJFvvKaf9StZVNAgMBAAGj
JzAlMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDAjANBgkqhkiG
9w0BAQsFAAOCAQEAOsz8bmnrUdpOAigDZWI8gPwQcqmP82RxLLUXakhwo6v31AEk
tQETDb9ypXo99/6KHXLKc0g0jRTnOPywqIqOvDRXbzsWHQOamvcyP0xnVY4VSwVg
3J+yR04sgS6XjN42AymR2IUFzZDY5g7bJYH0+LHFTqlkiI1vt7vdoWrpKU9XUmxP
kv5lLVI2DgvMosgcvib6c5JakpzZgw4jnhYI2bulAReYMdHSMBS/ZRTNKuggWXQj
IejnirR+JUeZYKkVJqAGXUGiMksNgE4P79N+mcAtvFhH8oolGCAXATH7tRJ74odd
Q2azcimFOuifoDYveer/vbGH8pjsKveesORTVw==
-----END CERTIFICATE-----'
+++ openssl x509 -text -noout -in /dev/fd/63
++++ printf %s '-----BEGIN CERTIFICATE-----
MIIC6DCCAdCgAwIBAgIJANAYEYivHZr6MA0GCSqGSIb3DQEBCwUAMBUxEzARBgNV
BAMTCmt1YmVybmV0ZXMwHhcNMjIxMjA4MTMyMjIwWhcNMzIxMjA1MTMyMjIwWjAp
MScwJQYDVQQDDB5zeXN0ZW06a3ViZS1jb250cm9sbGVyLW1hbmFnZXIwggEiMA0G
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDbCNdZWfw9BjCkkCDt1DvZUwp9Xn41
9K+Ijy2kelms5KW+qRF9mTChEtjIJG1jO+ul1cx2XBgi+x9G3R2ErPe3rX9tt0wu
3MK7BZC8KRzlAK8dRSkCL+28PsHpr+mcQxYNNZnpEHyyCahaVy2IfGtb4xBCp2CS
AaEUIXJkI4pjLIKjv11NRi3oBJbUh8CyUms/5RFhXxqUfsX7u85+NcXku1h429Wz
cRSF9PgSHuPoBiiJdtrHbQyErd0mLJEZxgLz9MEc2YiNSL6ORoV8Kfxba1Z05c1W
0waNxCrHA9uqO3NHMM9dagJhS3A5vyczvRgm6dVMrvDJFvvKaf9StZVNAgMBAAGj
JzAlMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDAjANBgkqhkiG
9w0BAQsFAAOCAQEAOsz8bmnrUdpOAigDZWI8gPwQcqmP82RxLLUXakhwo6v31AEk
tQETDb9ypXo99/6KHXLKc0g0jRTnOPywqIqOvDRXbzsWHQOamvcyP0xnVY4VSwVg
3J+yR04sgS6XjN42AymR2IUFzZDY5g7bJYH0+LHFTqlkiI1vt7vdoWrpKU9XUmxP
kv5lLVI2DgvMosgcvib6c5JakpzZgw4jnhYI2bulAReYMdHSMBS/ZRTNKuggWXQj
IejnirR+JUeZYKkVJqAGXUGiMksNgE4P79N+mcAtvFhH8oolGCAXATH7tRJ74odd
Q2azcimFOuifoDYveer/vbGH8pjsKveesORTVw==
-----END CERTIFICATE-----'
+++ awk -F ': ' '/Not After/{print$2}'
++ cert_expires='Dec 5 13:22:20 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:20 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/controller-manager.config 'Dec 5 13:22:20 2032 GMT'
/etc/kubernetes/controller-manager.config Dec 5 13:22:20 2032 GMT
+ for conf in '"${kubeconfs[@]}"'
+ [[ ! -r /etc/kubernetes/scheduler ]]
++ cert::check_kubeconfig_expiration /etc/kubernetes/scheduler
++ local config=/etc/kubernetes/scheduler.conf
++ local cert
++ local cert_expires
+++ grep client-certificate-data /etc/kubernetes/scheduler.conf
+++ base64 -d
+++ awk '{print$2}'
++ cert='-----BEGIN CERTIFICATE-----
MIIC3zCCAcegAwIBAgIJANAYEYivHZr7MA0GCSqGSIb3DQEBCwUAMBUxEzARBgNV
BAMTCmt1YmVybmV0ZXMwHhcNMjIxMjA4MTMyMjIwWhcNMzIxMjA1MTMyMjIwWjAg
MR4wHAYDVQQDDBVzeXN0ZW06a3ViZS1zY2hlZHVsZXIwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQCuQkja6VTRgUKRXb20HmMw+kABEGlr5z4zZDxYRRcu
FKhEDBLr3neIQLd4u+dehhW70QelAEK+1Z9tiu3dWxmnI5Z0W2dkurtdtyN7QMux
aknsmZKaeMlgdMR0e01AIuESlSOWsBQqnPtiqpbuQgSe5G0YwfXxQ/Jj22JuUB9t
P7sAEpFEBl2xDWlDWeNUajNx65R6YfjolSGwHp19WHSuA3Ed+DLpQAFTEaIMAOjf
YFNcXhJDEHdn+LYjs2oaQTzurGu7EhHoU5Y7ka6wos7PLRyrC8xDskbnh8pvQKpP
Z2oFk71TxLH5nMy3ENnimSEIPJM021Zm+gSN8JR7dlWbAgMBAAGjJzAlMA4GA1Ud
DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDAjANBgkqhkiG9w0BAQsFAAOC
AQEAZx1FiXfJgac58S3gULr1ICrIsl0Cn6vNJ2XKH8B3ZGucaFENEE8tq0auHcEn
lcN/bbPwO4TPuXNTBSE9M4F267I0/ybuFg54tnW+4jQUa2rRobo5GWVKvbQJk3cb
OLX2vbT5S2ctvqcYsHqEHTPivPug7APcfkGnWzp64p/7FiFhMrcX9R8+MhJDT4/G
CU1BEL2CypeVpgbwaZe84xoLxdBqHJ+PxxN7pfChWwPfP8ZqVmSv6ugURTJr5vRS
yejsgc9T19TaYbLBv8skR/EStHo6O+1gFLSC1h5hks0TYGF8k5IBlFJlNGoFlJgg
/rX0FKv0EHsUeaonDDDZFxdXew==
-----END CERTIFICATE-----'
+++ awk -F ': ' '/Not After/{print$2}'
+++ openssl x509 -text -noout -in /dev/fd/63
++++ printf %s '-----BEGIN CERTIFICATE-----
MIIC3zCCAcegAwIBAgIJANAYEYivHZr7MA0GCSqGSIb3DQEBCwUAMBUxEzARBgNV
BAMTCmt1YmVybmV0ZXMwHhcNMjIxMjA4MTMyMjIwWhcNMzIxMjA1MTMyMjIwWjAg
MR4wHAYDVQQDDBVzeXN0ZW06a3ViZS1zY2hlZHVsZXIwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQCuQkja6VTRgUKRXb20HmMw+kABEGlr5z4zZDxYRRcu
FKhEDBLr3neIQLd4u+dehhW70QelAEK+1Z9tiu3dWxmnI5Z0W2dkurtdtyN7QMux
aknsmZKaeMlgdMR0e01AIuESlSOWsBQqnPtiqpbuQgSe5G0YwfXxQ/Jj22JuUB9t
P7sAEpFEBl2xDWlDWeNUajNx65R6YfjolSGwHp19WHSuA3Ed+DLpQAFTEaIMAOjf
YFNcXhJDEHdn+LYjs2oaQTzurGu7EhHoU5Y7ka6wos7PLRyrC8xDskbnh8pvQKpP
Z2oFk71TxLH5nMy3ENnimSEIPJM021Zm+gSN8JR7dlWbAgMBAAGjJzAlMA4GA1Ud
DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDAjANBgkqhkiG9w0BAQsFAAOC
AQEAZx1FiXfJgac58S3gULr1ICrIsl0Cn6vNJ2XKH8B3ZGucaFENEE8tq0auHcEn
lcN/bbPwO4TPuXNTBSE9M4F267I0/ybuFg54tnW+4jQUa2rRobo5GWVKvbQJk3cb
OLX2vbT5S2ctvqcYsHqEHTPivPug7APcfkGnWzp64p/7FiFhMrcX9R8+MhJDT4/G
CU1BEL2CypeVpgbwaZe84xoLxdBqHJ+PxxN7pfChWwPfP8ZqVmSv6ugURTJr5vRS
yejsgc9T19TaYbLBv8skR/EStHo6O+1gFLSC1h5hks0TYGF8k5IBlFJlNGoFlJgg
/rX0FKv0EHsUeaonDDDZFxdXew==
-----END CERTIFICATE-----'
++ cert_expires='Dec 5 13:22:20 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:20 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/scheduler.config 'Dec 5 13:22:20 2032 GMT'
/etc/kubernetes/scheduler.config Dec 5 13:22:20 2032 GMT
+ for conf in '"${kubeconfs[@]}"'
+ [[ ! -r /etc/kubernetes/admin ]]
++ cert::check_kubeconfig_expiration /etc/kubernetes/admin
++ local config=/etc/kubernetes/admin.conf
++ local cert
++ local cert_expires
+++ grep client-certificate-data /etc/kubernetes/admin.conf
+++ base64 -d
+++ awk '{print$2}'
++ cert='-----BEGIN CERTIFICATE-----
MIIC8zCCAdugAwIBAgIJANAYEYivHZr8MA0GCSqGSIb3DQEBCwUAMBUxEzARBgNV
BAMTCmt1YmVybmV0ZXMwHhcNMjIxMjA4MTMyMjIwWhcNMzIxMjA1MTMyMjIwWjA0
MRcwFQYDVQQKDA5zeXN0ZW06bWFzdGVyczEZMBcGA1UEAwwQa3ViZXJuZXRlcy1h
ZG1pbjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMVJbVqTIzMU7DLL
CoOV6IW3vVjYIg8EnytEHRH7yJ1SnTxdq9FEwNKjW+bZk5vYygEswBkjgHQlGNdu
BJIma3lPTNCifDykV/kb1kV8o5eU5b6/A8nRr/tpIeUxTCK2T9v6Ep91KgflNzFk
AM+VNp3XWwVV4pBBeX6IXBBCmQdjyJ9qzeOZZGyP5aL/Q4JCw6v1FlHb+9Ja2fTB
NCSH3j57618KGXqfcAmUex+Si1gfou7OuqmUDjYbSGDGMFucQkEdkx5bK5QQRtRw
QfdEq9bB06iYdJng2uvq6GeW+0cx8ef/56AY2CCJjAIzpOtb2yOfIt6PkVYiLvyz
0a3wZ8sCAwEAAaMnMCUwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUF
BwMCMA0GCSqGSIb3DQEBCwUAA4IBAQCRD0+WujKggavu8tgzWlR93Wh2zoDz/Rkd
gFsErhjWuzEwUFjNNhIBlPzOubjoVvzG0KQEImIsSBTbmV5hShOwocqYAA1uRFW5
t11dbeTF5HqZbtmbhJ0y26SdTFIRD0xEydq91P6lB5TqEbis/+Il7y/hGvZKtRcB
Gf87fk5zdQlxAg5nx2i+wj73yYcQFlgEu+2RFo8XejEbqbkrQ470DA5W+TNpPbEH
rMmGsh6EeFP5hnntxkvAf7kBgirKbXfTpPveKP7yT/aiUB1KCuV4ZCSRWrA9sBvJ
hwbZFI2LriP6nlJh0IIiyoygS/G3IFQ1ner0jNIsYJNY4LmZKQD5
-----END CERTIFICATE-----'
+++ awk -F ': ' '/Not After/{print$2}'
+++ openssl x509 -text -noout -in /dev/fd/63
++++ printf %s '-----BEGIN CERTIFICATE-----
MIIC8zCCAdugAwIBAgIJANAYEYivHZr8MA0GCSqGSIb3DQEBCwUAMBUxEzARBgNV
BAMTCmt1YmVybmV0ZXMwHhcNMjIxMjA4MTMyMjIwWhcNMzIxMjA1MTMyMjIwWjA0
MRcwFQYDVQQKDA5zeXN0ZW06bWFzdGVyczEZMBcGA1UEAwwQa3ViZXJuZXRlcy1h
ZG1pbjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMVJbVqTIzMU7DLL
CoOV6IW3vVjYIg8EnytEHRH7yJ1SnTxdq9FEwNKjW+bZk5vYygEswBkjgHQlGNdu
BJIma3lPTNCifDykV/kb1kV8o5eU5b6/A8nRr/tpIeUxTCK2T9v6Ep91KgflNzFk
AM+VNp3XWwVV4pBBeX6IXBBCmQdjyJ9qzeOZZGyP5aL/Q4JCw6v1FlHb+9Ja2fTB
NCSH3j57618KGXqfcAmUex+Si1gfou7OuqmUDjYbSGDGMFucQkEdkx5bK5QQRtRw
QfdEq9bB06iYdJng2uvq6GeW+0cx8ef/56AY2CCJjAIzpOtb2yOfIt6PkVYiLvyz
0a3wZ8sCAwEAAaMnMCUwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUF
BwMCMA0GCSqGSIb3DQEBCwUAA4IBAQCRD0+WujKggavu8tgzWlR93Wh2zoDz/Rkd
gFsErhjWuzEwUFjNNhIBlPzOubjoVvzG0KQEImIsSBTbmV5hShOwocqYAA1uRFW5
t11dbeTF5HqZbtmbhJ0y26SdTFIRD0xEydq91P6lB5TqEbis/+Il7y/hGvZKtRcB
Gf87fk5zdQlxAg5nx2i+wj73yYcQFlgEu+2RFo8XejEbqbkrQ470DA5W+TNpPbEH
rMmGsh6EeFP5hnntxkvAf7kBgirKbXfTpPveKP7yT/aiUB1KCuV4ZCSRWrA9sBvJ
hwbZFI2LriP6nlJh0IIiyoygS/G3IFQ1ner0jNIsYJNY4LmZKQD5
-----END CERTIFICATE-----'
++ cert_expires='Dec 5 13:22:20 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:20 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/admin.config 'Dec 5 13:22:20 2032 GMT'
/etc/kubernetes/admin.config Dec 5 13:22:20 2032 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/ca ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/ca
++ local cert=/etc/kubernetes/pki/ca.crt
++ local cert_expires
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/ca.crt
+++ awk -F ': ' '/Not After/{print$2}'
++ cert_expires='Nov 12 07:26:23 2122 GMT'
++ printf '%s\n' 'Nov 12 07:26:23 2122 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/ca.crt 'Nov 12 07:26:23 2122 GMT'
/etc/kubernetes/pki/ca.crt Nov 12 07:26:23 2122 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/apiserver ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/apiserver
++ local cert=/etc/kubernetes/pki/apiserver.crt
++ local cert_expires
+++ awk -F ': ' '/Not After/{print$2}'
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/apiserver.crt
++ cert_expires='Dec 5 13:22:20 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:20 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/apiserver.crt 'Dec 5 13:22:20 2032 GMT'
/etc/kubernetes/pki/apiserver.crt Dec 5 13:22:20 2032 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/apiserver-kubelet-client ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/apiserver-kubelet-client
++ local cert=/etc/kubernetes/pki/apiserver-kubelet-client.crt
++ local cert_expires
+++ awk -F ': ' '/Not After/{print$2}'
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/apiserver-kubelet-client.crt
++ cert_expires='Dec 5 13:22:20 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:20 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/apiserver-kubelet-client.crt 'Dec 5 13:22:20 2032 GMT'
/etc/kubernetes/pki/apiserver-kubelet-client.crt Dec 5 13:22:20 2032 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/front-proxy-ca ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/front-proxy-ca
++ local cert=/etc/kubernetes/pki/front-proxy-ca.crt
++ local cert_expires
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/front-proxy-ca.crt
+++ awk -F ': ' '/Not After/{print$2}'
++ cert_expires='Nov 12 07:26:25 2122 GMT'
++ printf '%s\n' 'Nov 12 07:26:25 2122 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/front-proxy-ca.crt 'Nov 12 07:26:25 2122 GMT'
/etc/kubernetes/pki/front-proxy-ca.crt Nov 12 07:26:25 2122 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/front-proxy-client ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/front-proxy-client
++ local cert=/etc/kubernetes/pki/front-proxy-client.crt
++ local cert_expires
+++ awk -F ': ' '/Not After/{print$2}'
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/front-proxy-client.crt
++ cert_expires='Dec 5 13:22:20 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:20 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/front-proxy-client.crt 'Dec 5 13:22:20 2032 GMT'
/etc/kubernetes/pki/front-proxy-client.crt Dec 5 13:22:20 2032 GMT
+ cert::check_etcd_certs_expiration
+ local cert
+ local certs
+ certs=("${ETCD_CERT_CA}" "${ETCD_CERT_SERVER}" "${ETCD_CERT_PEER}" "${ETCD_CERT_HEALTHCHECK_CLIENT}" "${ETCD_CERT_APISERVER_ETCD_CLIENT}")
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/etcd/ca ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/etcd/ca
++ local cert=/etc/kubernetes/pki/etcd/ca.crt
++ local cert_expires
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/etcd/ca.crt
+++ awk -F ': ' '/Not After/{print$2}'
++ cert_expires='Nov 12 07:26:26 2122 GMT'
++ printf '%s\n' 'Nov 12 07:26:26 2122 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/etcd/ca.crt 'Nov 12 07:26:26 2122 GMT'
/etc/kubernetes/pki/etcd/ca.crt Nov 12 07:26:26 2122 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/etcd/server ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/etcd/server
++ local cert=/etc/kubernetes/pki/etcd/server.crt
++ local cert_expires
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/etcd/server.crt
+++ awk -F ': ' '/Not After/{print$2}'
++ cert_expires='Dec 5 13:22:18 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:18 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/etcd/server.crt 'Dec 5 13:22:18 2032 GMT'
/etc/kubernetes/pki/etcd/server.crt Dec 5 13:22:18 2032 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/etcd/peer ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/etcd/peer
++ local cert=/etc/kubernetes/pki/etcd/peer.crt
++ local cert_expires
+++ awk -F ': ' '/Not After/{print$2}'
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/etcd/peer.crt
++ cert_expires='Dec 5 13:22:18 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:18 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/etcd/peer.crt 'Dec 5 13:22:18 2032 GMT'
/etc/kubernetes/pki/etcd/peer.crt Dec 5 13:22:18 2032 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/etcd/healthcheck-client ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/etcd/healthcheck-client
++ local cert=/etc/kubernetes/pki/etcd/healthcheck-client.crt
++ local cert_expires
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/etcd/healthcheck-client.crt
+++ awk -F ': ' '/Not After/{print$2}'
++ cert_expires='Dec 5 13:22:18 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:18 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/etcd/healthcheck-client.crt 'Dec 5 13:22:18 2032 GMT'
/etc/kubernetes/pki/etcd/healthcheck-client.crt Dec 5 13:22:18 2032 GMT
+ for cert in '"${certs[@]}"'
+ [[ ! -r /etc/kubernetes/pki/apiserver-etcd-client ]]
++ cert::check_cert_expiration /etc/kubernetes/pki/apiserver-etcd-client
++ local cert=/etc/kubernetes/pki/apiserver-etcd-client.crt
++ local cert_expires
+++ openssl x509 -text -noout -in /etc/kubernetes/pki/apiserver-etcd-client.crt
+++ awk -F ': ' '/Not After/{print$2}'
++ cert_expires='Dec 5 13:22:19 2032 GMT'
++ printf '%s\n' 'Dec 5 13:22:19 2032 GMT'
+ printf '%-50s%-30s\n' /etc/kubernetes/pki/apiserver-etcd-client.crt 'Dec 5 13:22:19 2032 GMT'
/etc/kubernetes/pki/apiserver-etcd-client.crt Dec 5 13:22:19 2032 GMT
+ sleep 60
观察以上的输出，我们可以发现另外两个关键的变量：
KUBE_PATH=/etc/kubernetes
PKI_PATH=${KUBE_PATH}/pki
因此，我们即使不执行此脚本，也能知道该脚本更新以及备份kubernetes集群的范围只能是kubeadm方式部署的集群。
那么，这个脚本的意义在于可以把它里面的备份功能和重启服务功能独立出来。
这里先说一下脚本的重启服务功能，此功能是通过docker 重启容器的方式来变相的实现重启服务，因为我们都知道，kubeadm的关键服务都是静态pod的形式部署的嘛，自然是pod的容器重启了，服务就等于重启了。
这个服务重启的功能是证书更新的基础，如果证书更新完毕了，服务还没有重启，那么自然的，证书等于没有更新。
kubernetes集群核心服务重启脚本：
OK，服务重启的功能代码如下（这个是一个独立的脚本，当然了，适用于所有的kubeadm部署的集群，执行此脚本将会重启kubernetes集群的三个核心服务）：
#!/bin/bash
log::info() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][INFO] %b\n" "$@"
}
for item in "apiserver" "controller-manager" "scheduler"; do
docker ps | awk '/k8s_kube-'${item}'/{print$1}' | xargs -r -I '{}' docker restart {} >/dev/null 2>&1 || true
log::info "restarted ${item}"
done
执行此脚本后的输出如下；
[root@node4 ~]# bash 3333.sh
[2022-12-09T01:18:11.97+0800][INFO] restarted apiserver
[2022-12-09T01:18:12.10+0800][INFO] restarted controller-manager
[2022-12-09T01:18:12.20+0800][INFO] restarted scheduler
如何证明这三个服务重启了呢？这个比较简单，我们观察pod的状态即可，容器每重启一次，restarts都会增加1：
[root@node4 ~]# kubectl get po -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default nginx-6768c68f7b-stz8j 1/1 Running 1 (8h ago) 13h
kube-flannel kube-flannel-ds-wrg65 1/1 Running 1 (8h ago) 13h
kube-system coredns-c676cc86f-6ftmz 1/1 Running 2 (8h ago) 2d9h
kube-system coredns-c676cc86f-zwlr2 1/1 Running 2 (8h ago) 2d9h
kube-system etcd-node4 1/1 Running 3 (8h ago) 2d9h
kube-system kube-apiserver-node4 1/1 Running 3 (8h ago) 2d9h
kube-system kube-controller-manager-node4 1/1 Running 7 (3m39s ago) 2d9h
kube-system kube-flannel-ds-gnqt2 0/1 CrashLoopBackOff 120 (36s ago) 14h
kube-system kube-proxy-9dqdv 1/1 Running 3 (8h ago) 2d9h
kube-system kube-scheduler-node4 1/1 Running 7 (3m39s ago) 2d9h
kubernetes集群证书备份功能的提取：
当然了，我这里做了一点优化，让输出更加好看一点，备份成功日志等级为info，整体为蓝色，如果有备份，日志等级为warning，输出带有红色。
可能我的美工不是太好，大家有兴趣可以自己调整输出的颜色。
#!/bin/bash
#!author zsk
set +x
set -o errexit
set -o pipefail
NC='\033[0m'
RED='\033[31m'
GREEN='\033[32m'
YELLOW='\033[33m'
BLUE='\033[34m'
KUBE_PATH=/etc/kubernetes
log::info() {
printf "[${BLUE}$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${YELLOW}INFO${NC}] %b\n" "$@"
}
log::warning() {
printf "[$(date +'%Y-%m-%dT%H:%M:%S.%2N%z')][${RED}WARNING${NC}] \033[0m%b\n" "$@"
}
cert::backup_file() {
local file=${1}
if [[ ! -e ${file}.old-$(date +%Y%m%d) ]]; then
cp -rp "${file}" "${file}.old-$(date +%Y%m%d)"
log::info "${BLUE}backup ${file} to ${file}.old-$(date +%Y%m%d${NC})"
else
log::warning "does not backup, ${file}.old-$(date +%Y%m%d) already exists"
fi
}
cert::backup_file "${KUBE_PATH}"
脚本执行的输出：
查看脚本运行成果，可以看到确实是智能的备份好了：
[root@node4 pki]# pwd
/etc/kubernetes.old-20221209/pki
[root@node4 pki]# ls
apiserver.crt apiserver-etcd-client.key apiserver-kubelet-client.crt ca.crt ca.srl front-proxy-ca.crt front-proxy-ca.srl front-proxy-client.key sa.pub
apiserver-etcd-client.crt apiserver.key apiserver-kubelet-client.key ca.key etcd front-proxy-ca.key front-proxy-client.crt sa.key
当然了，这么一个脚本自然可以修改一哈，比如，MySQL数据库的物理文件备份，etcd数据库的物理文件备份等等都可以适用，如何改造我就不献丑了，大家可以发挥一哈聪明才智哦。
![Leetcode 1691. 堆叠长方体的最大高度 [Java/C++] 排序+动态规划(附详细证明过程）](https://img-blog.csdnimg.cn/db9cab8b3df74b89951f3aad8e551a6e.png)
