文章目录
- 高可用集群KEEPALIVED
- VRRP
- keepalived 部署
- 环境准备
- 开启通信功能
- 设置独立日志
- 设置独立子配置文件
- 抢占模式和非抢占模式
- 延迟抢占
- 非抢占模式
- 单播模式
- 邮箱
- 邮件通知脚本
- 双主架构
- 实现ipvs的高可用性
- lvs-dr
- VRRP Script
- 实现HAProxy高可用
文章相关连接如下:
- 如果想深入了解keepalived配置语法详情的相关知识,请点击:高可用集群KEEPALIVED的配置语法说明
高可用集群KEEPALIVED
keep alived
集群类型:
- LB:Load Balance负载均衡
LVS/HAProxy/nginx (http/upstream,stream/upstream) - HA:High Availability高可用集群 数据库、Redis
- SPoF:Single Point of Failure,解决单点故障
- HPC:High Performance Computing高性能集群
实现高可用:
提升系统高用性的解决方案:降低MTTR-Mean Time To Repair(平均故障时间)
解决方案:建立冗余机制
- active/passive主/备
- active/active双主
- active–>HEARTBEAT --passive
- active <–HEARTBEAT <–active
VRRP
VRRP:Virtual Router Redundancy Protocol虚拟路由冗余协议,解决静态网关单点风险
VRRP相关技术:
通告:心跳,优先级等;周期性
工作方式:抢占式,非抢占式
安全认证:
- 无认证
- 简单字符认证:预共享密钥
- MD5
工作模式:
- 主/备:单虚拟路由器
- 主/主:主/备(虚拟路由器1),备/主(虚拟路由器2)
keepalived 部署
解决VIP的问题
vrrp协议的软件实现,原生设计目的为了高可用ipvs服务
功能:
- 基于vrrp协议完成地址流动
- 为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)—解决lvs策略
- 为ipvs集群的各RS做健康状态检测
- 基于脚本调用接口完成脚本中定义的功能,进而影响集群事务,以此支持nginx、haproxy等服务
架构图
环境准备
使用企业7的版本来做:
- server安装httpd
- selinux与防火墙处于关闭状态
- 各节点时间必须同步:ntp, chrony
[root@realserver1 ~]# yum install httpd -y
[root@realserver1 ~]# echo 172.25.254.110 > /var/www/html/index.html
[root@realserver1 ~]# systemctl enable --now httpd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@realserver1 ~]# curl 172.25.254.110
[root@realserver2 ~]# yum install httpd -y
[root@realserver2 ~]# echo 172.25.254.120 > /var/www/html/index.html
[root@realserver2 ~]# systemctl enable --now httpd.service
[root@realserver2 ~]# curl 172.25.254.120
172.25.254.120
# 安装软件
[root@ka1 ~]# yum install keepalived.x86_64 -y
[root@ka2 ~]# yum install keepalived.x86_64 -y
环境效果:
配置keepalived
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
[root@ka1 ~]# systemctl restart keepalived.service
全局配置:
VIP配置:
# 在10主机上把配置文件拷贝到20服务器上
[root@ka1 ~]# scp /etc/keepalived/keepalived.conf root@172.25.254.20:/etc/keepalived/keepalived.conf
# 然后在20主机上修改配置
[root@ka2 ~]# vim /etc/keepalived/keepalived.conf
[root@ka2 ~]# systemctl restart keepalived.service
验证抓包:
[root@ka1 ~]# tcpdump -i eth0 -nn host 224.0.0.18
停止10服务后,20接替10进行服务
[root@ka1 ~]# systemctl stop keepalived.service
重新启动10后,因为10优先级更高,抢占模式而回到10主机上服务
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# tcpdump -i eth0 -nn host 224.0.0.18
开启通信功能
怎么使VIP通信?
默认情况下VIPping不通,表现为一直等待,包被丢弃,原因是为了安全考虑,能通信容易遭受攻击
默认情况下测试:
查看默认策略
进行更该:
[root@ka1 ~]# vim /etc/rsyslog.conf
[root@ka1 ~]# vim /etc/sysconfig/keepalived
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# iptables -nL
[root@ka1 ~]# ping 172.25.254.100
验证是否能ping通:能
查看更改后策略:策略消失
设置独立日志
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
KEEPALIVED_OPTIONS="-D -S 6"
[root@ka1 ~]# vim /etc/sysconfig/keepalived
[root@ka1 ~]# vim /etc/rsyslog.conf
[root@ka1 ~]# systemctl restart rsyslog.service
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# cat /var/log/keepalived.log
修该日志级别,日志修改范围0-7
修改采集的方法
重启两个服务,并查看
设置独立子配置文件
抢占模式和非抢占模式
延迟抢占
两台主机都要设置成backup模式
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
[root@ka1 ~]# systemctl restart keepalived.service
测试:
默认配置下VIP在10
关闭10的keepalived服务,因此20接管VIP
[root@ka1 ~]# systemctl stop keepalived.service
当10服务上线,在设定的时间内因为10优先级更高,因此到点会抢占VIP
到了设定的抢占时间5s,因优先级更高,则抢占成功
非抢占模式
两台主机都是backup,并且设置成非抢占模式
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# ifconfig
[root@ka1 ~]# systemctl stop keepalived.service
[root@ka2 ~]# vim /etc/keepalived/keepalived.conf
[root@ka2 ~]# systemctl restart keepalived.service
[root@ka2 ~]# ifconfig
20配置
验证:
当10挂了,VIP在20身上
因10优先级更高,但在非抢占模式下,即使10重新恢复服务,也不会抢占20的VIP,除非20服务挂了
单播模式
关闭非抢占模式
需要注释掉vrrp_strict,关闭VIP通信
有vip才有数据
两边都要设置单播,只设置一个主机单播无法通信,都认为对方挂了,各自启VIP
组播情况:
[root@ka1 ~]# tcpdump -i eth0 -nn host 224.0.0.18
修改成单播模式:
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka2 ~]# vim /etc/keepalived/keepalived.conf
[root@ka2 ~]# systemctl restart keepalived.service
抓包验证:
[root@ka1 ~]# tcpdump -i eth0 -nn src host 172.25.254.10 and dst 172.25.254.20
因为VIP在10 身上
关闭10的服务,因此VIP到20身上,用20验证
[root@ka1 ~]# systemctl stop keepalived.service
邮箱
单播情况下,一台主机出现问题,我们需要知道情况
获取邮箱授权码
启用邮箱配置:
[root@ka1 ~]# vim /etc/mail.rc
set bsdcompat
set from=3178535571@qq.com
set smtp=smtp.qq.com
set smtp-auth-user=3178535571@qq.com
set smtp-auth-password=此为邮箱字段生成的授权码
set smtp-auth=login
set ssl-verify=ignore
[root@ka1 ~]# echo test massage | mail -s test 3178535571@qq.com
发送测试邮件:
邮件通知脚本
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
[root@ka1 ~]# vim /etc/keepalived/mail.sh
[root@ka1 ~]# chmod +x /etc/keepalived/mail.sh
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka1 ~]# bash /etc/keepalived/mail.sh master
sh代码
[root@ka1 ~]# vim /etc/keepalived/mail.sh
#!/bin/bash
mail_dst="3178535571@qq.com"
send_message()
{
mail_sub="$HOSTNAME to be $1 vip move"
mail_msg="`date +%F\ %T`: vrrp move $HOSTNAME chage $1"
echo $mail_msg | mail -s "$mail_sub" $mail_dst
}
case $1 in
master)
send_message master
;;
backup)
send_message backup
;;
fault)
send_message fault
;;
*)
;;
esac
邮件测试:
重启服务会有VIP上下限的通知
为什么停止一个服务,没有收到fault的邮件通知?难道是我们代码有误?NO!NO!NO!因为服务已经停了,没有流量能发送fault的邮件,只需关注VIP变化就知道有机器服务出问题了
双主架构
master/slave的单主架构,同一时间只有一个Keepalived对外提供服务,此主机繁忙,而另一台主机却很空闲,利用率低下,可以使用master/master的双主架构,解决此问题。
master/.master的双主架构:
- 即将两个或以上VIP分别运行在不同的keepalived服务器,以实现服务器并行提供web访问的目的,提高服务器资源利用率
- 一个节点只能配一个MASTER,其他都是BACKUP
两个都要改
对10:
[root@ka1 ~]# vim /etc/keepalived/keepalived.conf
[root@ka1 ~]# systemctl restart keepalived.service
[root@ka2 ~]# vim /etc/keepalived/keepalived.conf
[root@ka2 ~]# systemctl restart keepalived.service
20
验证:
10和20都有VIP
一个节点只能配一个MASTER,其他都是BACKUP
即使是三节点的三主架构,每个节点也只有一个master:
#粗略框架显示:
#第一个节点ka1配置:
Vrrp instance 1:MASTER,优先级100
Vrrp instance 2:BACKUP,优先级80
Vrrp instance 3:BACKUP,优先级60
#第二个节点ka2配置:
Vrrp instance 1:BACKUP,优先级60
Vrrp instance 2:MASTER,优先级100
Vrrp instance 3:BACKUP,优先级80
#第三个节点ka3配置:
Vrrp instance 1:BACKUP,优先级80
Vrrp instance 2:BACKUP,优先级60
Vrrp instance 3:MASTER,优先级100
实现ipvs的高可用性
用keepalived控制lvs
110和120主机配置了web服务
lvs-dr
服务器配置环回,此处为永久配置
[root@realserver2 ~]# cd /etc/sysconfig/network-scripts/
[root@realserver2 network-scripts]# vim ifcfg-lo
[root@realserver2 network-scripts]# systemctl restart network
[root@realserver1 ~]# cd /etc/sysconfig/network-scripts/
[root@realserver1 network-scripts]# vim ifcfg-lo
[root@realserver1 network-scripts]# systemctl restart network
内核路由永久配置
两台服务器关闭rip响应
[root@realserver1 ~]# vim /etc/sysctl.d/arp.conf
[root@realserver1 ~]# sysctl --system
[root@realserver2 ~]# vim /etc/sysctl.d/arp.conf
[root@realserver2 ~]# sysctl --system
120
安装软件包
[root@ka1 keepalived]# yum install ipvsadm -y
[root@ka1 keepalived]# vim /etc/keepalived/keepalived.conf
[root@ka2 keepalived]# yum install ipvsadm -y
[root@ka2 keepalived]# vim /etc/keepalived/keepalived.conf
修改配置文件:添加策略
[root@ka1 keepalived]# vim /etc/keepalived/keepalived.conf
[root@ka1 keepalived]# systemctl restart keepalived.service
[root@ka1 keepalived]# ipvsadm -Ln
[root@ka2 keepalived]# vim /etc/keepalived/keepalived.conf
[root@ka2 keepalived]# systemctl restart keepalived.service
验证:
VRRP Script
以haproxy的状态选择VIP的迁移
keepalived 检测haproxy 的状态,并且出问题了后降低VIP
利用脚本实现主从角色切换
编写脚本
[root@ka1 keepalived]# vim /etc/keepalived/test.sh
[root@ka1 keepalived]# bash /etc/keepalived/test.sh
[root@ka1 keepalived]# echo $?
0
[root@ka1 keepalived]# vim /etc/keepalived/keepalived.conf
[root@ka1 keepalived]# systemctl restart keepalived.service
运行代码
此时VIP存在:
编写keepalived
创建脚本中的目录后,则检测结果会运行代码,减少自身优先级,主从角色切换
[root@ka1 keepalived]# touch /mnt/lee
[root@ka1 keepalived]# systemctl restart keepalived.service
[root@ka1 keepalived]# ifconfig
实现HAProxy高可用
haproxy + keepalived
两台主机都要装haproxy
[root@ka1 ~]# yum install haproxy -y
[root@ka2 ~]# yum install haproxy -y
需要在两个ka1和ka2两个节点启用内核参数,目的是为了即使另一台主机没有100的VIP,也能进行远程解析,在原100的VIP下线的时候,保证服务能正常进行
[root@ka1 ~]# vim /etc/sysctl.conf
[root@ka1 ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1
[root@ka2 ~]# vim /etc/sysctl.conf
[root@ka2 ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1
编辑haproxy
[root@ka1 ~]# vim /etc/haproxy/haproxy.cfg
[root@ka1 ~]# systemctl enable --now haproxy.service
[root@ka2 ~]# vim /etc/haproxy/haproxy.cfg
[root@ka2 ~]# systemctl enable --now haproxy.service
删除两台服务器的环回以及开启rip功能,步骤基本相同
[root@realserver2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo
[root@realserver2 ~]# systemctl restart network
[root@realserver2 ~]# ifconfig
[root@realserver1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo
[root@realserver1 ~]# systemctl restart network
[root@realserver1 ~]# ifconfig
恢复rip功能
配置keepalived
[root@rsa1 ~]# vim /etc/keepalived/test1.sh
[root@rsa1 ~]# chmod +x /etc/keepalived/test1.sh
[root@rsa1 ~]# vim /etc/keepalived/keepalived.conf
[root@rsa1 ~]# systemctl restart keepalived.service
10主机配置:
20主机配置
[root@rsa2 ~]# vim /etc/keepalived/test1.sh
[root@rsa2 ~]# chmod +x /etc/keepalived/test1.sh
[root@rsa2 ~]# vim /etc/keepalived/keepalived.conf
[root@rsa2 ~]# systemctl restart keepalived.service
配置haproxy
[root@rsa1 haproxy]# vim /etc/haproxy/haproxy.cfg
[root@rsa1 haproxy]# systemctl restart haproxy.service
10主机配置
20主机配置
```bash
[root@rsa2 haproxy]# vim /etc/haproxy/haproxy.cfg
[root@rsa2 haproxy]# systemctl restart haproxy.service
测试:
当一个服务器的haproxy下线时:服务没影响
[root@rsa1 ~]# systemctl stop haproxy.service