详细LVS实验配置

一、LVS的NAT模式

1、实验环境

主机名	ip	VIP	角色
lvs	192.168.0.100	172.25.254.100	调度器
webserver1	192.168.0.10，网关192.168.0.100	null	真实服务器（ RS ）
webserver2	192.168.0.20，网关192.168.0.100	null	真实服务器（ RS ）

2、配置命令

1）webserver1、2

ip配置

[root@webserver1 boot]#vmset.sh eth0 192.168.0.100 webserver1.hyl.org

[root@webserver1 boot]# vim /etc/NetworkManager/system-connections/eth0.nmconnection

[root@webserver1 boot]# nmcli connection reload

[root@webserver1 boot]# nmcli connection up eth0

[root@webserver1 boot]# route -n

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

0.0.0.0 192.168.0.100 0.0.0.0 UG 100 0 0 eth0

192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0

配置httpd服务

[root@webserver1 boot]# yum install httpd -y

[root@webserver1 boot]# echo webserver - 192.168.0.10 > /var/www/html/index.html

[root@webserver boot]# systemctl enable --now httpd

[root@webserver2 boot]# echo webserver2 - 192.168.0.20 > /var/www/html/index.html

[root@webserver2 boot]# systemctl enable --now httpd

2）lvs调度器

配置ip

[root@lvs boot]# vmset.sh eth0 172.25.254.100 lvs.hyl.org

[root@lvs boot]# vmset.sh eth1 192.168.0.100 lvs.hyl.org

连接已成功激活（D-Bus 活动路径：/org/freedesktop/NetworkManager/ActiveConnection/5）

[root@lvs boot]# vim /etc/NetworkManager/system-connections/eth1.nmconnection

[root@lvs boot]# nmcli connection reload

[root@lvs boot]# nmcli connection up eth1

启用内核路由功能

[root@lvs boot]# sysctl -a | grep ip_forward

net.ipv4.ip_forward = 0

net.ipv4.ip_forward_update_priority = 1

net.ipv4.ip_forward_use_pmtu = 0

[root@lvs boot]# vim /etc/sysctl.conf

net.ipv4.ip_forward = 1

[root@lvs boot]# sysctl -p

net.ipv4.ip_forward = 1

关闭火墙

[root@lvs boot]# systemctl stop firewalld.service

测试httpd服务

[root@lvs boot]# curl 192.168.0.10
webserver - 192.168.0.10

[root@lvs boot]# curl 192.168.0.20
webserver2 - 192.168.0.20

安装ipvsadm

[root@lvs boot]# yum install ipvsadm -y

添加调度策略

[root@lvs boot]# ipvsadm -A -t 172.25.254.100:80 -s rr

[root@lvs boot]# ipvsadm -a -t 172.25.254.100:80 -r 192.168.0.10:80 -m

[root@lvs boot]# ipvsadm -a -t 172.25.254.100:80 -r 192.168.0.20:80 -m

[root@lvs boot]# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 172.25.254.100:80 rr

-> 192.168.0.10:80 Masq 1 0 0

-> 192.168.0.20:80

保存规则

[root@lvs boot]# ipvsadm-save -n > /etc/sysconfig/ipvsadm

[root@lvs boot]# cat /etc/sysconfig/ipvsadm

-A -t 172.25.254.100:80 -s rr

-a -t 172.25.254.100:80 -r 192.168.0.10:80 -m

-a -t 172.25.254.100:80 -r 192.168.0.20:80 -m

[root@lvs boot]# ipvsadm -C

[root@lvs boot]# ipvsadm-restore < /etc/sysconfig/ipvsadm

[root@lvs boot]# cat /etc/sysconfig/ipvsadm

-A -t 172.25.254.100:80 -s rr

-a -t 172.25.254.100:80 -r 192.168.0.10:80 -m

-a -t 172.25.254.100:80 -r 192.168.0.20:80 -m

删除所有规则

[root@lvs boot]# ipvsadm -C

[root@lvs boot]# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

重新加载规则

[root@lvs boot]# ipvsadm -R < /etc/sysconfig/ipvsadm-config

[root@lvs boot]# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 172.25.254.100:80 rr

-> 192.168.0.101:80 Masq 1 0 0

-> 192.168.0.102:80 Masq 1 0 0

3、测试

[root@lvs boot]# for i in {1..10}; do curl 172.25.254.100; done

webserver2 - 192.168.0.20

webserver - 192.168.0.10

webserver2 - 192.168.0.20

webserver - 192.168.0.10

webserver2 - 192.168.0.20

webserver - 192.168.0.10

webserver2 - 192.168.0.20

webserver - 192.168.0.10

webserver2 - 192.168.0.20

webserver - 192.168.0.10

二、LVS的DR模式

1、实验环境

主机名	ip	VIP	角色
client	nat：172.25.254.200 网关：172.25.254.100	null	客户端
router	nat:172.25.254.100 仅主机：192.168.0.100	null	路由器
lvs	仅主机：192.168.0.50 网关：192.168.0.100	lo:192.168.0.200	调度器
webserver1	仅主机：192.168.0.10 网关：192.168.0.100	lo:192.168.0.200	RS1
webserver2	仅主机：192.168.0.20 网关：192.168.0.100	lo:192.168.0.200	RS2

2、实验配置

1）webserver1、2

ip配置

server1、2一样

[root@webserver2 ~]# cat /etc/NetworkManager/system-connections/eth0.nmconnection
[connection]
id=eth0
type=ethernet
interface-name=eth0

[ipv4]
address1=192.168.0.20/24,192.168.0.100
method=manual

[root@webserver2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.100 0.0.0.0 UG 100 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0

rs主机中使vip不对外响应

[root@webserver boot]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@webserver boot]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@webserver boot]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@webserver boot]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore

查看

[root@webserver boot]# sysctl -a | grep arp_ignore
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.lo.arp_ignore = 1

[root@webserver2 boot]# sysctl -a | grep arp_ignore
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.lo.arp_ignore = 1

在server设定vip

[root@webserver boot]# ip a a 192.168.0.200/32 dev lo
[root@webserver2 boot]# ip a a 192.168.0.200/32 dev lo

2）client

ip配置

[root@client ~]# vmset.sh eth0 172.25.254.200 client.hyl.org

3）router

ip配置

[root@router ~]# vmset.sh eth0 172.25.254.100 router.hyl.org

[root@router ~]# vmset.sh eth1 192.168.0.100 router.hyl.org

[root@router ~]# vim /etc/NetworkManager/system-connections/eth1.nmconnect ion

[root@router ~]# nmcli connection reload

[root@router ~]# nmcli connection up reth1

####eth0为nat网卡#######

[root@router ~]# cat /etc/NetworkManager/system-connections/eth0.nmconnection
[connection]
id=eth0
type=ethernet
interface-name=eth0

[ipv4]
address1=172.25.254.100/24,172.25.254.2
method=manual
dns=114.114.114.114;

#####eth1为仅主机网卡#######
[root@router ~]# cat /etc/NetworkManager/system-connections/eth1.nmconnec tion
[connection]
id=eth1
type=ethernet
interface-name=eth1

[ipv4]
address1=192.168.0.100/24
method=manual

启用内核路由功能

[root@router ~]# sysctl -a | grep ip_forward

net.ipv4.ip_forward = 0

net.ipv4.ip_forward_update_priority = 1

net.ipv4.ip_forward_use_pmtu = 0

[root@router ~]# vim /etc/sysctl.conf

[root@router ~]# sysctl -p

net.ipv4.ip_forward = 1

4）lvs

ip配置

[root@lvs boot]# vmset.sh eth1 192.168.0.50 lvs.hyl.org

[root@lvs boot]#vim /etc/NetworkManager/system-connections/eth1.nmconnect ion

[root@lvs boot]# nmcli connection reload

[root@lvs boot]#nmcli connection up reth1

在Ivs主机中添加vip

[root@lvs ~]# nmcli connection reload

[root@lvs ~]# nmcli connection up lo

配置策略

[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn

[root@lvs ~]# ipvsadm -A -t 192.168.0.200:80 -s wrr

[root@lvs ~]# ipvsadm -a -t 192.168.0.200:80 -r 192.168.0.10:80 -g -w 1

[root@lvs ~]# ipvsadm -a -t 192.168.0.200:80 -r 192.168.0.20:80 -g -w 2

[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.200:80 wrr
-> 192.168.0.10:80 Route 1 0 0
-> 192.168.0.20:80 Route 2 0 0

3、实验测试

[root@client ~]# for i in {1..10}
> do
> curl 192.168.0.200
> done
webserver2 - 192.168.0.20
webserver - 192.168.0.10
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver - 192.168.0.10
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver - 192.168.0.10
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20

三、LVS的火墙标记

1、轮询规则遇到错误

[root@lvs ~]# ipvsadm -E -t 192.168.0.200:80 -s rr
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.200:80 rr
-> 192.168.0.10:80 Route 1 0 0
-> 192.168.0.20:80 Route 2 0 1

######RS安装mod ssl模块让rs支持https#########

[root@webserver ~]# yum install mod_ssl -y

[root@webserver ~]# systemctl restart httpd

######查看httpd端口
[root@webserver ~]# netstat -ntulp | grep httpd
tcp6 0 0 :::443 :::* LISTEN 34138/httpd
tcp6 0 0 :::80 :::* LISTEN 34138/httpd

当访问vip时两次调度都到了测试错误
[root@client ~]# curl 192.168.0.200;curl -k https://192.168.0.200
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20

2、LVS主机中为端口做标记

#####在vs调度器中设定端口标签，人为80和443是一个整体
[root@lvs ~]# iptables -t mangle -A PREROUTING -d 192.168.0.200 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 66
[root@lvs ~]# iptables -t mangle -nL
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
MARK 6 -- 0.0.0.0/0 192.168.0.200 multiport dports 80,443 MARK set 0x42
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

3、lvs调度配置

设定调度规则

[root@lvs ~]# ipvsadm -C
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@lvs ~]# ipvsadm -A -f 66 -s rr
[root@lvs ~]# ipvsadm -a -f 66 -r 192.168.0.10 -g
[root@lvs ~]# ipvsadm -a -f 66 -r 192.168.0.20 -g
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 66 rr
-> 192.168.0.10:0 Route 1 0 0
-> 192.168.0.20:0 Route 1 0 0

4、测试

[root@client ~]# curl 192.168.0.200;curl -k https://192.168.0.200
webserver2 - 192.168.0.20
webserver - 192.168.0.10

5、lvs持久链接

在lvs调度器中设定

[root@lvs ~]# ipvsadm -E -f 66 -s rr -p

[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 66 rr persistent 360
-> 192.168.0.10:0 Route 1 0 0
-> 192.168.0.20:0 Route 1 0 0

测试

##############################
[root@client ~]# for i in {1..10}
> do
> curl 192.168.0.200
> done
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20
webserver2 - 192.168.0.20