单网口ubuntu主机配置virt-manager传统桥接bridge网络
虚拟机的网络桥接bridge模式往往需要物理宿主机有两个网口，一个网口1连接外网配置ip，另一个网口2空闲不配置ip，在virt-manager里配置虚拟机的网卡绑定网口2，从而实现虚拟机桥接网络模式。
由于办公环境只有一台单网口的主机，通过手动配置arp与路由的方式可实现下方的网络拓扑关系。如果单网口主机有相同配置虚拟机传统桥接网络需求的话，可以参考本文。
先贴下网络拓扑图：
主机A、B、C处于同一个内网172.30.120.0/24里，其中虚拟机C的宿主物理机是B。
按本文操作最终可实现A<->B、A<->C、B<->C互通。
- 宿主物理机B为ubuntu14.04系统，只有一个物理网口，网口信息如下：
krokodil@krokodil-SY-ZL-H110N-D3V:~$ ifconfig
... ...
enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.30.120.125 netmask 255.255.255.0 broadcast 172.30.120.255
inet6 fe80::10f4:3d60:14c5:c457 prefixlen 64 scopeid 0x20<link>
ether 00:e0:4c:5a:03:01 txqueuelen 1000 (以太网)
RX packets 45804 bytes 15342946 (15.3 MB)
RX errors 0 dropped 5188 overruns 0 frame 0
TX packets 20896 bytes 3041221 (3.0 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
... ...
krokodil@krokodil-SY-ZL-H110N-D3V:~$ nm-connection-editor
- 在物理机B上通过nm-connection-editor创建桥接网络
- 终端输入：nm-connection-editor
- 新建一项，类型选择网桥
- 添加设备
- 选择以太网
- 选中enp1s0物理网卡
- 配网络ip等信息
- 物理机B上，关闭nm-connection-editor后，再重新打开可见多了“bridge0 port1”和“网桥连接1”。ifconfig发现多了bridge0网口：
krokodil@krokodil-SY-ZL-H110N-D3V:~$ ifconfig
bridge0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 172.30.120.127 netmask 255.255.255.0 broadcast 172.30.120.255
ether de:91:be:83:e7:72 txqueuelen 1000 (以太网)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.30.120.125 netmask 255.255.255.0 broadcast 172.30.120.255
inet6 fe80::10f4:3d60:14c5:c457 prefixlen 64 scopeid 0x20<link>
ether 00:e0:4c:5a:03:01 txqueuelen 1000 (以太网)
RX packets 8250 bytes 7874949 (7.8 MB)
RX errors 0 dropped 84 overruns 0 frame 0
TX packets 6888 bytes 1845403 (1.8 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
... ...
krokodil@krokodil-SY-ZL-H110N-D3V:~$
- 物理机B上通过virt-manager创建新虚拟机，步骤略过，网络配置如下，配置成桥接模式，设备绑定到bridge0上：
- 虚拟机C安装成功后，把虚拟机内网卡配置好ip：
- 首先需要确保虚拟机C网口与物理机B的bridge0和enp1s0这两个网口互通，当前状态虚拟机C向物理机B ping包不通：
- 由于桥接原理，可知任意从虚拟机C eth0发出的包必会在物理机B bridge0网口抓到。配置好后，在虚拟机C内部ping物理机B网桥bridge0，同时对物理机B的bridge0与enp1s0抓包。
- 发现虚拟机C的icmp request发到了物理机B bridge0上，但由于物理机B不知道虚拟机C网口ip172.30.120.120的mac地址，所以reply包无法发出。
- 查询物理机B当前的arp规则，发现缺失网口enp1s0的ip为172.30.120.120的规则：
krokodil@krokodil-SY-ZL-H110N-D3V:~$ arp -n
地址 类型 硬件地址 标志 Mask 接口
172.30.120.254 (incomplete) bridge0
172.30.120.120 (incomplete) enp1s0
172.30.120.120 ether 52:54:00:32:94:c2 C bridge0
172.30.120.254 ether 04:fe:8d:8e:31:e1 C enp1s0
krokodil@krokodil-SY-ZL-H110N-D3V:~$
- 在物理机B配置一条arp规则，将虚拟机C内eth0的mac地址52:54:00:32:94:c2与ip172.30.120.120关联起来
krokodil@krokodil-SY-ZL-H110N-D3V:~$ sudo arp -s 172.30.120.120 52:54:00:32:94:c2 -i enp1s0
krokodil@krokodil-SY-ZL-H110N-D3V:~$ arp -n
地址 类型 硬件地址 标志 Mask 接口
172.30.120.254 (incomplete) bridge0
172.30.120.120 ether 52:54:00:32:94:c2 CM enp1s0
172.30.120.120 ether 52:54:00:32:94:c2 C bridge0
172.30.120.254 ether 04:fe:8d:8e:31:e1 C enp1s0
krokodil@krokodil-SY-ZL-H110N-D3V:~$
- 再重复在虚拟机C内向ping 172.30.120.127 -c 1，同时在物理机B抓包，发现虚拟机C仍然ping不通物理机B
- 这次抓包没有抓到arp，发现物理机B上reply包并没有从enp1s0转发到bridge0网口。
- 查看物理机B路由：
内核 IP 路由表
目标 网关 子网掩码 标志 跃点 引用 使用 接口
0.0.0.0 172.30.120.254 0.0.0.0 UG 100 0 0 enp1s0
0.0.0.0 172.30.120.254 0.0.0.0 UG 20425 0 0 bridge0
10.10.100.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 virbr0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.30.120.0 0.0.0.0 255.255.255.0 U 100 0 0 enp1s0
172.30.120.0 0.0.0.0 255.255.255.0 U 425 0 0 bridge0
- 关于172.30.120.0/24网段的包在物理机B上有两条路由，但因为100跳数小于425，所以默认向172.30.120.0/24网段的包向enp1s0转发。这就是为什么上面的reply包没有被转发到bridge0网口。
- 在物理机B添加一条关于172.30.120.120的路由规则，告知收到目的ip为172.30.120.120的包就直接转发给bridge0网口：
krokodil@krokodil-SY-ZL-H110N-D3V:~$ sudo route add 172.30.120.120 dev bridge0
krokodil@krokodil-SY-ZL-H110N-D3V:~$ route -n
内核 IP 路由表
目标 网关 子网掩码 标志 跃点 引用 使用 接口
0.0.0.0 172.30.120.254 0.0.0.0 UG 100 0 0 enp1s0
0.0.0.0 172.30.120.254 0.0.0.0 UG 20425 0 0 bridge0
10.10.100.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0 0 virbr0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.30.120.0 0.0.0.0 255.255.255.0 U 100 0 0 enp1s0
172.30.120.0 0.0.0.0 255.255.255.0 U 425 0 0 bridge0
172.30.120.120 0.0.0.0 255.255.255.255 UH 0 0 0 bridge0
krokodil@krokodil-SY-ZL-H110N-D3V:~$
- 此时，虚拟机C内网口eth0可以和物理机B的bridge0网口、物理网口enp1s0互通。
- 虚拟机C内ping包结果：
- 物理机B内ping包结果：
- 宿主机B与虚拟机C互通后，可以进一步配置同网段内（172.30.120.0/24）机器A与虚拟机C互通。此台机器A的ip为172.30.120.121，已验证可ping通物理机B的两个网口ip 172.30.120.125和172.30.120.127。
- 物理机A网口信息：
enaftgm1i0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.30.120.121 netmask 255.255.255.0 broadcast 172.30.120.255
inet6 fe80::54fe:45f6:974e:8db3 prefixlen 64 scopeid 0x20<link>
ether 00:07:3e:9d:55:8f txqueuelen 1000 (以太网)
RX packets 137322 bytes 43916437 (43.9 MB)
RX errors 0 dropped 4476 overruns 0 frame 0
TX packets 188258 bytes 22199429 (22.1 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 7 base 0xc000
root@sujing-GW-001N1B-FTF:~#
- 在当前状态下，在同网段机器A（172.30.120.121）上ping 172.30.120.120 -c 1，发包不通：
PING 172.30.120.120 (172.30.120.120) 56(84) bytes of data.
From 172.30.120.121 icmp_seq=1 Destination Host Unreachable
--- 172.30.120.120 ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms
- 在物理机A网口抓包，发现172.30.120.121不知道172.30.120.120的mac地址
tcpdump: listening on enaftgm1i0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:09:54.554190 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.30.120.120 tell 172.30.120.121, length 28
15:09:55.580834 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.30.120.120 tell 172.30.120.121, length 28
15:09:56.600821 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.30.120.120 tell 172.30.120.121, length 28
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
root@sujing-GW-001N1B-FTF:~#
- 查询物理机A（172.30.120.121）的arp表，看到这一条：
地址 类型 硬件地址 标志 Mask 接口
172.30.120.254 ether 04:fe:8d:8e:31:e1 C enaftgm1i0
172.30.120.120 (incomplete) enaftgm1i0
... ...
- 在物理机A（172.30.120.121）上增加arp规则：注意！这里的mac地址要填写enp1s0的mac地址00:e0:4c:5a:03:01
root@sujing-GW-001N1B-FTF:~# sudo arp -s 172.30.120.120 00:e0:4c:5a:03:01 -i enaftgm1i0
root@sujing-GW-001N1B-FTF:~# arp -n
地址 类型 硬件地址 标志 Mask 接口
172.30.120.254 ether 04:fe:8d:8e:31:e1 C enaftgm1i0
172.30.120.120 ether 00:e0:4c:5a:03:01 CM enaftgm1i0
... ... ... ...
-
然后在物理机A上重新ping 172.30.120.120，同时对物理机B的两个网口抓包
-
这次icmp request成功到达了物理机B的enp1s0网口，但是在bridge0网口看到虚拟机C 172.30.120.120发出的arp查询包。
-
确认虚拟机C内arp表，确实缺少121的mac地址：
-
在虚拟机C内部配置arp规则172.30.120.121 de:91:be:83:e7:72 （注意！！这里要配置bridge0 mac地址）
- 此时再在物理机A 172.30.120.121上ping包，就可通了。
root@sujing-GW-001N1B-FTF:~# ping 172.30.120.120 -c 1
PING 172.30.120.120 (172.30.120.120) 56(84) bytes of data.
64 bytes from 172.30.120.120: icmp_seq=1 ttl=63 time=0.731 ms
--- 172.30.120.120 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.731/0.731/0.731/0.000 ms
root@sujing-GW-001N1B-FTF:~#
- 至此，总结下：
172.30.120.121物理机A需要添加一条arp规则：172.30.120.120 enp1s0的mac地址
172.30.120.125与172.30.120.127物理机B需要添加一条路由规则：172.30.120.120 dev bridge0，以及一条arp规则172.30.120.120 eth0的mac地址
172.30.120.120虚拟机C需要添加一条arp规则：172.30.120.121 bridge0的mac地址
从物理机A 网口向 虚拟机C ping一个包，抓包看整个过程：
- 物理机A 172.30.120.121 enaftgm1i0(00:07:3e:9d:55:8f)向ip 172.30.120.120发request包，根据物理机A上的arp规则，向172.30.120.120发包的目的mac应配为00:e0:4c:5a:03:01（00:e0:4c:5a:03:01为物理机B 172.30.120.125 enp1s0的mac地址）。所以最终在网口enaftgm1i0抓到的request包，源mac地址是enaftgm1i0，目的mac地址是enp1s0。在通过内网的交换机与路由器时，此request包会被转发到00:e0:4c:5a:03:01所在的网口，即enp1s0网口。
root@sujing-GW-001N1B-FTF:~# tcpdump -i enaftgm1i0 -venn host 172.30.120.120
tcpdump: listening on enaftgm1i0, link-type EN10MB (Ethernet), capture size 262144 bytes
16:14:10.131325 00:07:3e:9d:55:8f > 00:e0:4c:5a:03:01, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 51167, offset 0, flags [DF], proto ICMP (1), length 84)
172.30.120.121 > 172.30.120.120: ICMP echo request, id 6552, seq 1, length 64
16:14:10.131851 00:e0:4c:5a:03:01 > 00:07:3e:9d:55:8f, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 54622, offset 0, flags [none], proto ICMP (1), length 84)
172.30.120.120 > 172.30.120.121: ICMP echo reply, id 6552, seq 1, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
root@sujing-GW-001N1B-FTF:~#
- 物理机B enp1s0(00:e0:4c:5a:03:01)抓包可看到request包（源mac地址是enaftgm1i0，目的mac地址是enp1s0）已到达物理机B。根据物理机B的路由规则，172.30.120.120的包应转发给bridge0网口：
tcpdump: listening on enp1s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:14:10.163279 00:07:3e:9d:55:8f > 00:e0:4c:5a:03:01, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 51167, offset 0, flags [DF], proto ICMP (1), length 84)
172.30.120.121 > 172.30.120.120: ICMP echo request, id 6552, seq 1, length 64
16:14:10.163690 00:e0:4c:5a:03:01 > 00:07:3e:9d:55:8f, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 54622, offset 0, flags [none], proto ICMP (1), length 84)
172.30.120.120 > 172.30.120.121: ICMP echo reply, id 6552, seq 1, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
root@krokodil-SY-ZL-H110N-D3V:~#
- bridge0(de:91:be:83:e7:72)抓包看到，获取到了request包（在enp1s0转发时，源mac地址替换成了bridge0，目的mac地址被替换成了eth0)。
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:14:10.163315 de:91:be:83:e7:72 > 52:54:00:32:94:c2, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 63, id 51167, offset 0, flags [DF], proto ICMP (1), length 84)
172.30.120.121 > 172.30.120.120: ICMP echo request, id 6552, seq 1, length 64
16:14:10.163677 52:54:00:32:94:c2 > de:91:be:83:e7:72, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 54622, offset 0, flags [none], proto ICMP (1), length 84)
172.30.120.120 > 172.30.120.121: ICMP echo reply, id 6552, seq 1, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
root@krokodil-SY-ZL-H110N-D3V:~#
- 172.30.120.120虚拟机C网口eth0抓包，抓到request和reply（源和目的mac分别是bridge0和eth0）：
- reply包发送和路由原理相同，略，看抓包即可明白。
