openstack通过namespace将不同项目(租户)的网络隔离,每个项目的管理员都需要对项目网络进行规划建设,这就导致不同项目之间会重复使用到某些网段,例如192.168.X.X就是管理员习惯使用的网段。
上一次我们新建位于vxlan10_net的cirros-vm1和位于vxlan11_net的cirros-vm3的实例并成功通信,今天讲一下原理。
cirros-vm1和cirros-vm3位于不同vxlan,tar设备的网卡信息并没有ip,但却可以通过router实现三层通信,如果没有 Gateway IP,router_100_101 是如何完成路由的呢?
router通过将ip网关信息封装在namespace实现不同vxlan之间的通信,但为何要多此一举呢?明明不需要namespace就可以实现不同vxlan之间的通信。答案就是为了实现不同项目(租户)可以重叠使用网络。
[root@localhost ~(keystone_admin)]# nova list
+--------------------------------------+------------+---------+------------+-------------+--------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------+---------+------------+-------------+--------------------------+
| 791556ab-6231-4370-8a81-972862da200f | cirros-vm1 | SHUTOFF | - | Shutdown | vxlan10_net=172.16.10.11 |
| 6173f9bf-bbed-4d71-96d3-f7ae660dc21c | cirros-vm2 | SHUTOFF | - | Shutdown | vxlan10_net=172.16.10.9 |
| ea42b670-c084-4e76-ae84-23d6617eb420 | cirros-vm3 | SHUTOFF | - | Shutdown | vxlan11_net=172.16.11.4 |
+--------------------------------------+------------+---------+------------+-------------+--------------------------+
###查看tap设备
[root@localhost ~(keystone_admin)]# virsh dumpxml 791556ab-6231-4370-8a81-972862da200f | grep tap
<target dev='tap2c798bf4-8e'/>
[root@localhost ~(keystone_admin)]# virsh dumpxml ea42b670-c084-4e76-ae84-23d6617eb420 | grep tap
<target dev='tap00515be6-9e'/>
###查看tap设备的网卡信息
[root@localhost ~(keystone_admin)]# ifconfig tap2c798bf4-8e
tap2c798bf4-8e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::fc16:3eff:feee:1305 prefixlen 64 scopeid 0x20<link>
ether fe:16:3e:ee:13:05 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1 bytes 90 (90.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
[root@localhost ~(keystone_admin)]# ifconfig tap00515be6-9e
tap00515be6-9e: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1450
inet6 fe80::fc16:3eff:febf:c0eb prefixlen 64 scopeid 0x20<link>
ether fe:16:3e:bf:c0:eb txqueuelen 1000 (Ethernet)
RX packets 14 bytes 1368 (1.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 10 bytes 1382 (1.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
l3 agent 会为每个 router 创建了一个 namespace,而router下的接口信息同样会同步到namespce的接口信息,通过 veth pair 与 TAP 相连,然后将 Gateway IP 配置在位于 namespace 里面的 veth interface 上,这样就能提供路由了。
###查看路由(namespace)列表
[root@localhost ~(keystone_admin)]# neutron router-list
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+--------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| id | name | tenant_id | external_gateway_info | distributed | ha |
+--------------------------------------+--------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
| 30bba1f4-96bb-459c-9760-dba01bde3a01 | router_10_11 | 26f0265d7ca347239ea13ed32279ff7a | {"network_id": "463adb80-e2f3-4a18-9629-cf966fc0b360", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "00897e13-a522-4b8a-943e-7943b652af5a", "ip_address": "172.24.4.4"}]} | False | False |
| 341a7e57-463d-411b-af0a-8cddc56777d1 | router1 | e035103daab74f818fcb4a8a7dbcfda6 | {"network_id": "463adb80-e2f3-4a18-9629-cf966fc0b360", "enable_snat": true, "external_fixed_ips": [{"subnet_id": "00897e13-a522-4b8a-943e-7943b652af5a", "ip_address": "172.24.4.2"}]} | False | False |
+--------------------------------------+--------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------+-------+
###查看具体路由(namespace)的接口信息
[root@localhost ~(keystone_admin)]# ip netns
qrouter-30bba1f4-96bb-459c-9760-dba01bde3a01 (id: 5)
qdhcp-53c25734-40ea-421e-8744-31d3e14819d0 (id: 4)
qdhcp-e64a94b3-3256-4a3d-9833-cc3c9f421e70 (id: 3)
qdhcp-d6c09c69-8bb0-476d-9c71-457251af87b9 (id: 1)
qdhcp-fcf7d046-0937-43af-8b94-32cc1edb24e1 (id: 2)
qrouter-341a7e57-463d-411b-af0a-8cddc56777d1 (id: 0)
id为5的路由(namespace)就是router_10_11的具体id,ip netns exec ID 可查看详细信息。
###查看路由(namespaceID:qrouter-30bba1f4-96bb-459c-9760-dba01bde3a01)详细信息
[root@localhost ~(keystone_admin)]# ip netns exec qrouter-30bba1f4-96bb-459c-9760-dba01bde3a01 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
33: qg-f86168a4-fc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fa:16:3e:b9:54:91 brd ff:ff:ff:ff:ff:ff
inet 172.24.4.4/24 brd 172.24.4.255 scope global qg-f86168a4-fc
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:feb9:5491/64 scope link
valid_lft forever preferred_lft forever
34: qr-03ede9be-8d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fa:16:3e:5f:04:7f brd ff:ff:ff:ff:ff:ff
inet 172.16.10.1/24 brd 172.16.10.255 scope global qr-03ede9be-8d
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe5f:47f/64 scope link
valid_lft forever preferred_lft forever
35: qr-3b2b1f99-dc: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether fa:16:3e:9a:3f:f7 brd ff:ff:ff:ff:ff:ff
inet 172.16.11.1/24 brd 172.16.11.255 scope global qr-3b2b1f99-dc
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe9a:3ff7/64 scope link
valid_lft forever preferred_lft forever
新建网络时候没有设置网关,默认子网首个地址为该网络的网关地址,此处可以看到router有三个接口,分别是33、34、35,33为对外网关,新建路由时默认就有,34、35则对应vxlan10、vxlan11接口,172.16.10.0网络在没有设置网关的情况下默认网关为172.16.10.1。
1. qr-03ede9be-8d 上设置了 Gateway IP 172.16.10.1,与 root namespace 中的 tap03ede9be-8d 组成 veth pair。
2. qr-3b2b1f99-dc 上设置了 Gateway IP 172.16.11.1,与 root namespace 中的 tap3b2b1f99-dc 组成 veth pair。
这里将路由信息封装在namespace中,在外部理解可以直接把namespace当做是路由,并不需要管理员去管什么是namespace,项目管理员只需要配置好vxlan网络和路由就可以实现三层通信,但在openstack的管理员视角,却巧妙地解决了网络重叠的问题。
名称不一致,只关注拓扑关系就行。
不使用namespace
使用namespace