openEuler 22.03 LTS SP3源码编译部署OpenStack-Caracal遇到的问题解决
- 问题一 给路由设置外部网关后Status为DOWN（使用的是OVN）
- 问题描述
- 临时的解决办法
- 永久解决办法（修改源代码）
- 问题二 分离卷一直显示分离中
- 问题描述
- 解决办法（不太严谨）
问题一 给路由设置外部网关后Status为DOWN（使用的是OVN）
问题描述
问题如下图
经过排查后发现是
Logical_Router_Port的gateway_chassis : []为空导致openstack给路由设置外部网关后Status为DOWN
临时的解决办法
临时的解决办法是手动去添加gateway_chassis
首先通过如下命令查看需要添加的gateway_chassis的_uuid
ovn-nbctl list Logical_Router_Port
再通过如下命令去查看需要添加的chassis值
ovn-sbctl show
在通过如下的命令去手动的添加即可
ovn-nbctl lrp-set-gateway-chassis _uuid值 Chassis值
永久解决办法（修改源代码）
只所以会出现这个状况是因为在代码中添加chassis无法匹配到合适的
具体如下
有关代码的路径如下
/usr/local/lib/python3.9/site-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb
代码文件名叫ovn_client.py
未修改源代码如下
def get_candidates_for_scheduling(self, physnet, cms=None,
chassis_physnets=None,
availability_zone_hints=None):
"""Return chassis for scheduling gateway router.
Criteria for selecting chassis as candidates
1) Chassis from cms with proper bridge mappings only (that means these
gateway chassis with the requested physical network).
2) Filter the available chassis accordingly to the routers
availability zone hints (if present)
If the logical router port belongs to a tunnelled network, there won't
be any candidate.
"""
# TODO(lucasagomes): Simplify the logic here, the CMS option has
# been introduced long ago and by now all gateway chassis should
# include it. This will match the logic in the is_gateway_chassis()
# (utils.py)
cms = cms or self._sb_idl.get_gateway_chassis_from_cms_options()
chassis_physnets = (chassis_physnets or
self._sb_idl.get_chassis_and_physnets())
candidates = set()
for chassis, physnets in chassis_physnets.items():
if (physnet and
physnet in physnets and
chassis in cms):
candidates.add(chassis)
candidates = list(candidates)
# Filter for availability zones
if availability_zone_hints:
LOG.debug('Filtering Chassis candidates by availability zone '
'hints: %s', ', '.join(availability_zone_hints))
candidates = [ch for ch in candidates
for az in availability_zone_hints
if az in utils.get_chassis_availability_zones(
self._sb_idl.lookup('Chassis', ch, None))]
LOG.debug('Chassis candidates for scheduling gateway router ports '
'for "%s" physical network: %s', physnet, candidates)
return candidates
主要的原因是获取到cms列表未空导致无法匹配到合适gateway_chassis将其添加
解决办法有两个
- 不需要修改源代码，去添加一个cms标记，让代码那个获取到，但是由于我是第一次遇到这个情况，我不知道要设置什么样的cms标记。所以无法从这方面入手。
- 修该源代码去添加一个处理cms列表为空的逻辑
添加一个处理cms列表为空的逻辑，代码如下：
def get_candidates_for_scheduling(self, physnet, cms=None, chassis_physnets=None, availability_zone_hints=None):
"""Return chassis for scheduling gateway router.
Criteria for selecting chassis as candidates:
1) Chassis from cms with proper bridge mappings only (that means these
gateway chassis with the requested physical network).
2) Filter the available chassis accordingly to the routers
availability zone hints (if present)
If the logical router port belongs to a tunnelled network, there won't
be any candidate.
"""
cms = cms or self._sb_idl.get_gateway_chassis_from_cms_options()
chassis_physnets = chassis_physnets or self._sb_idl.get_chassis_and_physnets()
candidates = set()
# If CMS is empty, we may assume all chassis are managed
managed_chassis = cms if cms else [chassis for chassis in chassis_physnets]
for chassis in managed_chassis:
physnets = chassis_physnets.get(chassis, [])
if physnet in physnets:
candidates.add(chassis)
# Convert candidates set to list
candidates = list(candidates)
# Filter for availability zones
if availability_zone_hints:
LOG.debug('Filtering Chassis candidates by availability zone hints: %s', ', '.join(availability_zone_hints))
filtered_candidates = []
for ch in candidates:
azs = utils.get_chassis_availability_zones(self._sb_idl.lookup('Chassis', ch, None))
if any(az in azs for az in availability_zone_hints):
filtered_candidates.append(ch)
candidates = filtered_candidates
LOG.debug('Chassis candidates for scheduling gateway router ports for "%s" physical network: %s', physnet, candidates)
return candidates
修改完成后重启一下neutron服务即可。
问题二 分离卷一直显示分离中
问题描述
创建了一个10G的卷
连接到实例
进行分离卷
解决办法（不太严谨）
根据日志提示
2024-05-27 20:08:29.656 59902 ERROR cinder.volume.api [req-e5dea5da-9565-4e0e-b05b-9dbc58f7deef req-87c34a80-75b3-4bdd-87bf-1136cbb88b96 950f09833b4e4dffaf4acf3ac9f1d4bd 840a040edf78409f92d7c7a128652718 - - default default] Detected user call to delete in-use attachment. Call must come from the nova service and nova must be configured to send the service token. Bug #2004555
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault [req-e5dea5da-9565-4e0e-b05b-9dbc58f7deef req-87c34a80-75b3-4bdd-87bf-1136cbb88b96 950f09833b4e4dffaf4acf3ac9f1d4bd 840a040edf78409f92d7c7a128652718 - - default default] Caught error: <class 'cinder.exception.ConflictNovaUsingAttachment'> Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API: cinder.exception.ConflictNovaUsingAttachment: Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault Traceback (most recent call last):
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/middleware/fault.py", line 84, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return req.get_response(self.application)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application(
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/osprofiler/web.py", line 111, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return request.get_response(self.application)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application(
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/keystonemiddleware/auth_token/__init__.py", line 340, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault response = req.get_response(self._app)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application(
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/routes/middleware.py", line 153, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault response = self.app(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 839, in __call__
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self._process_stack(request, action, action_args,
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 900, in _process_stack
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault action_result = self.dispatch(meth, request, action_args)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 995, in dispatch
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return method(req=request, **action_args)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 1160, in version_select
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return func.func(self, *args, **kwargs)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/v3/attachments.py", line 282, in delete
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault attachments = self.volume_api.attachment_delete(context, attachment)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/volume/api.py", line 2668, in attachment_delete
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault self.attachment_deletion_allowed(ctxt, attachment)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/volume/api.py", line 2659, in attachment_deletion_allowed
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault raise exception.ConflictNovaUsingAttachment(instance_id=server_id)
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault cinder.exception.ConflictNovaUsingAttachment: Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API
2024-05-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault
提示我using the Compute API
于是我去设置如下的内容
controller节点
vim /etc/nova/nova.conf
[cinder]
send_service_user_token = True #添加
compute节点
vim /etc/nova/nova.conf
[cinder]
send_service_user_token = True #添加
但是没有解决，还是提示同样的问题
于是去查看源代码
在/usr/local/lib/python3.9/site-packages/cinder/volume/api.py
源代码如下
def attachment_deletion_allowed(self,
ctxt: context.RequestContext,
attachment_or_attachment_id,
volume=None):
"""Check if deleting an attachment is allowed (Bug #2004555)
Allowed is based on the REST API policy, the status of the attachment,
where it is used, and who is making the request.
Deleting an attachment on the Cinder side while leaving the volume
connected to the nova host results in leftover devices that can lead to
data leaks/corruption.
OS-Brick may have code to detect it, but in some cases it is detected
after it has already been exposed, so it's better to prevent users from
being able to intentionally triggering the issue.
"""
# It's ok to delete an attachment if the request comes from a service
if self.is_service_request(ctxt):
return
if not attachment_or_attachment_id and volume:
if not volume.volume_attachment:
return
if len(volume.volume_attachment) == 1:
attachment_or_attachment_id = volume.volume_attachment[0]
if isinstance(attachment_or_attachment_id, str):
try:
attachment = objects.VolumeAttachment.get_by_id(
ctxt, attachment_or_attachment_id)
except exception.VolumeAttachmentNotFound:
attachment = None
else:
attachment = attachment_or_attachment_id
if attachment:
if volume:
if volume.id != attachment.volume_id:
raise exception.InvalidInput(
reason='Mismatched volume and attachment')
server_id = attachment.instance_uuid
# It's ok to delete if it's not connected to a vm.
if not server_id or not attachment.connection_info:
return
volume = volume or attachment.volume
nova = compute.API()
LOG.info('Attachment connected to vm %s, checking data on nova',
server_id)
# If nova is down the client raises 503 and we report that
try:
nova_volume = nova.get_server_volume(ctxt, server_id,
volume.id)
except nova.NotFound:
LOG.warning('Instance or volume not found on Nova, deleting '
'attachment locally, which may leave leftover '
'devices on Nova compute')
return
if nova_volume.attachment_id != attachment.id:
LOG.warning('Mismatch! Nova has different attachment id (%s) '
'for the volume, deleting attachment locally. '
'May leave leftover devices in a compute node',
nova_volume.attachment_id)
return
else:
server_id = ''
LOG.error('Detected user call to delete in-use attachment. Call must '
'come from the nova service and nova must be configured to '
'send the service token. Bug #2004555')
raise exception.ConflictNovaUsingAttachment(instance_id=server_id)
添加一个逻辑处理，通过实例的uuid再一次确认它是否有附加卷
instance_uuid = attachment.instance_uuid
try:
attachments = objects.VolumeAttachmentList.get_all_by_instance_uuid(ctxt, instance_uuid)
if attachments:
LOG.info("Instance {} has other attachments: {}".format(instance_uuid, [att.id for att in attachments]))
else:
LOG.info("No other attachments found for instance {}".format(instance_uuid))
return
except Exception as e:
LOG.error("Failed to retrieve attachments for instance {}: {}".format(instance_uuid, str(e)))
raise exception.VolumeBackendAPIException(reason=str(e))
修改后的完整的代码如下
def attachment_deletion_allowed(self,
ctxt: context.RequestContext,
attachment_or_attachment_id,
volume=None):
"""Check if deleting an attachment is allowed (Bug #2004555)
Allowed is based on the REST API policy, the status of the attachment,
where it is used, and who is making the request.
Deleting an attachment on the Cinder side while leaving the volume
connected to the nova host results in leftover devices that can lead to
data leaks/corruption.
OS-Brick may have code to detect it, but in some cases it is detected
after it has already been exposed, so it's better to prevent users from
being able to intentionally triggering the issue.
"""
# It's ok to delete an attachment if the request comes from a service
if self.is_service_request(ctxt):
return
if not attachment_or_attachment_id and volume:
if not volume.volume_attachment:
return
if len(volume.volume_attachment) == 1:
attachment_or_attachment_id = volume.volume_attachment[0]
if isinstance(attachment_or_attachment_id, str):
try:
attachment = objects.VolumeAttachment.get_by_id(
ctxt, attachment_or_attachment_id)
except exception.VolumeAttachmentNotFound:
attachment = None
else:
attachment = attachment_or_attachment_id
if attachment:
if volume:
if volume.id != attachment.volume_id:
raise exception.InvalidInput(
reason='Mismatched volume and attachment')
server_id = attachment.instance_uuid
# It's ok to delete if it's not connected to a vm.
if not server_id or not attachment.connection_info:
return
volume = volume or attachment.volume
nova = compute.API()
LOG.info('Attachment connected to vm %s, checking data on nova',
server_id)
# If nova is down the client raises 503 and we report that
try:
nova_volume = nova.get_server_volume(ctxt, server_id,
volume.id)
except nova.NotFound:
LOG.warning('Instance or volume not found on Nova, deleting '
'attachment locally, which may leave leftover '
'devices on Nova compute')
return
if nova_volume.attachment_id != attachment.id:
LOG.warning('Mismatch! Nova has different attachment id (%s) '
'for the volume, deleting attachment locally. '
'May leave leftover devices in a compute node',
nova_volume.attachment_id)
return
else:
server_id = ''
instance_uuid = attachment.instance_uuid
try:
attachments = objects.VolumeAttachmentList.get_all_by_instance_uuid(ctxt, instance_uuid)
if attachments:
LOG.info("Instance {} has other attachments: {}".format(instance_uuid, [att.id for att in attachments]))
else:
LOG.info("No other attachments found for instance {}".format(instance_uuid))
return
except Exception as e:
LOG.error("Failed to retrieve attachments for instance {}: {}".format(instance_uuid, str(e)))
raise exception.VolumeBackendAPIException(reason=str(e))
最终成功解决，但是这个方法不是很严谨
