文献速递：肿瘤分割---- 弱监督教师-学生网络用于非增强图像的肝脏肿瘤分割

文献速递介绍

准确的肝脏肿瘤分割对放射科医师来说是必不可少的，以诊断和治疗肝脏肿瘤并提高患者的生存率（Radtke 等人，2007年）。特别是，肝脏肿瘤分割帮助放射科医师1）确定肿瘤的阶段并制定肝脏手术计划（例如，肿瘤切除术和肝脏移植）。例如，当前常用的分期系统，如巴塞罗那诊所肝癌分期系统，考虑到肿瘤大小以及病变的多发性，以选择适合手术治疗或局部区域治疗的候选人（Chapiro 等人，2015年）。2）简化外科切除的规划。外科切除规划的主要约束之一是例如，加铝酸镓二甲葡胺（Gd-BOPTA，MultiHance）需要40至120分钟来将局灶性肝损害作为黑色病变显示出来，与增强的正常肝脏形成对比（Ibrahim 等，2020年）。

3）由对比剂材料和影像扫描造成的昂贵成本。如果注射过程中发生错误，对比剂材料将被浪费。此外，为了获得增强图像，需要在第一次非增强扫描后进行第二次扫描（动态对比增强磁共振成像），这将产生额外成本。从非增强图像中分割肝脏肿瘤可以避免由对比剂引起的上述缺点，但这是具有挑战性的。

Title

题目

Weakly-Supervised teacher-Student network for liver tumor segmentation from non-enhanced images

弱监督教师-学生网络用于非增强图像的肝脏肿瘤分割

Abstract

摘要

Accurate liver tumor segmentation without contrast agents (non-enhanced images) avoids the contrastagent-associated time-consuming and high risk, which offers radiologists quick and safe assistance to diagnose and treat the liver tumor. However, without contrast agents enhancing, the tumor in liver images presents low contrast and even invisible to naked eyes. Thus the liver tumor segmentation from non-enhanced images is quite challenging. We propose a Weakly-Supervised Teacher-Student network (WSTS) to address the liver tumor segmentation in non-enhanced images by leveraging additional boxlevel-labeled data (labeled with a tumor bounding-box). WSTS deploys a weakly-supervised teacherstudent framework (TCH-ST), namely, a Teacher Module learns to detect and segment the tumor in enhanced images during training, which facilitates a Student Module to detect and segment the tumor in non-enhanced images independently during testing. To detect the tumor accurately, the WSTS proposes a Dual-strategy DRL (DDRL), which develops two tumor detection strategies by creatively introducing a relative-entropy bias in the DRL. To accurately predict a tumor mask for the box-levellabeled enhanced image and thus improve tumor segmentation in non-enhanced images, the WSTS proposes an Uncertainty-Sifting Self-Ensembling (USSE). The USSE exploits the weakly-labeled data with self-ensembling and evaluates the prediction reliability with a newly-designed Multi-scale Uncertaintyestimation. WSTS is validated with a 2D MRI dataset, where the experiment achieves 83.11% of Dice and 85.12% of Recall in 50 patient testing data after training by 200 patient data (half amount data is boxlevel-labeled). Such a great result illustrates the competence of WSTS to segment the liver tumor from non-enhanced images. Thus, WSTS has excellent potential to assist radiologists by liver tumor segmentation without contrast-agents.

无需对比剂（非增强图像）的精确肝脏肿瘤分割避免了与对比剂相关的耗时和高风险，为放射科医生快速且安全地诊断和治疗肝脏肿瘤提供了帮助。然而，由于没有对比剂增强，肝脏图像中的肿瘤对比度低，甚至肉眼看不见。因此，从非增强图像中分割肝脏肿瘤是相当具有挑战性的。我们提出了一种弱监督教师-学生网络（WSTS）来解决非增强图像中的肝脏肿瘤分割问题，通过利用额外的盒级标记数据（用肿瘤边界盒标记）。WSTS 部署了一种弱监督教师-学生框架（TCH-ST），即教师模块在训练期间学习检测和分割增强图像中的肿瘤，从而帮助学生模块在测试期间独立地检测和分割非增强图像中的肿瘤。为了准确检测肿瘤，WSTS 提出了一种双策略 DRL（DDRL），通过创新地引入相对熵偏差到 DRL 中，开发了两种肿瘤检测策略。为了准确预测盒级标记增强图像的肿瘤掩膜，从而改善非增强图像中的肿瘤分割，WSTS 提出了不确定性筛选自组合（USSE）。USSE 利用带有自组合的弱标记数据，并使用新设计的多尺度不确定性估计来评估预测的可靠性。WSTS 通过一个 2D MRI 数据集进行验证，实验在 200 名患者数据（一半为盒级标记数据）训练后，在 50 名患者测试数据中实现了 83.11% 的 Dice 和 85.12% 的 Recall。这样优秀的结果表明 WSTS 能够从非增强图像中分割肝脏肿瘤。因此，WSTS 具有极大的潜力，可以在无需对比剂的情况下帮助放射科医生进行肝脏肿瘤分割。

Methods

方法

The WSTS leverages a Teacher Module to exploit the tumor knowledge in the enhanced image as guidance to train a Student Module so that the Student Module is able to detect and segment the tumor from the non-enhanced image independently in the testing stage (Fig. 5). In detail, 1) the Teacher Module (Section 3.1) deploys the Dual-strategy DRL (DDRL) and the Uncertainty-Sifting Self-Ensembling (USSE) to obtain a tumor mask in the enhanced image. The DDRL deploys two newly-designed Relative-entropybiased Actor-Critics (RACs) to develop tumor detection strategies and determine the tumor location. The USSE designs a Multiscale Uncertainty-estimation (MU) and integrates it with the SelfEnsembling (SE) to predict a pixel-level tumor mask for the boxlevel-label data. 2) The Student Module (Section 3.2) employs a Student DDRL (SDDRL) and a Student DenseUNet (SDUNet) to learn tumor segmentation under the guidance of the Teacher Module in the non-enhanced image. The SDDRL imitates the DDRL to learn tumor detection strategies. The SDUNet utilizes the USSE’s tumor mask as a pseudo-pixel-level label (additional to the manual pixellevel label) to learn the tumor segmentation. 3.1. Teacher module The Teacher Module employs the Dual-strategy DRL (DDRL) and the Uncertainty-Sifting Self-Ensembling (USSE) to detect the tumorWSTS

利用教师模块来利用增强图像中的肿瘤知识作为指导，训练一个学生模块，使学生模块能够在测试阶段独立地从非增强图像中检测和分割肿瘤（见图 5）。具体来说，1）教师模块（第3.1节）部署双策略深度强化学习（DDRL）和不确定性筛选自集成（USSE）来在增强图像中获得肿瘤掩膜。DDRL部署了两个新设计的相对熵偏置的行动者-评论家（RACs）来发展肿瘤检测策略并确定肿瘤位置。USSE设计了一个多尺度不确定性估计（MU），并将其与自集成（SE）集成，为盒子级标记数据预测像素级肿瘤掩膜。2）学生模块（第3.2节）采用学生版双策略深度强化学习（SDDRL）和学生版密集联合网络（SDUNet），在非增强图像中在教师模块的指导下学习肿瘤分割。SDDRL模仿DDRL学习肿瘤检测策略。SDUNet利用USSE的肿瘤掩膜作为伪像素级标签（除了手动像素级标签），来学习肿瘤分割。3.1教师模块，教师模块采用双策略深度强化学习（DDRL）和不确定性筛选自集成（USSE）来检测肿瘤。

Conclusions

结论

This paper proposes a Weakly-Supervised Teacher-Student network (WSTS) to segment the liver tumor from non-enhanced images. WSTS creatively deploys a weakly-supervised teacher-student framework to exploit the visible tumor feature in the enhanced image as guidance and assist the segmentation in the non-enhanced images. In particular, WSTS proposes DDRL to exploit the tumor spatial feature in the enhanced image and transfer the feature via DRL strategies to guide the tumor detection in the non-enhanced image. WSTS also proposes the USSE, which predicts tumor masks for the box-level-labeled enhanced image. Thus the mask is transferred as the tumor shape feature to improve tumor segmentation in the non-enhanced image. The experiment confirms the effectiveness of WSTS, where WSTS achieves 83.11% of Dice and 85.12% of Recall in the 50 patient testing data after training by 200 patients (100 patients with the pixel-level label and 100 patients with the box-level label). These results demonstrate WSTS can be an efficient computer aided-diagnostic tool to assist radiologists with tumor segmentation without contrast-agents.

本文提出了一种用于从非增强图像中分割肝脏肿瘤的弱监督教师-学生网络（WSTS）。WSTS 创新性地部署了一种弱监督的教师-学生框架，利用增强图像中可见的肿瘤特征作为指导，帮助非增强图像中的分割。特别地，WSTS 提出了 DDRL，以利用增强图像中的肿瘤空间特征，并通过 DRL 策略传输这些特征，指导非增强图像中的肿瘤检测。WSTS 还提出了 USSE，用于预测盒级标记的增强图像的肿瘤掩膜。因此，该掩膜作为肿瘤形状特征被转移，以提高非增强图像中的肿瘤分割。实验确认了 WSTS 的有效性，其中 WSTS 在 200 名患者（100 名患者具有像素级标记，100 名患者具有盒级标记）的训练后，在 50 名患者的测试数据中达到了 83.11% 的 Dice 和 85.12% 的 Recall。这些结果表明 WSTS 可以成为一种有效的计算机辅助诊断工具，帮助放射科医生在不使用对比剂的情况下进行肿瘤分割。

Figure

图

Fig. 1. Almost all existing tumor segmentation methods are only suitable for enhanced images where a contrast agent (CA) is used to improve tumor visibility and thus raises several issues (side-effect, time-consuming, and high cost). Also, the firstly-scanned non-enhanced image is wasted. We propose a method to segment the liver tumor from non-enhanced images, which is safe, quick, and economical because it avoids the use of CAs and the second scanning

图 1. 几乎所有现有的肿瘤分割方法仅适用于增强图像，其中使用了对比剂（CA）来提高肿瘤的可见性，因此引发了一些问题（副作用、耗时和高成本）。此外，最初扫描的非增强图像被浪费了。我们提出了一种从非增强图像中分割肝脏肿瘤的方法，这种方法安全、快速、经济，因为它避免了使用对比剂和第二次扫描。

Fig. 2. Tumor segmentation from non-enhanced images facing the following challenges: 1) Some tumors are barely visible to naked eyes. 2) Normal tissues in the background present intricate and tumor-like appearance and thus may cause interference to the tumor segmentation. 3) The pixel-level-labeled data is limited, and the box-level-labeled data is relatively abundant. It is desirable but challenging to deploy the box-level-labeled data for tumor segmentation training.

图 2. 从非增强图像中进行肿瘤分割面临以下挑战：1）一些肿瘤肉眼几乎无法看到。2）背景中的正常组织呈现复杂且类似肿瘤的外观，因此可能对肿瘤分割造成干扰。3）像素级标记数据有限，而盒子级标记数据相对丰富。利用盒子级标记数据进行肿瘤分割训练是可取的但具有挑战性。

Fig. 3. The teacher-student framework draws support from the enhanced image to facilitate the tumor segmentation in non-enhanced images in the training stage. Thus during the testing stage, the student can segment the tumor independently from non-enhanced images

图 3. 教师-学生框架在训练阶段利用增强图像来辅助非增强图像中的肿瘤分割。因此，在测试阶段，学生可以独立地从非增强图像中分割出肿瘤。

Fig. 4. DRL is a candidate to transfer the tumor spatial feature from the enhanced image to the non-enhanced image via a tumor detection strategy. However, traditional DRL may stick into sub-optimal and cause inaccurate tumor detection. SE is a candidate to transfer the tumor shape feature from the enhanced image to the non-enhanced image by predicting pixel-level tumor masks for the box-level-labeled data. However, traditional SE may produce unreliable and noisy masks

图 4. 深度强化学习（DRL）是一种通过肿瘤检测策略将肿瘤空间特征从增强图像转移到非增强图像的候选方法。然而，传统的深度强化学习可能陷入次优状态，导致肿瘤检测不准确。形状编码（SE）是一种通过预测盒子级标记数据的像素级肿瘤掩膜，将肿瘤形状特征从增强图像转移到非增强图像的候选方法。然而，传统的形状编码可能产生不可靠和噪声过多的掩膜。

Fig. 5. Our WSTS leverages a Teacher Module to train a Student Module so that the Student Module is able to segment the tumor from the non-enhanced image independently. The Teacher Module deploys a DDRL and a USSE to detect and segment the tumor in the enhanced image. The Student Module employs an SDDRL and an SDUNet to learn tumor segmentation under the guidance of the Teacher Module in the non-enhanced image

图 5. 我们的WSTS利用一个教师模块来训练一个学生模块，使学生模块能够独立地从非增强图像中分割出肿瘤。教师模块部署了一个深度强化学习检测（DDRL）和一个不确定形状编码（USSE）来在增强图像中检测和分割肿瘤。学生模块采用一个简化的深度强化学习检测（SDDRL）和一个简化的深度学习网络（SDUNet），在非增强图像中在教师模块的指导下学习肿瘤分割。

Fig. 6. The Dual-strategy DRL (DDRL) deploys two newly-proposed RACs to detect the tumor with respective strategy and fuses their detection results as ultimate tumor detection. The DDRL stimulates each RAC to 1) explore various strategies by maximizing the entropy of explored strategy distribution; 2) take the other RAC’s decision into consideration when learning its own strategy by maximizing the relative entropy between the developed strategies.

图 6. 双策略深度强化学习（DDRL）部署了两个新提出的相对熵偏置的行动者-评论家（RACs），以各自的策略检测肿瘤，并融合他们的检测结果作为最终的肿瘤检测。DDRL激励每个RAC：1）通过最大化探索策略分布的熵来探索各种策略；2）在学习自己的策略时考虑另一个RAC的决策，通过最大化已发展策略之间的相对熵。

Fig. 7. The Uncertainty-Sifting Self-Ensembling (USSE) builds an online model and a target model to build the self-consistency loss and exploit the box-level-labeled data. The prediction of the target model sifted by the Multi-scale Uncertainty-estimation (MU) is employed as a reliable prediction to constrain the segmentation of the online model. The MU creatively introduces multi-scale attention into the uncertainty-estimation to increase the observational uncertainty and thus improve the estimation accuracy of the prediction uncertainty.

图 7. 不确定性筛选自集成（USSE）构建了一个在线模型和一个目标模型，以建立自一致性损失并利用盒子级标记数据。目标模型的预测通过多尺度不确定性估计（MU）进行筛选，作为可靠的预测来约束在线模型的分割。MU创新地将多尺度注意力引入到不确定性估计中，以增加观察不确定性，从而提高预测不确定性的估计准确性。

Fig. 8. Tumor segmentation visualization presents that WSTS achieves a high overlap between the tumor prediction (green area) and the ground-truth (blue contour). In the non-enhanced image, the tumor is low-contrast and barely visible compared with that in the enhanced image. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

图 8. 肿瘤分割可视化展示了WSTS在肿瘤预测（绿色区域）和真实值（蓝色轮廓）之间实现了高度重叠。在非增强图像中，与增强图像相比，肿瘤对比度低，几乎不可见。（关于本图例中颜色引用的解释，读者可参考本文的网络版本。）

Fig. 9. The ROC Curves of control experiments (left) and inter-comparison experiments (right) show every part of WSTS increases AUC to varying degrees and WSTS has the largest AUC compared with other fully-supervised methods.

图 9. 控制实验（左）和交叉比较实验（右）的 ROC 曲线显示 WSTS 的每个部分都不同程度地增加了 AUC，而且与其他全监督方法相比，WSTS 的 AUC 最大。

Fig. 10. The accuracy of tumor proposal predicted by DDRL during training under different temperature values, where the proposal accuracy is evaluated by IoU between the proposal and the ground-truth.

图 10. 在不同温度值下，DDRL 在训练期间预测的肿瘤建议的准确性，其中建议的准确性通过建议与真实标记之间的 IoU 来评估。

Fig. 11. Ablation experimental result related to the DRL method shows SAC achieves the best performance among the three DRL algorithms

图 11. 与 DRL 方法相关的消融实验结果显示，在三种 DRL 算法中，SAC 实现了最佳性能。

Fig. 12. Ablation experimental result related to semi-supervised segmentation methods shows the USSE achieves the best performance among the four SE algorithms.

图 12. 与半监督分割方法相关的消融实验结果显示，在四种 SE 算法中，USSE 实现了最佳性能。

Fig. 13. The tumor segmentation comparison of the WSTS and other methods shows the performance of WSTS approaches these fully-supervised methods and even better than the U-Net. The first and second row illustrates enhanced images and the image with mask superimposed. The third row shows non-enhanced images. The fourth row shows the ground-truth, and the rest rows illustrate the segmentation of five methods on non-enhanced images.

图 13. WSTS 与其他方法的肿瘤分割比较显示 WSTS 的性能接近这些全监督方法，甚至优于 U-Net。第一行和第二行展示了增强图像和叠加了掩膜的图像。第三行显示非增强图像。第四行展示真实标记，其余行展示了五种方法在非增强图像上的分割结果。