Title

题目

Realistic morphology-preserving generative modelling of the brain

保持形态真实性的大脑生成建模

01

文献速递介绍

医学影像研究通常受到数据稀缺和可用性的限制。治理、隐私问题和获取成本都限制了医学影像数据的访问，加上深度学习算法对数据的高需求，限制了医疗AI领域的进展。最近，生成模型被用于合成逼真的自然图像，提出了解决数据稀缺问题的潜在方案。但当前的生成模型是否能够合成形态学上正确的样本？在这项工作中，我们展示了一个三维人脑生成模型，该模型在必要的规模上进行训练，以生成多样、逼真、高分辨率且保持形态的样本，并以患者特征（例如年龄和病理）为条件。我们证明了该模型生成的合成样本能够保留生物和疾病表型，并且足够逼真，可以在下游的成熟图像分析工具中使用。虽然所提出的模型在未来有广泛的应用前景，例如异常检测和在有限数据下学习，但其生成能力可以直接缓解数据稀缺、数据可用性有限和算法公平性的问题。

Method

方法

First, we review how the VQ-VAE and transformer pipeline works. Following that we introduce our VQ-VAE and the transformer. Lastly, we detail the implementation of the model (see Fig. 3 for the models’ architecture).

首先，我们回顾VQ-VAE和Transformer管道的工作原理。随后介绍我们的VQ-VAE和Transformer。最后，我们详细说明模型的实现（见图3了解模型的架构）。

Results

结果

We used two datasets, the UKB dataset formed of 39,679 neurologically healthy participants and the ADNI dataset, which encompasses 765 unique participants. Further details on the datasets, preprocessing and augmentation can be found in Supplementary Section A.The methods we are comparing against are the state-of-the-art medical imaging generative models Hierarchical Amortized GAN (HA-GAN)12, CCE-GAN10 and Least Squares GAN (LS-GAN)34. In the ‘Quantitative image fidelity evaluation’ section we present a quantitative evaluation of their realism and in the ‘Morphological evaluation’ section we assess the morphological correctness of the synthetic samples. Finally, we provide a detailed ablation study of our pipeline, showcasing how each design’s choice and model’s scale influenced the results. This can be found in Supplementary Section B.

我们使用了两个数据集：UKB数据集由39,679名神经系统健康的参与者组成，ADNI数据集包含765名独特的参与者。有关数据集、预处理和增强的更多详细信息可以在附录A中找到。

我们比较的方法是当前最先进的医学影像生成模型，包括分层摊销GAN（Hierarchical Amortized GAN，HA-GAN）12、CCE-GAN10和最小二乘GAN（Least Squares GAN，LS-GAN）34。在“量化图像保真度评估”部分，我们对其逼真度进行了定量评估，在“形态学评估”部分，我们评估了合成样本的形态学正确性。最后，我们对我们的流水线进行了详细的消融研究，展示了每个设计选择和模型规模如何影响结果。详细内容可在附录B中找到。

Figure

图

Fig. 1 | A diagram illustrating the present analysis. a Collecting longitudinalclinical data from the Parkinson’s Progression Markers Initiative (PPMI) and Parkinson’s Disease Biomarkers Program (PDBP) cohorts and conducting necessarydata cleaning and preprocessing. b Development of a deep phenotypic progressionembedding (DPPE) model to learn a progression embedding vector for each individual, which encodes his/her PD symptom progression trajectory. cCluster analysiswith the learned embedding vectors to identify PD subtypes, each of which reveal aunique PD progression pattern. d Identifying CSF biomarkers and imaging markersthe discovered PD subtypes. e Construction of PD subtype-specific molecularmodules based on genetic and transcriptomic data, along with human proteinprotein interactome (PPI) network analyses, using network medicine approaches.f In silico drug repurposing based on subtype-specific molecular profiles and validation of drug candidates’ treatment efficiency based on analysis of large-scale realworld patient databases, i.e., the INSIGHT and OneFlorida + . g Architecture of theDPPE model. Specifically, DPPE engaged two Long-Short Term Memory (LSTM)units—one as encoder receiving an individual’s longitudinal clinical records andcompacting them into a low-dimensional embedding space; while another taking theindividual’s embedding vector to reconstruct the original clinical records. DPPE wastrained by minimizing the reconstruction difference.

图1 | 说明本次分析的示意图。a 从帕金森进展标志物计划（PPMI）和帕金森病生物标志物项目（PDBP）队列中收集纵向临床数据，并进行必要的数据清理和预处理。b 开发深度表型进展嵌入（DPPE）模型，为每个个体学习一个进展嵌入向量，该向量编码其PD症状进展轨迹。c 使用学习到的嵌入向量进行聚类分析，识别PD亚型，每个亚型揭示出独特的PD进展模式。d 识别所发现PD亚型的脑脊液（CSF）生物标志物和成像标志物。e 基于遗传和转录组数据构建PD亚型特异性分子模块，并结合人类蛋白-蛋白相互作用（PPI）网络分析，采用网络医学方法。f 基于亚型特异性分子特征进行计算机模拟药物再利用，并通过分析大规模真实世界患者数据库（如INSIGHT和OneFlorida+）验证药物候选者的治疗效果。g DPPE模型的架构。具体来说，DPPE使用了两个长短期记忆（LSTM）单元——一个作为编码器接收个体的纵向临床记录并将其压缩到低维嵌入空间；另一个则接收个体的嵌入向量以重建原始临床记录。DPPE通过最小化重建差异进行训练。

Fig. 2 | Random synthetic samples from ADNI dataset. Random synthetic samples from the LS-GAN, CCE-GAN and HA-GAN baseline trained on ADNI together with our proposed model trained on ADNI and the a real participant from the datasets. All three planes of visualization (axial, coronal and sagittal) are presented. Additionally an axial zoomed in visualization of the cerebellum is showcased due to the higher number of cortical folds entailing more highfrequency details. All the visualization planes are from the same synthetic samples and real participant.

图2 | 来自ADNI数据集的随机合成样本。显示了从LS-GAN、CCE-GAN和HA-GAN基线模型（在ADNI数据集上训练）生成的随机合成样本，以及我们提出的模型在ADNI数据集上训练生成的样本和数据集中一位真实参与者的样本。所有三个可视化平面（轴状、冠状和矢状）均已展示。此外，还展示了小脑的轴向放大可视化，因为小脑具有较多的皮层褶皱，包含更多高频细节。所有可视化平面均来自相同的合成样本和真实参与者。

Fig. 3 | VBM t-statistics maps of grey matter for the ADNI pathological dataset. Maps for all models and real data are shown. The displayed t-statistics range is [0, 8] (colour scale) and is based on the VBM of the real data. The t-statistics were corrected following the procedure used in ref. 37.

图3 | ADNI病理数据集的灰质VBM t-统计图。显示了所有模型和真实数据的t-统计图。显示的t-统计量范围是[0, 8]（颜色尺度），基于真实数据的VBM。t-统计量根据参考文献37中的程序进行了校正。

Table

表

Table 1 | Quantitative evaluation of the image fidelity and diversity of synthetic samples

表1 | 合成样本图像保真度和多样性的定量评估

Table 2 | Evaluation of the image fidelity and diversity of the synthetic samples across multiple modalities

表2 | 多种模态下合成样本图像保真度和多样性的评估

Table 3 | Tissue volumes based on SynthSeg segmentations

表3 | 基于SynthSeg分割的组织体积