Physics-Informed Latent Diffusion for Multimodal Brain MRI Synthesis

Sven Lüpke1, Yousef Yaganeh1,2, Ehsan Adeli3, Nassir Navab1,2, Azade Farshad1,2
1Technical University of Munich, 2Munich Center for Machine Learning, 3Stanford University
5th International Workshop on Multiscale Multimodal Medical Imaging
Paper arXiv Code
Teaser Image

Given T1-weighted, T2-weighted, and FLAIR MRI scans at training time, our model learns to synthesize novel maps of the tissue's proton density (PD), longitudinal relaxation time (T1), and transverse relaxation time (T2).

Abstract

Recent advances in generative models for medical imaging have shown promise in representing multiple modalities. However, the variability in modality availability across datasets limits the general applicability of the synthetic data they produce. To address this, we present a novel physics-informed generative model capable of synthesizing a variable number of brain MRI modalities, including those not present in the original dataset. Our approach utilizes latent diffusion models and a two-step generative process: first, unobserved physical tissue property maps are synthesized using a latent diffusion model, and then these maps are combined with a physical signal model to generate the final MRI scan. Our experiments demonstrate the efficacy of this approach in generating unseen MR contrasts and preserving physical plausibility. Furthermore, we validate the distributions of generated tissue properties by comparing them to those measured in real brain tissue.

Teaser Image

Overview of our model's architecture, which combines a physical MR signal model with a product-of-experts (PoE) multimodal variational autoencoder and a latent diffusion model.

MPRAGE

Teaser Image

Spin Echo

Teaser Image

FLAIR

Teaser Image

Using the tissue property maps generated by our model, we apply different MR signal models with varying acquisition parameters (echo time - repetition time - inversion time) to synthesize contrasts not seen during training.

Citation

@article{lupke2024physics,
  author    = {L{\"u}pke, Sven and Yeganeh, Yousef and Adeli, Ehsan and Navab, Nassir and Farshad, Azade},
  title     = {Physics-Informed Latent Diffusion for Multimodal Brain MRI Synthesis},
  journal   = {5th International Workshop on Multiscale Multimodal Medical Imaging},
  year      = {2024},
}