SpringerLink
Log in

Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis

  • Conference paper
  • First Online:
Advances in Knowledge Discovery and Data Mining (PAKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13937))

Included in the following conference series:

  • 1147 Accesses

Abstract

While recent advances in large-scale foundational computer vision models show promising results, their application to the medical domain has not yet been explored in detail. In this paper, we progress into the realms of large-scale modeling in medical synthesis by proposing Cheff - a foundational cascaded latent diffusion model, which generates highly-realistic chest radiographs providing state-of-the-art quality on a 1-megapixel scale. We further propose MaCheX, which is a unified interface for public chest datasets and forms the largest open collection of chest X-rays up to date. With Cheff conditioned on radiological reports, we further guide the synthesis process over text prompts and unveil the research area of report-to-chest-X-ray generation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 96.29
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 128.39
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    MaChex: https://github.com/saiboxx/machex.

  2. 2.

    Cheff: https://github.com/saiboxx/chexray-diffusion.

References

  1. Ali, H., Murad, S., Shah, Z.: Spot the fake lungs: generating synthetic medical images using neural diffusion models. ar**v:2211.00902 [cs, eess] (2022)

  2. Bustos, A., Pertusa, A., Salinas, J.M., de la Iglesia-Vayá, M.: PadChest: a large chest X-ray image dataset with multi-label annotated reports. Med. Image Anal. 66, 101797 (2020)

    Article  Google Scholar 

  3. Chambon, P., Bluethgen, C., Langlotz, C.P., Chaudhari, A.: Adapting pretrained vision-language foundational models to medical imaging domains. arxiv:2210.04133 [cs] (2022)

  4. Dhariwal, P., Nichol, A.: Diffusion models beat GANs on image synthesis. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  5. Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  6. Falk, T., et al.: U-net: deep learning for cell counting, detection, and morphometry. Nat. Meth. 16, 67–70 (2019)

    Article  Google Scholar 

  7. Glocker, B., Jones, C., Bernhardt, M., Winzeck, S.: Risk of bias in chest X-ray foundation models. ar**v:2209.02965 (2022)

  8. Han, L., Lyu, Y., Peng, C., Zhou, S.K.: Gan-based disentanglement learning for chest x-ray rib suppression. Med. Image Anal. 77, 102369 (2022)

    Article  Google Scholar 

  9. Han, T., et al.: Breaking medical data sharing boundaries by using synthesized radiographs. Sci. Adv. 6, eabb7973 (2020)

    Article  Google Scholar 

  10. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems, vol. 33 (2020)

    Google Scholar 

  11. Irvin, J., et al.: CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33 (2019)

    Google Scholar 

  12. Johnson, A.E.W., et al.: MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Sci. Data 6, 317 (2019)

    Article  Google Scholar 

  13. Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: 6th International Conference on Learning Representations, ICLR (2018)

    Google Scholar 

  14. Liu, F., Wu, X., Ge, S., Fan, W., Zou, Y.: Exploring and distilling posterior and prior knowledge for radiology report generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)

    Google Scholar 

  15. Liu, G., et al.: Clinically accurate chest X-ray report generation. In: Machine Learning for Healthcare Conference (2019)

    Google Scholar 

  16. Nguyen, H.Q., et al.: VinDr-CXR: an open dataset of chest x-rays with radiologist’s annotations. ar**v:2012.15029 [eess] (2022)

  17. Nichol, A., Dhariwal, P.: Improved denoising diffusion probabilistic models. In: International Conference on Machine Learning (2021)

    Google Scholar 

  18. Nichol, A., et al.: GLIDE: towards photorealistic image generation and editing with text-guided diffusion models. In: International Conference on Machine Learning (2022)

    Google Scholar 

  19. Packhäuser, K., Gündel, S., Münster, N., Syben, C., Christlein, V., Maier, A.: Deep learning-based patient re-identification is able to exploit the biometric nature of medical chest X-ray data. Sci. Rep. 12, 14851 (2022)

    Article  Google Scholar 

  20. Packhäuser, K., Folle, L., Thamm, F., Maier, A.: Generation of anonymous chest radiographs using latent diffusion models for training thoracic abnormality classification systems. arxiv:2211.01323 [cs, eess] (2022)

  21. Rajpurkar, P., et al.: CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. ar**v:1711.05225 [cs, stat] (2017)

  22. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text-conditional image generation with CLIP latents. arxiv:2204.06125 [cs] (2022)

  23. Reis, E.P., et al.: BRAX, a Brazilian labeled chest X-ray dataset. Sci. Data 9, 487 (2022)

    Article  Google Scholar 

  24. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022)

    Google Scholar 

  25. Saharia, C., et al.: Photorealistic text-to-image diffusion models with deep language understanding. ar**v:2205.11487 (2022)

  26. Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D.J., Norouzi, M.: Image super-resolution via iterative refinement. IEEE Trans. Pattern Anal. Mach. Intell. 45, 4713–4726 (2022)

    MATH  Google Scholar 

  27. Segal, B., Rubin, D.M., Rubin, G., Pantanowitz, A.: Evaluating the clinical realism of synthetic chest X-rays generated using progressively growing GANs. SN Comput. Sci. 2(4), 1–17 (2021). https://doi.org/10.1007/s42979-021-00720-7

    Article  Google Scholar 

  28. Sellergren, A.B., et al.: Simplified transfer learning for chest radiography models using less data. Radiology 305(2), 454–465 (2022)

    Article  Google Scholar 

  29. Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., Ganguli, S.: Deep unsupervised learning using nonequilibrium thermodynamics. In: International Conference on Machine Learning (2015)

    Google Scholar 

  30. Sundaram, S., Hulkund, N.: GAN-based data augmentation for chest X-ray classification. ar**v:2107.02970 (2021)

  31. Tang, Y., Tang, Y., Zhu, Y., **ao, J., Summers, R.M.: A disentangled generative model for disease decomposition in chest X-rays via normal image synthesis. Med. Image Anal. 67, 101839 (2021)

    Article  Google Scholar 

  32. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  33. Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2017)

    Google Scholar 

  34. Weber, T., Ingrisch, M., Bischl, B., Rügamer, D.: Implicit embeddings via GAN inversion for high resolution chest radiographs. In: Medical Image Computing and Computer Assisted Intervention (MICCAI), Medical Applications with Disentanglement (2022)

    Google Scholar 

  35. Wolleb, J., Bieder, F., Sandkühler, R., Cattin, P.C.: Diffusion models for medical anomaly detection. In: Medical Image Computing and Computer Assisted Intervention (MICCAI), vol. 13438 (2022)

    Google Scholar 

  36. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018)

    Google Scholar 

Download references

Acknowledgments

The authors gratefully acknowledge LMU Klinikum for providing computing resources on their Clinical Open Research Engine (CORE). This work has been partially funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) as part of BERD@NFDI - grant number 460037581.

Author information

Authors and Affiliations

  1. Department of Statistics, LMU Munich, Munich, Germany

    Tobias Weber, Bernd Bischl & David Rügamer

  2. Department of Radiology, University Hospital, LMU Munich, Munich, Germany

    Tobias Weber & Michael Ingrisch

  3. Munich Center for Machine Learning (MCML), Munich, Germany

    Tobias Weber, Michael Ingrisch, Bernd Bischl & David Rügamer

Authors
  1. Tobias Weber
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Ingrisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernd Bischl
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Rügamer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tobias Weber .

Editor information

Editors and Affiliations

  1. Kyoto University, Kyoto, Japan

    Hisashi Kashima

  2. IBM Research, Thomas J. Watson Research Center, Yorktown Heights, NY, USA

    Tsuyoshi Ide

  3. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Chih Peng

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Weber, T., Ingrisch, M., Bischl, B., Rügamer, D. (2023). Cascaded Latent Diffusion Models for High-Resolution Chest X-ray Synthesis. In: Kashima, H., Ide, T., Peng, WC. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2023. Lecture Notes in Computer Science(), vol 13937. Springer, Cham. https://doi.org/10.1007/978-3-031-33380-4_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33380-4_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33379-8

  • Online ISBN: 978-3-031-33380-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation