Abstract
The cryo-electron microscopy (cryo-EM) becomes popular for macromolecular structure determination. However, the 2D images captured by cryo-EM are of high noise and often mixed with multiple heterogeneous conformations and contamination, imposing a challenge for denoising. Traditional image denoising methods and simple denoising autoencoder cannot work well when the signal-to-noise ratio (SNR) of images is meager and contamination distribution is complex. Thus it is desired to develop new effective denoising techniques to facilitate further research such as 3D reconstruction, 2D conformation classification, and so on. In this chapter, we approach the robust denoising problem for cryo-EM images by introducing a family of generative adversarial networks (GANs), called β-GAN, which is able to achieve robust estimation of certain distributional parameters under Huber contamination model with statistical optimality. To address the denoising challenges, for example, the traditional image generative model might be contaminated by a small portion of unknown outliers, β-GANs are exploited to enhance the robustness of denoising autoencoder. Our proposed method is evaluated by both a simulated dataset on the Thermus aquaticus RNA polymerase (RNAP) and a real-world dataset on the Plasmodium falciparum 80S ribosome dataset (EMPIAR-10028), in terms of mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and 3D reconstruction as well. Quantitative comparisons show that equipped with some designs of β-GANs and the robust ℓ1-autoencoder, one can stabilize the training of GANs and achieve the state-of-the-art performance of robust denoising with low SNR data and against possible information contamination. Our proposed methodology thus provides an effective tool for robust denoising of cryo-EM 2D images and helpful for 3D structure reconstruction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
β-GAN has two parameters: α and β, written as (α, β)-GAN in this chapter.
- 2.
We set the same architecture and parameters as https://github.com/nashory/pggan-pytorch and the input image size is 128 × 128.
References
Agostinelli, F., Anderson, M., Lee, H.: Adaptive multi-column deep neural networks with application to robust image denoising. In: Advances in Neural Information Processing Systems, pp. 1493–1501 (2013)
Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54(11), 4311–4322 (2006)
Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: Proceeding of the International Conference on Machine Learning, pp. 214–223 (2017)
Bae, B., Feklistov, A., Lass-Napiorkowska, A., Landick, R., Darst, S.: Structure of a bacterial RNA polymerase holoenzyme open promoter complex. Elife 4, e08504 (2015)
Bai, X.C., McMullan, G., Scheres, S.: How Cryo-EM is revolutionizing structural biology. Trends Biochem. Sci. 40(1), 49–57 (2015)
Baldi, P.: Autoencoders, unsupervised learning, and deep architectures. In: Proceedings of ICML Workshop on Unsupervised and Transfer Learning, JMLR Workshop and Conference Proceedings, pp. 37–49 (2012)
Bau, D., Zhu, J.Y., Wulff, J., Peebles, W., Strobelt, H., Zhou, B., Torralba, A.: Seeing what a gan cannot generate. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4502–4511 (2019)
Bhamre, T., Zhang, T., Singer, A.: Denoising and covariance estimation of single particle Cryo-EM images. J. Struct. Biol. 195(1), 72–81 (2016)
Browning, D., Busby, S.: The regulation of bacterial transcription initiation. Nat. Rev. Microbiol. 2(1), 57–65 (2004)
Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3155–3164 (2018)
Chen, J., Chiu, C., Gopalkrishnan, S., Chen, A., Olinares, P., Saecker, R., Winkelman, J., Maloney, M., Chait, B., Ross, W. et al.: Stepwise promoter melting by bacterial RNA polymerase. Mol. Cell 78, 275–288.e6 (2020)
Cox, M., Cox, T.: Multidimensional scaling. In: Handbook of Data Visualization. Springer, Berlin, pp. 315–347 (2008)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Dai, Z., Yang, Z., Yang, F., Cohen, W.W., Salakhutdinov, R.: Good semi-supervised learning that requires a bad gan. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 6513–6523 (2017)
Dong, Z., Liu, G., Ni, G., Jerwick, J., Duan, L., Zhou, C.: Optical coherence tomography image denoising using a generative adversarial network with speckle modulation. J. Biophotonics 13(4), e201960135 (2020)
Frank, J.: Three-dimensional electron microscopy of macromolecular assemblies: visualization of biological molecules in their native state. Oxford University Press, New York (2006)
Gao, C., Liu, J., Yao, Y., Zhu, W.: Robust estimation and generative adversarial nets. In: Interational Conference on Learning Representation, New Orleans (2019)
Gao, C., Yao, Y., Zhu, W.: Generative adversarial nets for robust scatter estimation: a proper scoring rule perspective. J. Mach. Learn. Res. 21, 160–161 (2020)
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Gu, H., Unarta, I.C., Huang, X., Yao, Y.: Robust autoencoder gan for cryo-em image denoising (2020)
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.: Improved training of Wasserstein GANs. In: Advances in Neural Information Processing Systems, pp. 5767–5777 (2017)
Hua, Y., Li, R., Zhao, Z., Chen, X., Zhang, H.: Gan-powered deep distributional reinforcement learning for resource management in network slicing. IEEE J. Sel. Areas Commun. 38(2), 334–349 (2019)
Huber, P.: Robust estimation of a location parameter. In: Breakthroughs in Statistics. Springer, New York, pp. 492–518 (1992)
Karras, T., Aila, T., Laine, S., Lehtinen, J.: Progressive growing of GANs for improved quality, stability, and variation. In: International Conference on Learning Representation, Vancouver (2018)
Kenzaki, H., Koga, N., Hori, N., Kanada, R., Li, W., Okazaki, K., Yao, X.Q., Takada, S.: CafeMol: a coarse-grained biomolecular simulator for simulating proteins at work. J. Chem. Theory Comput. 7(6), 1979–1989 (2011)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representation, San Diego (2015)
Krivov, G., Shapovalov, M., Dunbrack R.L. Jr.: Improved prediction of protein side-chain conformations with SCWRL4. Proteins: Struct. Funct. Bioinform. 77(4), 778–795 (2009)
Krull, A., Buchholz, T.O., Jug, F.: Noise2void-learning denoising from single noisy images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2129–2137 (2019)
Kühlbrandt, W.: The resolution revolution. Science 343(6178), 1443–1444 (2014)
Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., Aila, T.: Noise2noise: learning image restoration without clean data. In: Proceeding of the International Conference on Machine Learning, pp. 2965–2974 (2018)
Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11), 2579–2605 (2008)
Marabini, R., Masegosa, I., San Martın, M., Marco, S., Fernandez, J., De la Fraga, L., Vaquerizo, C., Carazo, J.: Xmipp: an image processing package for electron microscopy. J. Struct. Biol. 116(1), 237–240 (1996)
Moore, B., Kelley, L., Barber, J., Murray, J., MacDonald, J.: High–quality protein backbone reconstruction from alpha carbons using Gaussian mixture models. J. Comput. Chem. 34(22), 1881–1889 (2013)
Murakami, K., Masuda, S., Darst, S.: Structural basis of transcription initiation: Rna polymerase holoenzyme at 4 å resolution. Science 296(5571), 1280–1284 (2002)
Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Advances in Neural Information Processing Systems, pp. 849–856 (2002)
Okazaki, K., Koga, N., Takada, S., Onuchic, J., Wolynes, P.: Multiple-basin energy landscapes for large-amplitude conformational motions of proteins: structure-based molecular dynamics simulations. Proc. Natl. Acad. Sci. 103(32), 11844–11849 (2006)
Punjani, A., Rubinstein, J.L., Fleet, D.J., Brubaker, M.A.: CryoSPARC: algorithms for rapid unsupervised Cryo-EM structure determination. Nat. Methods 14(3), 290 (2017)
Quan, Y., Chen, M., Pang, T., Ji, H.: Self2self with dropout: Learning self-supervised denoising from single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1890–1898 (2020)
Sarmad, M., Lee, H.J., Kim, Y.M.: Rl-gan-net: a reinforcement learning agent controlled gan network for real-time point cloud shape completion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5898–5907 (2019)
Scheres, S.: Processing of structurally heterogeneous Cryo-EM data in RELION. In: Methods in Enzymology. Elsevier, Academic Press, vol. 579, pp. 125–157 (2016)
Shen, P.: The 2017 Nobel Prize in Chemistry: Cryo-EM comes of age. Anal. Bioanal. Chem. 410(8), 2053–2057 (2018)
Su, M., Zhang, H., Schawinski, K., Zhang, C., Cianfrocco, M.: Generative adversarial networks as a tool to recover structural information from cryo-electron microscopy data. BioRxiv, p. 256792 (2018)
Tenenbaum, J., De Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Tran, L., Nguyen, S.M., Arai, M.: GAN-based noise model for denoising real images. In: Proceedings of the Asian Conference on Computer Vision (2020)
Tripathi, S., Lipton, Z.C., Nguyen, T.Q.: Correction by projection: denoising images with generative adversarial networks (2018)
Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceeding of the International Conference on Machine Learning, pp. 1096–1103 (2008)
Wang, J., Yin, C.C (2013) A zernike-moment-based non-local denoising filter for cryo-em images. Sci. China Life Sci. 56(4), 384–390
Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Wang, F., Gong, H., Liu, G., Li, M., Yan, C., **a, T., Li, X., Zeng, J.: DeepPicker: a deep learning approach for fully automated particle picking in Cryo-EM. J. Struct. Biol. 195(3), 325–336 (2016)
Warren, B.E.: X-Ray Diffraction. Courier Corporation. Dover Publications; Reprint Edition (1990)
Wei, D.Y., Yin, C.C.: An optimized locally adaptive non-local means denoising filter for cryo-electron microscopy data. J. Struct. Biol. 172(3), 211–218 (2010)
Wong, W., Bai, X.C., Brown, A., Fernandez, I., Hanssen, E., Condron, M., Tan, Y.H., Baum, J., Scheres, S.: Cryo-EM structure of the Plasmodium falciparum 80s ribosome bound to the anti-protozoan drug emetine. Elife 3, e03080 (2014)
Wüthrich, K.: NMR with proteins and nucleic acids. Europhys. News 17(1), 11–13 (1986)
**an, Y., Gu, H., Wang, W., Huang, X., Yao, Y., Wang, Y., Cai, J.F.: Data-driven tight frame for cryo-em image denoising and conformational classification. In: Proceeding of the IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 544–548 (2018)
**e, J., Xu, L., Chen, E.: Image denoising and inpainting with deep neural networks. In: Advances in Neural Information Processing Systems, pp. 341–349 (2012)
Yang, Q., Yan, P., Zhang, Y., Yu, H., Shi, Y., Mou, X., Kalra, M., Zhang, Y., Sun, L., Wang, G.: Low-dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss. IEEE Trans. Med. Imaging 37(6), 1348–1357 (2018)
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Zhong, E., Bepler, T., Davis, J., Berger, B.: Reconstructing continuous distributions of 3D protein structure from Cryo-EM images. In: International Conference on Learning Representation, Addis Ababa (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this entry
Cite this entry
Gu, H., **an, Y., Unarta, I.C., Yao, Y. (2023). Generative Adversarial Networks for Robust Cryo-EM Image Denoising. In: Chen, K., Schönlieb, CB., Tai, XC., Younes, L. (eds) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Cham. https://doi.org/10.1007/978-3-030-98661-2_126
Download citation
DOI: https://doi.org/10.1007/978-3-030-98661-2_126
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98660-5
Online ISBN: 978-3-030-98661-2
eBook Packages: Mathematics and StatisticsReference Module Computer Science and Engineering