Abstract
Recurrent neural networks (RNNs) were designed for dealing with time-series data and have recently been used for creating predictive models from functional magnetic resonance imaging (fMRI) data. However, gathering large fMRI datasets for learning is a difficult task. Furthermore, network interpretability is unclear. To address these issues, we utilize multitask learning and design a novel RNN-based model that learns to discriminate between classes while simultaneously learning to generate the fMRI time-series data. Employing the long short-term memory (LSTM) structure, we develop a discriminative model based on the hidden state and a generative model based on the cell state. The addition of the generative model constrains the network to learn functional communities represented by the LSTM nodes that are both consistent with the data generation as well as useful for the classification task. We apply our approach to the classification of subjects with autism vs. healthy controls using several datasets from the Autism Brain Imaging Data Exchange. Experiments show that our jointly discriminative and generative model improves classification learning while also producing robust and meaningful functional communities for better model understanding.
This work was supported by NIH grants R01MH100028 and R01NS035193.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abraham, A., et al.: Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example. Neuroimage 147, 736–745 (2017)
Adate, A., Tripathy, B.: S-LSTM-GAN: shared recurrent neural networks with adversarial training. In: 2nd International Conference on Data Engineering and Communication Technology (2019)
Carroll, J., Chang, J.: Analysis of individual differences in multidimensional scaling via an n-way generalization of eckart-young decomposition. Psychometrika 35, 283–319 (1970)
Caruana, R.: Mach. Learn. 28, 41 (1997). https://doi.org/10.1023/A:1007379606734
Craddock, C., et al.: The neuro bureau preprocessing initiative: open sharing of preprocessed neuroimaging data and derivatives. In: Neuroinformatics (2013)
Di Martino, A., et al.: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol. Psychiatry 16, 659 (2014)
Dvornek, N.C., Ventola, P., Pelphrey, K.A., Duncan, J.S.: Identifying autism from resting-state fMRI using long short-term memory networks. In: Wang, Q., Shi, Y., Suk, H.-I., Suzuki, K. (eds.) MLMI 2017. LNCS, vol. 10541, pp. 362–370. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67389-9_42
Graves, A.: Generating sequences with recurrent neural networks (2014). https://arxiv.org/abs/1308.0850
Güçlü, U., van Gerven, M.A.J.: Modeling the dynamics of human brain activity with recurrent neural networks. Front. Comput. Neurosci. 11, 7 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. In: Neural Computation (1997)
Jun, E., Kang, E., Choi, J., Suk, H.I.: Modeling regional dynamics in low-frequency fluctuation and its application to autism spectrum disorder diagnosis. NeuroImage 184, 669–686 (2019)
Kaiser, M., et al.: Neural signatures of autism. Proc. Natl. Acad. Sci. USA 107, 21223–21228 (2010)
Li, H., Parikh, N.A., He, L.: A novel transfer learning approach to enhance deep neural network classification of brain functional connectomes. Front. Neurosci. 12, 491 (2018)
Li, H., Fan, Y.: Brain decoding from functional MRI using long short-term memory recurrent neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 320–328. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_37
Yarkoni, T., Poldrack, R.A., Nichols, T.E., Van Essen, D.C., Wager, T.D.: Large-scale automated synthesis of human functional neuroimaging data. Nature Methods (2011). www.neurosynth.org
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Dvornek, N.C., Li, X., Zhuang, J., Duncan, J.S. (2019). Jointly Discriminative and Generative Recurrent Neural Networks for Learning from fMRI. In: Suk, HI., Liu, M., Yan, P., Lian, C. (eds) Machine Learning in Medical Imaging. MLMI 2019. Lecture Notes in Computer Science(), vol 11861. Springer, Cham. https://doi.org/10.1007/978-3-030-32692-0_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-32692-0_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32691-3
Online ISBN: 978-3-030-32692-0
eBook Packages: Computer ScienceComputer Science (R0)