SpringerLink
Log in

Data-Driven Modeling of Partially Observed Biological Systems

  • Original Paper
  • Published:
Communications on Applied Mathematics and Computation Aims and scope Submit manuscript

Abstract

We present a numerical approach for modeling unknown dynamical systems using partially observed data, with a focus on biological systems with (relatively) complex dynamical behavior. As an extension of the recently developed deep neural network (DNN) learning methods, our approach is particularly suitable for practical situations when (i) measurement data are available for only a subset of the state variables, and (ii) the system parameters cannot be observed or measured at all. We demonstrate that, with a properly designed DNN structure with memory terms, effective DNN models can be learned from such partially observed data containing hidden parameters. The learned DNN model serves as an accurate predictive tool for system analysis. Through a few representative biological problems, we demonstrate that such DNN models can capture qualitative dynamical behavior changes in the system, such as bifurcations, even when the parameters controlling such behavior changes are completely unknown throughout not only the model learning process but also the system prediction process. The learned DNN model effectively creates a “closed” model involving only the observables when such a closed-form model does not exist mathematically.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/. Software available from tensorflow.org

  2. Bongard, J., Lipson, H.: Automated reverse engineering of nonlinear dynamical systems. Proc. Natl. Acad. Sci. USA 104, 9943–9948 (2007)

    Article  Google Scholar 

  3. Chan, S., Elsheikh, A.: A machine learning approach for efficient uncertainty quantification using multiscale methods. J. Comput. Phys. 354, 494–511 (2018)

    Article  MathSciNet  Google Scholar 

  4. Chen, Z., Churchill, V., Wu, K., **u, D.: Deep neural network modeling of unknown partial differential equations in nodal space. J. Comput. Phys. 449, 110782 (2022)

    Article  MathSciNet  Google Scholar 

  5. Chen, Z., **u, D.: On generalized residual network for deep learning of unknown dynamical systems. J. Comput. Phys. 438, 110362 (2021)

    Article  MathSciNet  Google Scholar 

  6. Daniels, B.C., Nemenman, I.: Automated adaptive inference of phenomenological dynamical models. Nature Communications 6, 8133 (2015)

  7. Daniels, B.C., Nemenman, I.: Efficient inference of parsimonious phenomenological models of cellular dynamics using S-systems and alternating regression. PloS One 10, e0119821 (2015)

    Article  Google Scholar 

  8. DeAngelis, D.L., Yurek, S.: Equation-free modeling unravels the behavior of complex ecological systems. Proc. Natl. Acad. Sci. USA 112, 3856–3857 (2015).

  9. E, W.N., Engquist, B., Huang, Z.: Heterogeneous multiscale method: a general methodology for multiscale modeling. Phys. Rev. B 67, 092101 (2003)

    Article  Google Scholar 

  10. Fall, C., Marland, E., Wagner, J., Tyson, J.: Computational Cell Biology. Springer Science+Business Media, Inc., Berlin (2010)

    Google Scholar 

  11. Fu, X., Chang, L.-B., **u, D.: Learning reduced systems via deep neural networks with memory. J. Mach. Learn. Model. Comput. 1, 97–118 (2020)

    Article  Google Scholar 

  12. Fu, X., Mao, W., Chang, L.-B., **u, D.: Modeling unknown dynamical systems with hidden parameters. J. Mach. Learn. Model. Comput. 3, 79-95 (2022)

  13. Giannakis, D., Majda, A.J.: Nonlinear Laplacian spectral analysis for time series with intermittency and low-frequency variability. Proc. Natl. Acad. Sci. USA 109, 2222–2227 (2012)

    Article  MathSciNet  Google Scholar 

  14. Gonzalez-Garcia, R., Rico-Martinez, R., Kevrekidis, I.G.: Identification of distributed parameter systems: a neural net based approach. Comput. Chem. Eng. 22, S965–S968 (1998)

    Article  Google Scholar 

  15. Grimm, V., Railsback, S.F.: Individual-Based Modeling and Ecology. Princeton University Press, Princeton (2005)

  16. Han, J., Jentzen, A., E, W.N.: Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. 115, 8505–8510 (2018)

    Article  MathSciNet  Google Scholar 

  17. Hau, D.T., Coiera, E.W.: Learning qualitative models from physiological signals internal accession date only. Technical report (1995)

  18. Hesthaven, J., Ubbiali, S.: Non-intrusive reduced order modeling of nonlinear problems using neural networks. J. Comput. Phys. 363, 55–78 (2018)

    Article  MathSciNet  Google Scholar 

  19. Hodgkin, A.L., Huxley, A.F.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol. 117, 500–544 (1952).

  20. Karumuri, S., Tripathy, R., Bilionis, I., Panchal, J.: Simulator-free solution of high-dimensional stochastic elliptic partial differential equations using deep neural networks, J. Comp. Phys. 404, 109120 (2019)

  21. Kevrekidis, I.G., Gear, C.W., Hyman, J.M., Kevrekidid, P.G., Runborg, O., Theodoropoulos, C.: Equation-free, coarse-grained multiscale computation: enabling mocroscopic simulators to perform system-level analysis. Commun. Math. Sci. 1, 715–762 (2003)

    Article  MathSciNet  Google Scholar 

  22. Khoo, Y., Lu, J., Ying, L.: Solving parametric PDE problems with artificial neural networks. Euro.Jnl. Appl. Math. 32, 21–435 (2018). ar**v:1707.03351

  23. Long, Z., Lu, Y., Dong, B.: PDE-Net 2.0: learning PDEs from data with a numeric-symbolic hybrid deep network. J. Comput. Phys. 399, 108925 (2019). ar**v:1812.04426 [math.ST]

  24. Long, Z., Lu, Y., Ma, X., Dong, B.: PDE-net: learning PDEs from data. Proceedings of the 35th International Conference on Machine Learning. PMLR 80, 3208–3216 (2018)

  25. Maki, L.W., Keizer, J.: Mathematical analysis of a proposed mechanism for oscillatory insulin secretion in perifused hit-15 cells. Bull. Math. Biol. 57, 569–591 (1995)

    Article  Google Scholar 

  26. Mangan, N.M., Brunton, S.L., Proctor, J.L., Kutz, J.N.: Inferring biological networks by sparse identification of nonlinear dynamics. IEEE Trans. Mol. Biol. Multi-Scale Commun. 2, 52–63 (2016)

    Article  Google Scholar 

  27. Mardt, A., Pasquali, L., Wu, H., Noe, F.: VAMPnets for deep learning of molecular kinetics. Nat. Commun. 9, 5 (2018)

    Article  Google Scholar 

  28. Mori, H.: Transport, collective motion, and Brownian motion. Prog. Theor. Phys. 33, 423–455 (1965)

    Article  Google Scholar 

  29. Morris, C., Lecar, H.: Voltage oscillations in the barnacle giant muscle fiber (1981). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1327511/

  30. Pawar, S., Rahman, S.M., Vaddireddy, H., San, O., Rasheed, A., Vedula, P.: A deep learning enabler for nonintrusive reduced order modeling of fluid flows. Phys. Fluids 31, 085101 (2019)

    Article  Google Scholar 

  31. Perretti, C.T., Munch, S.B., Sugihara, G.: Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data. Proc. Natl. Acad. Sci. USA 110, 5253–5257 (2013). https://doi.org/10.1073/pnas.1216076110

    Article  Google Scholar 

  32. Qin, T., Chen, Z., Jakeman, J., **u, D.: Data-driven learning of non-autonomous systems. SIAM J. Sci. Comput. 43, A1607–A1624 (2021)

    Article  Google Scholar 

  33. Qin, T., Chen, Z., Jakeman, J., **u, D.: Deep learning of parameterized equations with applications to uncertainty quantification. Int. J. Uncertain. Quantif. 11, 63–82 (2021)

    Article  MathSciNet  Google Scholar 

  34. Qin, T., Wu, K., **u, D.: Data driven governing equations approximation using deep neural networks. J. Comput. Phys. 395, 620–635 (2019)

    Article  MathSciNet  Google Scholar 

  35. Raissi, M.: Deep hidden physics models: deep learning of nonlinear partial differential equations. J. Mach. Learn. Res. 19, 1–24 (2018)

    MathSciNet  Google Scholar 

  36. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics informed deep learning (part I): data-driven solutions of nonlinear partial differential equations. J. Comput. Phys. 378, 686–707 (2019). ar**v:1711.10561

  37. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics informed deep learning (part II): data-driven discovery of nonlinear partial differential equations (2017). ar**v:1711.10566

  38. Raissi, M., Perdikaris, P., Karniadakis, G.E.: Multistep neural networks for data-driven discovery of nonlinear dynamical systems (2018). ar**v:1801.01236

  39. Ray, D., Hesthaven, J.: An artificial neural network as a troubled-cell indicator. J. Comput. Phys. 367, 166–191 (2018)

    Article  MathSciNet  Google Scholar 

  40. Rinzel, J., Ermentrout, G.: Analysis of neural excitability and oscillations. In: Koch, C., Segev, I. (eds.) Methods in Neuronal Modeling, 2nd edn., pp. 251–291. MIT Press, Cambridge (1998)

    Google Scholar 

  41. Rudy, S.H., Kutz, J.N., Brunton, S.L.: Deep learning of dynamics and signal-noise decomposition with time-step** constraints. J. Comput. Phys. 396, 483–506 (2019)

    Article  MathSciNet  Google Scholar 

  42. Schmidt, M.D., Lipson, H.: Distilling free-form natural laws from experimental data. Science 324, 81–85 (2009)

    Article  Google Scholar 

  43. Schmidt, M.D., Vallabhajosyula, R.R., Jenkins, J.W., Hood, J.E., Soni, A.S., Wikswo, J.P., Lipson, H.: Automated refinement and inference of analytical models for metabolic networks. Phys. Biol. 8, 055011 (2011)

    Article  Google Scholar 

  44. Sugihara, G., May, R., Ye, H., Hsieh, C., Deyle, E., Fogarty, M., Munch, S.: Detecting causality in complex ecosystems. Science 338, 496–500 (2012)

    Article  Google Scholar 

  45. Sun, Y., Zhang, L., Schaeffer, H.: NeuPDE: neural network based ordinary and partial differential equations for modeling time-dependent data. Proc. Machine Learning Res. 107, 352–372 (2020). ar**v:1908.03190

  46. Tripathy, R., Bilionis, I.: Deep UQ: learning deep neural network surrogate model for high dimensional uncertainty quantification. J. Comput. Phys. 375, 565–588 (2018)

    Article  MathSciNet  Google Scholar 

  47. Tsumoto, K., Kitajima, H., Yoshinaga, T., Aihara, K., Kawakami, H.: Bifurcations in Morris-Lecar neuron model. Neurocomputing 69, 293–316 (2006). https://doi.org/10.1016/j.neucom.2005.03.006

    Article  Google Scholar 

  48. Voss, H.U., Kolodner, P., Abel, M., Kurths, J.: Amplitude equations from spatiotemporal binary-fluid convection data. Phys. Rev. Lett. 83, 3422 (1999)

    Article  Google Scholar 

  49. Wang, Q., Ripamonti, N., Hesthaven, J.: Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism. J. Comput. Phys. 410, 109402 (2020)

    Article  MathSciNet  Google Scholar 

  50. Wang, Y., Shen, Z., Long, Z., Dong, B.: Learning to discretize: solving 1D scalar conservation laws via deep reinforcement learning. Commun. Comput. Phys. 28, 2158–2179 (2020). ar**v:1905.11079

  51. Wood, S.N., Thomas, M.B.: Super-sensitivity to structure in biological models. Proc. R. Soc. B: Biol. Sci. 266, 565–570 (1999). https://doi.org/10.1098/rspb.1999.0673

    Article  Google Scholar 

  52. Wu, K., **u, D.: Data-driven deep learning of partial differential equations in modal space. J. Comput. Phys. 408, 109307 (2020)

    Article  MathSciNet  Google Scholar 

  53. Ye, H., Beamish, R.J., Glaser, S.M., Grant, S.C.H., Hsieh, C., Richards, L.J., Schnute, J.T., Sugihara, G.: Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling. Proc. Natl. Acad. Sci. USA 112, E1569–E1576 (2015)

    Article  Google Scholar 

  54. Yodzis, P.: The indeterminacy of ecological interactions as perceived through perturbation. Technical Report (1988)

  55. Zhu, Y., Zabaras, N.: Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification. J. Comput. Phys. 366, 415–447 (2018)

    Article  MathSciNet  Google Scholar 

  56. Zwanzig, R.: Nonlinear generalized Langevin equations. J. Stat. Phys. 9, 215–220 (1973)

    Article  Google Scholar 

Download references

Funding

This work was partially supported by the NSF (No. DMS-1813071) (Chou) and the AFSOR (No. FA9550-22-1-0011) (**u).

Author information

Authors and Affiliations

  1. Department of Mathematics, The Ohio State University, Columbus, OH, 43221, USA

    Wei-Hung Su, Ching-Shan Chou & Dongbin **u

Authors
  1. Wei-Hung Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching-Shan Chou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongbin **u
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongbin **u.

Ethics declarations

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Su, WH., Chou, CS. & **u, D. Data-Driven Modeling of Partially Observed Biological Systems. Commun. Appl. Math. Comput. 6, 739–754 (2024). https://doi.org/10.1007/s42967-023-00317-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42967-023-00317-2

Keywords

Mathematics Subject Classification

Search

Navigation