SpringerLink
Log in

Optimal experimental design for linear time invariant state–space models

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

The linear time invariant state–space model representation is common to systems from several areas ranging from engineering to biochemistry. We address the problem of systematic optimal experimental design for this class of model. We consider two distinct scenarios: (i) steady-state model representations and (ii) dynamic models described by discrete-time representations. We use our approach to construct locally D-optimal designs by incorporating the calculation of the determinant of the Fisher Information Matrix and the parametric sensitivity computation in a Nonlinear Programming formulation. A global optimization solver handles the resulting numerical problem. The Fisher Information Matrix at convergence is used to determine model identifiability. We apply the methodology proposed to find approximate and exact optimal experimental designs for static and dynamic experiments for models representing a biochemical reaction network where the experimental purpose is to estimate kinetic constants.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Anderson, J., Chang, Y.C., Papachristodoulou, A.: Model decomposition and reduction tools for large-scale networks in systems biology. Automatica 47(6), 1165–1174 (2011). https://doi.org/10.1016/j.automatica.2011.03.010

    Article  MathSciNet  MATH  Google Scholar 

  • Asprey, S., Macchietto, S.: Statistical tools for optimal dynamic model building. Comput. Chem. Eng. 24(2), 1261–1267 (2000)

    Google Scholar 

  • Atkinson, A.C., Donev, A.N., Tobias, R.D.: Optimum Experimental Designs, with SAS. Oxford University Press, Oxford (2007)

    MATH  Google Scholar 

  • Banga, J.R., Versyck, K.J., Van Impe, J.F.: Computation of optimal identification experiments for nonlinear dynamic process models: a stochastic global optimization approach. Ind. Eng. Chem. Res. 41, 2425–2430 (2002)

    Google Scholar 

  • Barz, T., López Cárdenas, D.C., Arellano-Garcia, H., Wozny, G.: Experimental evaluation of an approach to online redesign of experiments for parameter determination. AIChE J. 59(6), 1981–1995 (2013)

    Google Scholar 

  • Bay, J.: Fundamentals of Linear State Space Systems. WCB/McGraw-Hill, New York (1999)

    Google Scholar 

  • Bellu, G., Saccomani, M.P., Audoly, S., D’Angiò, L.: DAISY: a new software tool to test global identifiability of biological and physiological systems. Comput. Methods Programs Biomed. 88(1), 52–61 (2007)

    MATH  Google Scholar 

  • Boer, E., Hendrix, E.: Global optimization problems in optimal design of experiments in regression models. J. Glob. Optim. 18, 385–398 (2000)

    MathSciNet  MATH  Google Scholar 

  • Bouvin, J., Cajot, S., D’Huys, P.J., Ampofo-Asiama, J., Anné, J., Van Impe, J., Geeraerd, A., Bernaerts, K.: Multi-objective experimental design for 13C-based metabolic flux analysis. Math. Biosci. 268, 22–30 (2015)

    MathSciNet  MATH  Google Scholar 

  • Brown, M., He, F., Yeung, L.F.: Robust measurement selection for biochemical pathway experimental design. Int. J. Bioinform. Res. Appl. 4(4), 400–416 (2008)

    Google Scholar 

  • Bryson, A.: Dynamic Optimization. Bibliyografya Ve Indeks. Addison Wesley Longman, Boston (1999)

    Google Scholar 

  • Byrd, R.H., Hribar, M.E., Nocedal, J.: An interior point algorithm for large-scale nonlinear programming. SIAM J. Optim. 9(4), 877–900 (1999)

    MathSciNet  MATH  Google Scholar 

  • Chaloner, K., Larntz, K.: Optimal Bayesian design applied to logistic regression experiments. J. Stat. Plan. Inference 59, 191–208 (1989)

    MathSciNet  MATH  Google Scholar 

  • Chen, R.B., Chang, S.P., Wang, W., Tung, H.C., Wong, W.K.: Minimax optimal designs via particle swarm optimization methods. Stat. Comput. 25(5), 975–988 (2015)

    MathSciNet  MATH  Google Scholar 

  • Chis, O.T., Villaverde, A.F., Banga, J.R., Balsa-Canto, E.: On the relationship between sloppiness and identifiability. Math. Biosci. 282, 147–161 (2016)

    MathSciNet  MATH  Google Scholar 

  • Cobelli, C., Thomaseth, K.: Optimal input design for identification of compartmental models. Theory and application to a model of glucose kinetics. Math. Biosci. 77(1), 267–286 (1985)

    MathSciNet  MATH  Google Scholar 

  • Coleman, T.F., Li, Y.: On the convergence of reflective Newton methods for large-scale nonlinear minimization subject to bounds. Math. Program. 67(2), 189–224 (1994)

    MATH  Google Scholar 

  • Crampin, E.J., Schnell, S., McSharry, P.E.: Mathematical and computational techniques to deduce complex biochemical reaction mechanisms. Prog. Biophys. Mol. Biol. 86(1), 77–112 (2004)

    Google Scholar 

  • Draper, N.R., Hunter, W.G.: Design of experiments for parameter estimation in multiresponse situations. Biometrika 53(3/4), 525–533 (1966)

    MathSciNet  MATH  Google Scholar 

  • Drud, A.: CONOPT: a GRG code for large sparse dynamic nonlinear optimization problems. Math. Program. 31, 153–191 (1985)

    MathSciNet  MATH  Google Scholar 

  • Drud, A.: CONOPT—a large-scale GRG code. ORSA J. Comput. 6(2), 207–216 (1994)

    MATH  Google Scholar 

  • Duarte, B.P.M., Wong, W.K.: A semi-infinite programming based algorithm for finding minimax optimal designs for nonlinear models. Stat. Comput. 24(6), 1063–1080 (2014)

    MathSciNet  MATH  Google Scholar 

  • Duarte, B.P.M., Wong, W.K.: Finding Bayesian optimal designs for nonlinear models: a semidefinite programming-based approach. Int. Stat. Rev. 83(2), 239–262 (2015)

    MathSciNet  Google Scholar 

  • Duarte, B.P.M., Wong, W.K., Atkinson, A.C.: A semi-infinite programming based algorithm for determining \(T-\)optimum designs for model discrimination. J. Multivar. Anal. 135, 11–24 (2015)

    MathSciNet  MATH  Google Scholar 

  • Duarte, B.P.M., Wong, W.K., Oliveira, N.M.C.: Model-based optimal design of experiments—semidefinite and nonlinear programming formulations. Chemometr. Intell. Lab. Syst. 151, 153–163 (2016)

    Google Scholar 

  • Duarte, B.P.M., Wong, W.K., Dette, H.: Adaptive grid semidefinite programming for finding optimal designs. Stat. Comput. 28(2), 441–460 (2018)

    MathSciNet  MATH  Google Scholar 

  • Duarte, B.P.M., Granjo, J.F.O., Wong, W.K.: Optimal exact designs of experiments via mixed integer nonlinear programming. Stat. Comput. 30, 93–112 (2020)

    MathSciNet  MATH  Google Scholar 

  • Eisenberg, M.C., Hayashi, M.A.: Determining identifiable parameter combinations using subset profiling. Math. Biosci. 256, 116–126 (2014)

    MathSciNet  MATH  Google Scholar 

  • Espie, D., Macchietto, S.: The optimal design of dynamic experiments. AIChE J. 35(2), 223–229 (1989)

    Google Scholar 

  • Fedorov, V.V.: The design of experiments in the multiresponse case. Theory Probab. Appl. 16, 323–332 (1971)

    MATH  Google Scholar 

  • Fedorov, V.V.: Theory of Optimal Experiments. Academic Press, Cambridge (1972)

    Google Scholar 

  • Fedorov, V.V., Leonov, S.L.: Optimal Design for Nonlinear Response Models. Chapman and Hall/CRC Press, Boca Raton (2014)

    MATH  Google Scholar 

  • Franklin, G.F., Powell, J.D., Workman, M.L.: Digital Control of Dynamic Systems, 3rd edn. Addison-Wesley, Reading (1990)

    MATH  Google Scholar 

  • Frøysa, H.G., Skaug, H.J., Alendal, G.: Experimental design for parameter estimation in steady-state linear models of metabolic networks. Math. Biosci. 319, 108291 (2020)

    MathSciNet  MATH  Google Scholar 

  • Gaivoronski, A.: Linearization methods for optimization of functionals which depend on probability measures. In: Prékopa, A., Wets, R.J.B. (eds.) Stochastic Programming 84 Part II, Mathematical Programming Studies, vol. 28, pp. 157–181. Springer, Berlin (1986)

    Google Scholar 

  • Galvanin, F., Boschiero, A., Barolo, M., Bezzo, F.: Model-based design of experiments in the presence of continuous measurement systems. Ind. Eng. Chem. Res. 50(4), 2167–2175 (2011)

    Google Scholar 

  • Galvanin, F., Barolo, M., Pannocchia, G., Bezzo, F.: Online model-based redesign of experiments with erratic models: a disturbance estimation approach. Comput. Chem. Eng. 42, 138–151 (2012)

    Google Scholar 

  • GAMS Development Corporation: GAMS—A User’s Guide, GAMS Release 24.2.1. GAMS Development Corporation, Washington, DC, USA (2013)

  • Gill, P.E., Murray, W., Saunders, M.A.: SNOPT: an SQP algorithm for large-scale constrained optimization. SIAM Rev. 47(1), 99–131 (2005)

    MathSciNet  MATH  Google Scholar 

  • Golub, G., Van Loan, C.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences. Johns Hopkins University Press, Baltimore (2013)

    Google Scholar 

  • Goodwin, G.C., Payne, R.L.: Dynamic System Identification. Experiment Design and Data Analysis. Academic Press, New York (1977)

    MATH  Google Scholar 

  • Guillaume, J.H., Jakeman, J.D., Marsili-Libelli, S., Asher, M., Brunner, P., Croke, B., Hill, M.C., Jakeman, A.J., Keesman, K.J., Razavi, S., Stigter, J.D.: Introductory overview of identifiability analysis: a guide to evaluating whether you have the right type of data for your modeling purpose. Environ. Modell. Softw. 119, 418–432 (2019)

    Google Scholar 

  • Hangos, K.M., Szederkényi, G., Alonso, A.A.: Reaction kinetic form for lumped process system models. IFAC Proc. Vol. 46(14), 48–53 (2013)

    Google Scholar 

  • Harman, R., Jurík, T.: Computing \(c-\)optimal experimental designs using the Simplex method of linear programming. Comput. Stat. Data Anal. 53(2), 247–254 (2008)

    MathSciNet  MATH  Google Scholar 

  • Heredia-Langner, A., Montgomery, D.C., Carlyle, W.M., Borror, C.M.: Model-robust optimal designs: a genetic algorithm approach. J. Qual. Technol. 36, 263–279 (2004)

    Google Scholar 

  • Hoang, M.D., Barz, T., Merchan, V.A., Biegler, L.T., Arellano-Garcia, H.: Simultaneous solution approach to model-based experimental design. AIChE J. 59(11), 4169–4183 (2013)

    Google Scholar 

  • Kalaba, R.E., S**arn, K.: Optimal inputs and sensitivities for parameter estimation. J. Optim. Theory Appl. 11, 56–67 (1973). https://doi.org/10.1007/BF00934291

    Article  MathSciNet  MATH  Google Scholar 

  • Kalaba, R., S**arn, K.: Control, Identification, and Input Optimization. Springer, Berlin (1982)

    MATH  Google Scholar 

  • Kiefer, J.: General equivalence theory for optimum design (approximate theory). Ann. Stat. 2, 849–879 (1974)

    MathSciNet  MATH  Google Scholar 

  • Kiefer, J., Wolfowitz, J.: The equivalence of two extremum problem. Can. J. Math. 12, 363–366 (1960)

    MathSciNet  MATH  Google Scholar 

  • Körkel, S., Kostina, E., Bock, H.G., Schlöder, J.P.: Numerical methods for optimal control problems in design of robust optimal experiments for nonlinear dynamic processes. Optim. Methods Softw. 19(3–4), 327–338 (2004)

    MathSciNet  MATH  Google Scholar 

  • Ljung, L.: System Identification: Theory for the User, 2nd edn. Prentice-Hall Inc, Hoboken (1999)

    MATH  Google Scholar 

  • Loskot, P., Atitey, K., Mihaylova, L.: Comprehensive review of models and methods for inferences in bio-chemical reaction networks. Front. Genet. 10, 549 (2019). https://doi.org/10.3389/fgene.2019.00549

    Article  Google Scholar 

  • Maidens, J., Arcak, M.: Semidefinite relaxations in optimal experiment design with application to substrate injection for hyperpolarized MRI. In: 2016 American Control Conference (ACC), pp. 2023–2028 (2016)

  • Masoudi, E., Holling, H., Duarte, B.P.M., Wong, W.K.: A metaheuristic adaptive cubature based algorithm to find Bayesian optimal designs for nonlinear models. J. Comput. Graph. Stat. 28, 1–16 (2019)

    MathSciNet  Google Scholar 

  • Mehra, R.: Optimal input signals for parameter estimation in dynamic systems-Survey and new results. IEEE Trans. Autom. Control 19(6), 753–768 (1974)

    MathSciNet  MATH  Google Scholar 

  • Molchanov, I., Zuyev, S.: Steepest descent algorithm in a space of measures. Stat. Comput. 12, 115–123 (2002)

    MathSciNet  Google Scholar 

  • Ng, T.S., Goodwin, G.C.: On optimal choice of sampling strategies for linear system identification. Int. J. Control 23(4), 459–475 (1976)

    MATH  Google Scholar 

  • Papp, D.: Optimal designs for rational function regression. J. Am. Stat. Assoc. 107, 400–411 (2012)

    MathSciNet  MATH  Google Scholar 

  • Perry, M., Wynn, H., Bates, R.: Principal components analysis in sensitivity studies of dynamic systems. Probab. Eng. Mech. 21(4), 454–460 (2006)

    Google Scholar 

  • Pronzato, L.: Optimal experimental design and some related control problems. Automatica 44, 303–325 (2008)

    MathSciNet  MATH  Google Scholar 

  • Pronzato, L., Pázman, A.: Algorithms: a survey. In: Design of Experiments in Nonlinear Models: Asymptotic Normality, Optimality Criteria and Small-Sample Properties, pp. 277–333. Springer New York (2013). https://doi.org/10.1007/978-1-4614-6363-4_9

  • Pronzato, L., Zhigljavsky, A.A.: Algorithmic construction of optimal designs on compact sets for concave and differentiable criteria. J. Stat. Plan. Inference 154, 141–155 (2014)

    MathSciNet  MATH  Google Scholar 

  • Pukelsheim, F.: Optimal Design of Experiments. SIAM, Philadelphia (1993)

    MATH  Google Scholar 

  • Pukelsheim, F., Rieder, S.: Efficient rounding of approximate designs. Biometrika 79, 763–770 (1992)

    MathSciNet  Google Scholar 

  • Rudolph, P.E., Herrendörfer, G.: Optimal experimental design and accuracy of parameter estimation for nonlinear regression models used in long-term selection. Biom. J. 37(2), 183–190 (1995). https://doi.org/10.1002/bimj.4710370209

    Article  MathSciNet  MATH  Google Scholar 

  • Ruszczyński, A.: Nonlinear Optimization. Princeton University Press, Princeton (2006)

    MATH  Google Scholar 

  • Saccomani, M.P., Audoly, S., Bellu, G., D’Angiò, L., Cobelli, C.: Global identifiability of nonlinear model parameters. IFAC Proc. Vol. 30(11), 233–238 (1997)

    MATH  Google Scholar 

  • Sagnol, G.: Plans d’expériences optimaux et application à l’estimation des matrices de trafic dans les grands réseaux. Ph.D. thesis, L’École Nationale Supérieure des Mines de Paris (2010)

  • Sagnol, G.: Computing optimal designs of multiresponse experiments reduces to second-order cone programming. J. Stat. Plan. Inference 141(5), 1684–1708 (2011)

    MathSciNet  MATH  Google Scholar 

  • Sagnol, G., Harman, R.: Computing exact \(D-\)optimal designs by mixed integer second order cone programming. Ann. Stat. 43(5), 2198–2224 (2015a)

    MathSciNet  MATH  Google Scholar 

  • Sagnol, G., Harman, R.: Optimal designs for steady-state Kalman filters. In: Steland, A., Rafajłowicz, E., Szajowski, K. (eds.) Stochastic Models, Statistics and Their Applications, pp. 149–157. Springer, Cham (2015)

    Google Scholar 

  • Silvey, S.D.: Optimal Design. Chapman & Hall, London (1980)

    MATH  Google Scholar 

  • Singhal, H., Michailidis, G.: Optimal experiment design in a filtering context with application to sampled network data. Ann. Appl. Stat. 4(1), 78–93 (2010)

    MathSciNet  MATH  Google Scholar 

  • Telen, D., Logist, F., Quirynen, R., Houska, B., Diehl, M., Van Impe, J.: Optimal experiment design for nonlinear dynamic (bio)chemical systems using sequential semidefinite programming. AIChE J. 60(5), 1728–1739 (2014)

    Google Scholar 

  • Telen, D., Nimmegeers, P., Impe, J.V.: Uncertainty in optimal experiment design: comparing an online versus offline approaches. IFAC-PapersOnLine 51(2), 771–776 (2018)

    Google Scholar 

  • Titterington, D.M.: Aspects of optimal design in dynamic systems. Technometrics 22(3), 287–299 (1980)

    MathSciNet  MATH  Google Scholar 

  • Ugray, Z., Lasdon, L., Plummer, J., Glover, F., Kelly, J., Martí, R.: A multistart scatter search heuristic for smooth nlp and minlp problems. In: Metaheuristic Optimization via Memory and Evolution, pp. 25–51. Springer (2005)

  • van der Schaft, A.J., Rao, S., Jayawardhana, B.: A network dynamics approach to chemical reaction networks. Int. J. Control 89(4), 731–745 (2016)

    MathSciNet  MATH  Google Scholar 

  • Vandenberghe, L., Boyd, S.: Applications of semidefinite programming. Appl. Numer. Math. 29, 283–299 (1999)

    MathSciNet  MATH  Google Scholar 

  • Varma, A., Palsson, B.O.: Metabolic flux balancing: basic concepts, scientific and practical use. Bio/Technology 12(10), 994–998 (1994). https://doi.org/10.1038/nbt1094-994

    Article  Google Scholar 

  • Villaverde, A.F.: Observability and structural identifiability of nonlinear biological systems. Complexity 8497093, 1–12 (2019)

    MATH  Google Scholar 

  • Walter, E.: Identifiability of State Space Models: With Applications to Transformation Systems. Lecture Notes in Biomathematics. Springer, Berlin (2013)

    Google Scholar 

  • Walter, E., Pronzato, L.: How to design experiments that are robust to parameter uncertainty. IFAC Proc. Vol. 18(5), 921–926 (1985)

    Google Scholar 

  • Walter, E., Pronzato, L.: Qualitative and quantitative experiment design for nonlinear models. IFAC Proc. Vol. 21(1), 69–80 (1988)

    Google Scholar 

  • Whittle, P.: Some general points in the theory of optimal experimental design. J. R. Stat. Soc. Ser. B 35, 123–130 (1973)

    MathSciNet  MATH  Google Scholar 

  • Wiechert, W., Möllney, M., Petersen, S., de Graaf, A.A.: A universal framework for 13C metabolic flux analysis. Metab. Eng. 3, 265–283 (2001)

    Google Scholar 

  • Woods, D.C.: Robust designs for binary data: applications of simulated annealing. J. Stat. Comput. Simul. 80(1), 29–41 (2010)

    MathSciNet  MATH  Google Scholar 

  • Yang, M., Biedermann, S., Tang, E.: On optimal designs for nonlinear models: a general and efficient algorithm. J. Am. Stat. Assoc. 108(504), 1411–1420 (2013)

    MathSciNet  MATH  Google Scholar 

  • Zarrop, M.B.: Optimal Experimental Design for Dynamic System Identification: Lecture Notes in Control and Information Sciences 21. Springer, New York (1979)

    Google Scholar 

Download references

Acknowledgements

We wish to thank to the Associate Editor and two anonymous reviewers for their many comments and suggestions that helped to improve the quality of the paper.

Author information

Authors and Affiliations

  1. Instituto Superior de Engenharia de Coimbra, Department of Chemical and Biological Engineering, Instituto Politécnico de Coimbra, Rua Pedro Nunes, Quinta da Nora, 3030–199, Coimbra, Portugal

    Belmiro P. M. Duarte

  2. CIEPQPF, Department of Chemical Engineering, University of Coimbra, Rua Sílvio Lima – Pólo II, 3030-790, Coimbra, Portugal

    Belmiro P. M. Duarte & Nuno M. C. Oliveira

  3. Department of Statistics, London School of Economics, London, WC2A 2AE, UK

    Anthony C. Atkinson

Authors
  1. Belmiro P. M. Duarte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anthony C. Atkinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nuno M. C. Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Belmiro P. M. Duarte.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duarte, B.P.M., Atkinson, A.C. & Oliveira, N.M.C. Optimal experimental design for linear time invariant state–space models. Stat Comput 31, 45 (2021). https://doi.org/10.1007/s11222-021-10020-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11222-021-10020-y

Keywords

Mathematics Subject Classification

Search

Navigation