SpringerLink
Log in

Input Uncertainty in Stochastic Simulation

  • Chapter
  • First Online:
The Palgrave Handbook of Operations Research

  • 1781 Accesses

Abstract

Stochastic simulation requires input probability distributions to model systems with random dynamic behavior. Given the input distributions, random behavior is simulated using Monte Carlo techniques. This randomness means that statistical characterizations of system behavior based on finite-length simulation runs have Monte Carlo error. Simulation output analysis and optimization methods that account for Monte Carlo error have been in place for many years. But there is a second source of uncertainty in characterizing system behavior that results from error in estimating the input probability distributions. When the input distributions represent real-world phenomena but are determined based on finite samples of real-world data, sampling error gives imperfect characterization of these distributions. This estimation error propagates to simulated system behavior causing what we call input uncertainty. This chapter summarizes the relatively recent development of methods for simulation output analysis and optimization that take both input uncertainty and Monte Carlo error into account.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 181.89
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 235.39
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info
Hardcover Book
EUR 235.39
Price includes VAT (Germany)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ankenman, B., Nelson, B. L., and Staum, J. (2010). Stochastic kriging for simulation metamodeling. Operations Research, 58(2):371–382.

    Article  Google Scholar 

  2. Asmussen, S. and Glynn, P. W. (2007). Stochastic Simulation: Algorithms and Analysis, volume 57. Springer Science & Business Media.

    Google Scholar 

  3. Bai, Y., Balch, T., Chen, H., Dervovic, D., Lam, H., and Vyetrenko, S. (2021). Calibrating over-parametrized simulation models: A framework via eligibility set. ar**v preprint ar**v:2105.12893.

  4. Balci, O. and Sargent, R. G. (1982). Some examples of simulation model validation using hypothesis testing. In Proceedings of the 14th Winter Simulation Conference, volume 2, pages 621–629.

    Google Scholar 

  5. Barton, R. R. and Schruben, L. W. (2001). Resampling methods for input modeling. In Proceedings of the 2001 Winter Simulation Conference, pages 372–378. IEEE.

    Google Scholar 

  6. Barton, R. R. (2012). Tutorial: Input uncertainty in outout analysis. In Proceedings of the 2012 Winter Simulation Conference, pages 67–78. IEEE.

    Google Scholar 

  7. Barton, R. R., Chick, S. E., Cheng, R. C., Henderson, S. G., Law, A. M., Schmeiser, B. W., Leemis, L. M., Schruben, L. W., and Wilson, J. R. (2002). Panel discussion on current issues in input modeling. In Proceedings of the 2002 Winter Simulation Conference, pages 353–369. IEEE.

    Google Scholar 

  8. Barton, R. R., Lam, H., and Song, E. (2018). Revisiting direct bootstrap resampling for input model uncertainty. In Proceedings of the 2018 Winter Simulation Conference, pages 1635–1645. IEEE.

    Google Scholar 

  9. Barton, R. R., Nelson, B. L., and **e, W. (2014). Quantifying input uncertainty via simulation confidence intervals. INFORMS Journal on Computing, 26(1):74–87.

    Google Scholar 

  10. Barton, R. R. and Schruben, L. W. (1993). Uniform and bootstrap resampling of empirical distributions. In Proceedings of the 1993 Winter Simulation Conference, pages 503–508. IEEE.

    Google Scholar 

  11. Bayarri, M. J., Berger, J. O., Paulo, R., Sacks, J., Cafeo, J. A., Cavendish, J., Lin, C.-H., and Tu, J. (2007). A framework for validation of computer models. Technometrics, 49(2):138–154.

    Google Scholar 

  12. Bayraksan, G. and Love, D. K. (2015). Data-driven stochastic programming using phi-divergences. In Tutorials in Operations Research, pages 1–19. INFORMS.

    Google Scholar 

  13. Bechhofer, R. E. (1954). A single-sample multiple decision procedure for ranking means of normal populations with known variances. The Annals of Mathematical Statistics, 25(1):16 – 39.

    Google Scholar 

  14. Ben-Tal, A., Den Hertog, D., De Waegenaere, A., Melenberg, B., and Rennen, G. (2013). Robust solutions of optimization problems affected by uncertain probabilities. Management Science, 59(2):341–357.

    Google Scholar 

  15. Ben-Tal, A., El Ghaoui, L., and Nemirovski, A. (2009). Robust Optimization. Princeton University Press.

    Google Scholar 

  16. Bertsimas, D., Brown, D. B., and Caramanis, C. (2011). Theory and applications of robust optimization. SIAM Review, 53(3):464–501.

    Google Scholar 

  17. Bertsimas, D., Gupta, V., and Kallus, N. (2018). Robust sample average approximation. Mathematical Programming, 171(1-2):217–282.

    Google Scholar 

  18. Blanchet, J., Kang, Y., and Murthy, K. (2019). Robust Wasserstein profile inference and applications to machine learning. Journal of Applied Probability, 56(3):830-857.

    Google Scholar 

  19. Blanchet, J. and Murthy, K. (2019). Quantifying distributional model risk via optimal transport. Mathematics of Operations Research, 44(2):565–600.

    Google Scholar 

  20. Box, G. E. P. (1976). Science and statistics. Journal of the American Statistical Association, 71(356):791–799.

    Google Scholar 

  21. Cakmak, S., Wu, D., and Zhou, E. (2020). Solving Bayesian risk optimization via nested stochastic gradient estimation. ar**v:2007.05860.

    Google Scholar 

  22. Chen, L., Ma, W., Natarajan, K., Simchi-Levi, D., and Yan, Z. (2018). Distributionally robust linear and discrete optimization with marginals. Available at SSRN 3159473.

    Google Scholar 

  23. Chen, R. and Paschalidis, I. C. (2018). A robust learning approach for regression models based on distributionally robust optimization. Journal of Machine Learning Research, 19(13):1–48.

    Google Scholar 

  24. Cheng, R. C. and Holland, W. (1997). Sensitivity of computer simulation experiments to errors in input data. Journal of Statistical Computation and Simulation, 57(1–4):219–241.

    Google Scholar 

  25. Cheng, R. C. and Holland, W. (1998). Two-point methods for assessing variability in simulation output. Journal of Statistical Computation and Simulation, 60(3):183–205.

    Google Scholar 

  26. Cheng, R. C. H. and Holland, W. (2004). Calculation of confidence intervals for simulation output. ACM Transactions on Modeling and Computer Simulation, 14(4).

    Google Scholar 

  27. Chick, S. E. (2001). Input distribution selection for simulation experiments: Accounting for input uncertainty. Operations Research, 49(5):744–758.

    Google Scholar 

  28. Chick, S. E. (2006). Bayesian ideas and discrete event simulation: Why, what and how. In Perrone, L. F., Wieland, F. P., Liu, J., Lawson, B. G., Nicol, D. M., and Fujimoto, R. M., editors, Proceedings of the 2006 Winter Simulation Conference, pages 96–106. IEEE.

    Google Scholar 

  29. Corlu, C. G., Akcay, A., and **e, W. (2020). Stochastic simulation under input uncertainty: A Review. Operations Research Perspectives, 7:100162.

    Google Scholar 

  30. Corlu, C. and Biller, B. (2013). A subset selection procedure under input parameter uncertainty. In Proceedings of the 2013 Winter Simulation Conference, pages 463–473. IEEE.

    Google Scholar 

  31. Corlu, C. G. and Biller, B. (2015). Subset selection for simulations accounting for input uncertainty. In Proceedings of the 2015 Winter Simulation Conference, pages 437–446. IEEE.

    Google Scholar 

  32. Cranmer, K., Brehmer, J., and Louppe, G. (2020). The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48):30055–30062.

    Google Scholar 

  33. Currin, C., Mitchell, T., Morris, M., and Ylvisaker, D. (1991). Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. Journal of the American Statistical Association, 86(416):953–963.

    Google Scholar 

  34. Davison, A. C. and Hinkley, D. V. (1997). Bootstrap Methods and their Application. Number 1. Cambridge University Press.

    Google Scholar 

  35. Delage, E. and Ye, Y. (2010). Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations Research, 58(3):595–612.

    Google Scholar 

  36. Dhara, A., Das, B., and Natarajan, K. (2021). Worst-case expected shortfall with univariate and bivariate marginals. INFORMS Journal on Computing, 33(1):370–389.

    Google Scholar 

  37. Doan, X. V., Li, X., and Natarajan, K. (2015). Robustness to dependency in portfolio optimization using overlap** marginals. Operations Research, 63(6):1468–1488.

    Google Scholar 

  38. Duchi, J., Glynn, P., and Namkoong, H. (2016). Statistics of robust optimization: A generalized empirical likelihood approach. ar**v preprint ar**v:1610.03425.

  39. Duchi, J. C., Glynn, P. W., and Namkoong, H. (2021). Statistics of robust optimization: A generalized empirical likelihood approach. Mathematics of Operations Research, 46(3):946–969.

    Google Scholar 

  40. Efron, B. and Tibshirani, R. J. (1994). An Introduction to the Bootstrap. CRC press.

    Google Scholar 

  41. Esfahani, P. M. and Kuhn, D. (2018). Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Mathematical Programming, 171(1-2):115–166.

    Google Scholar 

  42. Fan, W., Hong, L. J., and Zhang, X. (2020). Distributionally robust selection of the best. Management Science, 66(1):190–208.

    Google Scholar 

  43. Feng, M. and Song, E. (2019). Efficient input uncertainty quantification via green simulation using sample path likelihood ratios. In Proceedings of the 2019 Winter Simulation Conference, pages 3693–3704. IEEE.

    Google Scholar 

  44. Feng, M. and Staum, J. (2015). Green simulation designs for repeated experiments. In Proceedings of the 2015 Winter Simulation Conference, pages 403–413. IEEE.

    Google Scholar 

  45. Feng, M. and Staum, J. (2017). Green simulation: Reusing the output of repeated experiments. ACM Transactions on Modeling and Computer Simulation, 27(4):1–28.

    Google Scholar 

  46. Feng, M. B. and Song, E. (2021). Optimal nested simulation experiment design via likelihood ratio method. ar**v preprint ar**v:2008.13087v2.

  47. Fox, B. L. and Glynn, P. W. (1989). Replication schemes for limiting expectations. Probability in the Engineering and Informational Sciences, 3(3):299–318.

    Google Scholar 

  48. Gao, R. and Kleywegt, A. J. (2016). Distributionally robust stochastic optimization with Wasserstein distance. ar**v preprintar**v:1604.02199.

  49. Gao, S., **ao, H., Zhou, E., and Chen, W. (2017). Robust ranking and selection with optimal computing budget allocation. Automatica, 81:30–36.

    Google Scholar 

  50. Ghaoui, L. E., Oks, M., and Oustry, F. (2003). Worst-case value-at-risk and robust portfolio optimization: A conic programming approach. Operations Research, 51(4):543–556.

    Google Scholar 

  51. Ghosh, S. and Lam, H. (2015). Mirror descent stochastic approximation for computing worst-case stochastic input models. In Proceedings of the 2015 Winter Simulation Conference, pages 425–436. IEEE.

    Google Scholar 

  52. Ghosh, S. and Lam, H. (2019). Robust analysis in stochastic simulation: Computation and performance guarantees. Operations Research, 67(1):232–249.

    Google Scholar 

  53. Glasserman, P. and Xu, X. (2014). Robust risk measurement and model risk. Quantitative Finance, 14(1):29–58.

    Google Scholar 

  54. Glasserman, P. and Yang, L. (2018). Bounding wrong-way risk in CVA calculation. Mathematical Finance, 28(1):268–305.

    Google Scholar 

  55. Glynn, P. W. (1990). Likelihood ratio gradient estimation for stochastic systems. Communications of the ACM, 33(10):75–84.

    Google Scholar 

  56. Glynn, P. W. and Iglehart, D. L. (1990). Simulation output analysis using standardized time series. Mathematics of Operations Research, 15(1):1–16.

    Google Scholar 

  57. Glynn, P. W. and Lam, H. (2018). Constructing simulation output intervals under input uncertainty via data sectioning. In Proceedings of the 2018 Winter Simulation Conference, pages 1551–1562. IEEE.

    Google Scholar 

  58. Goeva, A., Lam, H., Qian, H., and Zhang, B. (2019). Optimization-based calibration of simulation input models. Operations Research, 67(5):1362–1382.

    Google Scholar 

  59. Goeva, A., Lam, H., and Zhang, B. (2014). Reconstructing input models via simulation optimization. In Proceedings of the 2014 Winter Simulation Conference, pages 698–709. IEEE.

    Google Scholar 

  60. Goh, J. and Sim, M. (2010). Distributionally robust optimization and its tractable approximations. Operations Research, 58(4):902–917.

    Google Scholar 

  61. Hampel, F. R. (1974). The influence curve and its role in robust estimation. Journal of the American Statistical Association, 69(346):383–393.

    Google Scholar 

  62. Hanasusanto, G. A., Roitch, V., Kuhn, D., and Wiesemann, W. (2015). A distributionally robust perspective on uncertainty quantification and chance constrained programming. Mathematical Programming, 151(1):35–62.

    Google Scholar 

  63. Henderson, S. G. (2003). Input modeling: Input model uncertainty: Why do we care and what should we do about it? In Chick, S., Sánchez, P. J., Ferrin, D., and Morrice, D. J., editors, Proceedings of the 2003 Winter Simulation Conference, pages 90–100. IEEE.

    Google Scholar 

  64. Higdon, D., Gattiker, J., Williams, B., and Rightley, M. (2008). Computer model calibration using high-dimensional output. Journal of the American Statistical Association, 103(482):570–583.

    Google Scholar 

  65. Hsu, J. C. (1984). Constrained simultaneous confidence intervals for multiple comparisons with the best. Annals of Statististics, 12(3):1136–1144.

    Google Scholar 

  66. Hu, Z., Cao, J., and Hong, L. J. (2012). Robust simulation of global warming policies using the DICE model. Management Science, 58(12):2190–2206.

    Google Scholar 

  67. Hu, Z. and Hong, L. J. (2015). Robust simulation of stochastic systems with input uncertainties modeled by statistical divergences. In 2015 Winter Simulation Conference, pages 643–654. IEEE.

    Google Scholar 

  68. Jiang, R. and Guan, Y. (2016). Data-driven chance constrained stochastic program. Mathematical Programming, 158(1):291–327.

    Google Scholar 

  69. Jones, D. R., Schonlau, M., and Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492.

    Google Scholar 

  70. Kennedy, M. C. and O’Hagan, A. (2001). Bayesian calibration of computer models. Journal of the Royal Statistical Society. Series B, Statistical Methodology, 63(3):425–464.

    Google Scholar 

  71. Kim, S.-H. and Nelson, B. L. (2001). A fully sequential procedure for indifference-zone selection in simulation. ACM Transactions on Modeling and Computer Simulation, 11(3):251–273.

    Google Scholar 

  72. Kim, S.-H. and Nelson, B. L. (2006). Chapter 17 selecting the best system. In Henderson, S. G. and Nelson, B. L., editors, Simulation, volume 13 of Handbooks in Operations Research and Management Science, pages 501–534. Elsevier.

    Google Scholar 

  73. Kleijnen, J. P. (1995). Verification and validation of simulation models. European Journal of Operational Research, 82(1):145–162.

    Google Scholar 

  74. Lakshmanan, S. and Venkateswaran, J. (2017). Robust simulation based optimization with input uncertainty. In Proceedings of the 2017 Winter Simulation Conference, pages 2257–2267. IEEE.

    Google Scholar 

  75. Lam, H. (2016a). Advanced tutorial: Input uncertainty and robust analysis in stochastic simulation. In Proceedings of the 2016 Winter Simulation Conference, pages 178–192. IEEE.

    Google Scholar 

  76. Lam, H. (2016b). Robust sensitivity analysis for stochastic systems. Mathematics of Operations Research, 41(4):1248–1275.

    Google Scholar 

  77. Lam, H. (2018). Sensitivity to serial dependency of input processes: A robust approach. Management Science, 64(3):1311–1327.

    Google Scholar 

  78. Lam, H. (2019). Recovering best statistical guarantees via the empirical divergence-based distributionally robust optimization. Operations Research, 67(4):1090–1105.

    Google Scholar 

  79. Lam, H. and Mottet, C. (2017). Tail analysis without parametric models: A worst-case perspective. Operations Research, 65(6):1696–1711.

    Google Scholar 

  80. Lam, H. and Qian, H. (2016). The empirical likelihood approach to simulation input uncertainty. In Proceedings of the 2016 Winter Simulation Conference, pages 791-802. IEEE.

    Google Scholar 

  81. Lam, H. and Qian, H. (2017). Optimization-based quantification of simulation input uncertainty via empirical likelihood. ar**v preprintar**v:1707.05917.

  82. Lam, H. and Qian, H. (2018a). Subsampling to enhance efficiency in input uncertainty quantification. Operations Research, published online in Articles in Advance, 03 Dec 2021.

    Google Scholar 

  83. Lam, H. and Qian, H. (2018b). Subsampling variance for input uncertainty quantification. In 2018 Winter Simulation Conference, pages 1611–1622. IEEE.

    Google Scholar 

  84. Lam, H. and Qian, H. (2019). Random perturbation and bagging to quantify input uncertainty. In 2019 Winter Simulation Conference, pages 320–331. IEEE.

    Google Scholar 

  85. Lam, H. and Zhang, J. (2020). Distributionally constrained stochastic gradient estimation using noisy function evaluations. In Proceedings of the 2020 Winter Simulation Conference, pages 445–456. IEEE.

    Google Scholar 

  86. Lam, H., Zhang, X., and Plumlee, M. (2017). Improving prediction from stochastic simulation via model discrepancy learning. In Proceedings of the 2017 Winter Simulation Conference, pages 1808–1819. IEEE.

    Google Scholar 

  87. Lam, H. and Zhou, E. (2017). The empirical likelihood approach to quantifying uncertainty in sample average approximation. Operations Research Letters, 45(4):301–307.

    Google Scholar 

  88. Lewis, P. A. and Orav, E. J. (2017). Simulation Methodology for Statisticians, Operations Analysts, and Engineers. Chapman and Hall/CRC.

    Google Scholar 

  89. Li, B., Jiang, R., and Mathieu, J. L. (2017). Ambiguous risk constraints with moment and unimodality information. Mathematical Programming, 173:151–192.

    Google Scholar 

  90. Miller, B. L. and Wagner, H. M. (1965). Chance constrained programming with joint constraints. Operations Research, 13(6):930–945.

    Google Scholar 

  91. Morgan, L. E., Nelson, B. L., Titman, A. C., and Worthington, D. J. (2019). Detecting bias due to input modelling in computer simulation. European Journal of Operational Research, 279(3):869–881.

    Google Scholar 

  92. Nelson, B. (2013). Foundations and Methods of Stochastic Simulation: A First Course. Springer Science & Business Media.

    Google Scholar 

  93. Ng, S. H. and Chick, S. E. (2006). Reducing parameter uncertainty for stochastic systems. ACM Transactions on Modeling and Computer Simulation, 16(1):26–51.

    Google Scholar 

  94. Oakley, J. E. and O’Hagan, A. (2004). Probabilistic sensitivity analysis of complex models: a Bayesian approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 66(3):751–769.

    Google Scholar 

  95. Oakley, J. E. and Youngman, B. D. (2017). Calibration of stochastic computer simulators using likelihood emulation. Technometrics, 59(1):80–92.

    Google Scholar 

  96. Owen, A. B. (2001). Empirical Likelihood. CRC press.

    Google Scholar 

  97. O’Hagan, A., Kennedy, M. C., and Oakley, J. E. (1999). Uncertainty analysis and other inference tools for complex computer codes. In Bernardo, J., Berger, J., Dawid, A., and Smith, A., editors, Bayesian Statistics 6: Proceedings of the Sixth Valencia International Meeting, pages 503–524. Oxford Science Publications.

    Google Scholar 

  98. Pearce, M. and Branke, J. (2017). Bayesian simulation optimization with input uncertainty. In Proceedings of the 2017 Winter Simulation Conference, pages 2268–2278. IEEE.

    Google Scholar 

  99. Phuong Le, H. and Branke, J. (2020). Bayesian optimization searching for robust solutions. In Proceedings of the 2020 Winter Simulation Conference, pages 2844–2855. IEEE.

    Google Scholar 

  100. Picheny, V. (2015). Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction. Statistics and Computing, 25(6):1265–1280.

    Google Scholar 

  101. Plumlee, M. and Lam, H. (2016). Learning stochastic model discrepancy. In Proceedings of the 2016 Winter Simulation Conference, pages 413–424. IEEE.

    Google Scholar 

  102. Popescu, I. (2005). A semidefinite programming approach to optimal-moment bounds for convex classes of distributions. Mathematics of Operations Research, 30(3):632–657.

    Google Scholar 

  103. Reiman, M. I. and Weiss, A. (1989). Sensitivity analysis for simulations via likelihood ratios. Operations Research, 37(5):830–844.

    Google Scholar 

  104. Rinott, Y. (1978). On two-stage selection procedures and related probability-inequalities. Communications in Statistics - Theory and Methods, 7(8):799–811.

    Google Scholar 

  105. Rubin, D. B. (1981). The Bayesian bootstrap. The Annals of Statistics, 9(1):130–134.

    Google Scholar 

  106. Rubinstein, R. Y. (1986). The score function approach for sensitivity analysis of computer simulation models. Mathematics and Computers in Simulation, 28(5):351–379.

    Google Scholar 

  107. Saltelli, A., Tarantola, S., Campolongo, F., and Ratto, M. (2004). Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. John Wiley & Sons.

    Google Scholar 

  108. Sargent, R. G. (2005). Verification and validation of simulation models. In Proceedings of the 2005 Winter Simulation Conference, pages 130–143. IEEE.

    Google Scholar 

  109. Schmeiser, B. (1982). Batch size effects in the analysis of simulation output. Operations Research, 30(3):556–568.

    Google Scholar 

  110. Schruben, L. (1983). Confidence interval estimation using standardized time series. Operations Research, 31(6):1090–1108.

    Google Scholar 

  111. Schruben, L. and Kulkarni, R. (1982). Some consequences of estimating parameters for the M/M/1 queue. Operations Research Letters, 1(2):75–78.

    Google Scholar 

  112. Schruben, L. W. (1980). Establishing the credibility of simulations. Simulation, 34(3):101–105.

    Google Scholar 

  113. Scott, W., Frazier, P., and Powell, W. (2011). The correlated knowledge gradient for simulation optimization of continuous parameters using Gaussian process regression. SIAM Journal on Optimization, 21(3):996–1026.

    Google Scholar 

  114. Shafer, G. (1976). Statistical evidence. In A Mathematical Theory of Evidence, pages 237–273. Princeton University Press.

    Google Scholar 

  115. Shi, Z., Gao, S., **ao, H., and Chen, W. (2019). A worst-case formulation for constrained ranking and selection with input uncertainty. Naval Research Logistics, 66(8):648–662.

    Google Scholar 

  116. Song, E. (2021). Sequential bayesian risk set inference for robust discrete optimization via simulation. ar**v preprintar**v:2101.07466.

  117. Song, E. and Nelson, B. L. (2015). Quickly assessing contributions to input uncertainty. IIE Transactions, 47(9):893–909.

    Google Scholar 

  118. Song, E. and Nelson, B. L. (2019). Input–output uncertainty comparisons for discrete optimization via simulation. Operations Research, 67(2):562–576.

    Google Scholar 

  119. Song, E., Nelson, B. L., and Hong, L. J. (2015). Input uncertainty and indifference-zone ranking & selection. In Proceedings of the 2015 Winter Simulation Conference, pages 414–424. IEEE.

    Google Scholar 

  120. Song, E., Nelson, B. L., and Pegden, C. D. (2014). Advanced tutorial: Input uncertainty quantification. In Proceedings of the 2014 Winter Simulation Conference, pages 162–176. IEEE.

    Google Scholar 

  121. Song, E., Nelson, B. L., and Staum, J. (2016). Shapley effects for global sensitivity analysis: Theory and computation. SIAM/ASA Journal on Uncertainty Quantification, 4(1):1060–1083.

    Google Scholar 

  122. Song, E. and Shanbhag, U. V. (2019). Stochastic approximation for simulation optimization under input uncertainty with streaming data. In Proceedings of the 2019 Winter Simulation Conference, pages 3597–3608. IEEE.

    Google Scholar 

  123. Sun, Y., Apley, D. W., and Staum, J. (2011). Efficient nested simulation for estimating the variance of a conditional expectation. Operations Research, 59(4):998–1007.

    Google Scholar 

  124. Tarantola, A. (2005). Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM.

    Google Scholar 

  125. Tuo, R., Wu, C. J., et al. (2015). Efficient calibration for imperfect computer models. The Annals of Statistics, 43(6):2331–2352.

    Google Scholar 

  126. Ungredda, J., Pearce, M., and Branke, J. (2020). Bayesian optimisation vs. input uncertainty reduction. ar**v:2006.00643.

    Google Scholar 

  127. Van Parys, B. P., Goulart, P. J., and Kuhn, D. (2016). Generalized Gauss inequalities via semidefinite programming. Mathematical Programming, 156(1-2):271–302.

    Google Scholar 

  128. Van der Vaart, A. W. (2000). Asymptotic Statistics, volume 3. Cambridge University Press.

    Google Scholar 

  129. Villemonteix, J., Vazquez, E., and Walter, E. (2008). An informational approach to the global optimization of expensive-to-evaluate functions. Journal of Global Optimization, 44(4):509.

    Google Scholar 

  130. Wang, H., Ng, S. H., and Zhang, X. (2020a). A Gaussian process based algorithm for stochastic simulation optimization with input distribution uncertainty. In Proceedings of the 2020 Winter Simulation Conference, pages 2899–2910. IEEE.

    Google Scholar 

  131. Wang, H., Yuan, J., and Ng, S. H. (2020b). Gaussian process based optimization algorithms with input uncertainty. IISE Transactions, 52(4):377–393.

    Google Scholar 

  132. Wang, H., Zhang, X., and Ng, S. H. (2021). A nonparametric Bayesian approach for simulation optimization with input uncertainty. ar**v:2008.02154.

    Google Scholar 

  133. Wiesemann, W., Kuhn, D., and Sim, M. (2014). Distributionally robust convex optimization. Operations Research, 62(6):1358–1376.

    Google Scholar 

  134. Wong, R. K. W., Storlie, C. B., and Lee, T. C. M. (2017). A frequentist approach to computer model calibration. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(2):635–648.

    Google Scholar 

  135. Wu, D. and Zhou, E. (2017). Ranking and selection under input uncertainty: fixed confidence and fixed budget. ar**v preprintar**v:1708.08526.

  136. **e, W., Li, C., Wu, Y., and Zhang, P. (2021). A Bayesian nonparametric framework for uncertainty quantification in simulation. SIAM Journal on Uncertainty Quantification, 9(4):1527–1552.

    Google Scholar 

  137. **e, W., Nelson, B. L., and Barton, R. R. (2014). A Bayesian framework for quantifying uncertainty in stochastic simulation. Operations Research, 62(6):1439–1452.

    Google Scholar 

  138. **e, W., Nelson, B. L., and Barton, R. R. (2016). Multivariate input uncertainty in output analysis for stochastic simulation. ACM Transactions on Modeling and Computer Simulation, 27(1):5:1–5:22.

    Google Scholar 

  139. Xu, J., Zheng, Z., and Glynn, P. W. (2020). Joint resource allocation for input data collection and simulation. In Proceedings of the 2020 Winter Simulation Conference, pages 2126–2137. IEEE.

    Google Scholar 

  140. Zazanis, M. A. and Suri, R. (1993). Convergence rates of finite-difference sensitivity estimates for stochastic systems. Operations Research, 41(4):694–703.

    Google Scholar 

  141. Zhou, E. and Liu, T. (2018). Online quantification of input uncertainty for parametric models. In Proceedings of the 2018 Winter Simulation Conference, pages 1587–1598. IEEE.

    Google Scholar 

  142. Zhou, E. and **e, W. (2015). Simulation optimization when facing input uncertainty. In Proceedings of the 2015 Winter Simulation Conference, pages 3714–3724. IEEE.

    Google Scholar 

  143. Zouaoui, F. and Wilson, J. R. (2003). Accounting for parameter uncertainty in simulation input modeling. IIE Transactions, 35(9):781–792.

    Google Scholar 

  144. Zouaoui, F. and Wilson, J. R. (2004). Accounting for input-model and input-parameter uncertainties in simulation. IIE Transactions, 36(11):1135–1151.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Penn State University, University Park, PA, USA

    Russell R. Barton & Eunhye Song

  2. Columbia University, New York, NY, USA

    Henry Lam

Authors
  1. Russell R. Barton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Henry Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eunhye Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Russell R. Barton .

Editor information

Editors and Affiliations

  1. Kent Business School, University of Kent, Canterbury, UK

    Saïd Salhi

  2. Lancaster University, Lancaster, UK

    John Boylan

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Barton, R.R., Lam, H., Song, E. (2022). Input Uncertainty in Stochastic Simulation. In: Salhi, S., Boylan, J. (eds) The Palgrave Handbook of Operations Research . Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-96935-6_17

Download citation

Publish with us

Policies and ethics

Search

Navigation