Abstract
Validation of models used in safety-critical applications requires an extensive validation protocol to build confidence in their predictive capability. A recent predictive validation methodology, introduced by the author and collaborators, is applied in this study to validate the predictions of a Gaussian puff dispersion model. The methodology is based on cross-validation principles in the context of predicting unobserved or difficult to observe quantities of interest. The challenge in this context is that predictions of quantities of interest have no associated observations that they can be compared to. Thus, assessing how close predictions are from observations using validation metrics such as root-mean-square error is not feasible. The study addresses the issue of partitioning the data into a calibration and a validation set, and defining relevant validation metrics based on sensitivity analysis that can support the ultimate goal of predictive modeling. While the framework is general and can be applied to a wide range of problems, in this paper it is used to find an optimal partition of chemical concentration sensors to assess the validity of Gaussian puff predictions in a region of interest.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Abbreviations
- \(p(\cdot )\) :
-
Probability density function (pdf)
- \(p(\cdot \vert \cdot )\) :
-
Conditional pdf
- F(t), G(t) :
-
Cumulative distribution function (cdf)
- d(F(t), G(t)) :
-
Distance between two cdfs
- θ :
-
Model parameters
- D c , D v :
-
Calibration and validation data
- M D , M Q :
-
Data and quantity of interest (QoI) metric
- M D ∗, M Q ∗ :
-
Data and QoI thresholds
- s k , s ∗ :
-
kth partitioning of the data and optimal partition
- \(\mathcal{N}(\mu,\sigma ^{2})\) :
-
Gaussian density function with mean μ and variance σ 2
- \(\log \mathcal{N}(\mu,\sigma ^{2})\) :
-
Log-normal density function with mean μ and variance σ 2
References
AIAA Computational Fluid Dynamics Committee on Standards (1998) AIAA guide for verification and validation of computational fluid dynamics simulations. AIAA, G-077-1998
Alqallaf F, Gustafson P (2001) On cross-validation of bayesian models. Can J Stat 29(2):333–340
Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Stat Surv 4:40–79
ASME Committee V&V 10 (2006) Standard for verification and validation in computational solid mechanics. ASME
ASME Committee V&V 20 (2009) Standard for verification and validation in computational fluid dynamics and heat transfer. ASME
 Babuška I, Nobile F, Tempone R (2008) A systematic approach to model validation based on Bayesian updates and prediction related rejection criteria. Comput Methods Appl Mech Eng 197:2517–2539
Babuška I, Nobile F, Tempone R (2008) A systematic approach to model validation based on bayesian updates and prediction related rejection criteria. Comput Methods Appl Mech Eng 197(29–32):2517–2539 [validation challenge workshop]
Balci O (1997) Verification validation and accreditation of simulation models. In: Proceedings of the 29th winter simulation conference, WSC ’97. IEEE Computer Society, Washington, pp 135–141
Ferson S, Oberkampf WL, Ginzburg L (2008) Model validation and predictive capability for the thermal challenge problem. Comput Methods Appl Mech Eng 197(29–32):2408–2430
Howson C, Urbach P (1989) Scientific reasoning: the bayesian approach. Open Court Publishing Company, Chicago
Jaynes ET (2003) Probability theory: the logic of science. Cambridge University Press, Cambridge
Morrison RE, Bryant CM, Terejanu G, Prudhomme S, Miki K (2013) Data partition methodology for validation of predictive models. Comput Math Appl 66(10):2114–2125
Oberkampf W, Roy C (2010) Verification and validation in scientific computing. Cambridge University Press, Cambridge
Oberkampf WL, Trucano TG (2007) Verification and validation benchmarks. Technical Report SAND2007-0853, Sandia National Laboratories, Unlimited Release
Roache PJ (2008) Perspective: validation-what does it mean? ASME J Fluids Eng 131(3):1–3
Roache PJ (2009) Fundamentals of verification and validation. Hermosa Publishers, Albuquerque
Sargent RG (1998) Verification and validation of simulation models. In: Proceedings of the 30th winter simulation conference, WSC ’98. IEEE Computer Society Press, Los Alamitos, pp 121–130
Stockie J (2011) The mathematics of atmospheric dispersion modeling. SIAM Rev 53(2):349–372
Terejanu G, Upadhyay RR, Miki K (2012) Bayesian experimental design for the active nitridation of graphite by atomic nitrogen. Exp Therm Fluid Sci 36:178–193
Vehtari A, Lampien J (2002) Bayesian model assessment and comparison using cross-validation predictive densities. Neural Comput 14:2439–2468
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 The Society for Experimental Mechanics, Inc.
About this paper
Cite this paper
Terejanu, G. (2015). Predictive Validation of Dispersion Models Using a Data Partitioning Methodology. In: Atamturktur, H., Moaveni, B., Papadimitriou, C., Schoenherr, T. (eds) Model Validation and Uncertainty Quantification, Volume 3. Conference Proceedings of the Society for Experimental Mechanics Series. Springer, Cham. https://doi.org/10.1007/978-3-319-15224-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-15224-0_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15223-3
Online ISBN: 978-3-319-15224-0
eBook Packages: EngineeringEngineering (R0)