Abstract
Stress testing in simulation plays a critical role in the validation of safety-critical systems, including aircraft, cars, medical devices, and spacecraft. The analysis of failure events is important in understanding the causes and conditions of failure, informing improvements to the system, and the estimation and categorization of risk. However, stress testing of safety-critical systems can be very challenging. Finding failure events can be difficult due to the size and complexity of the system, interactions with an environment over many time steps, and rarity of failure events. While Monte Carlo sampling is frequently used in practice, it can be very inefficient when the algorithm is undirected. We present adaptive stress testing (AST), an accelerated stress testing method for finding the most likely path to a failure event. Adaptive stress testing formulates stress testing as a sequential decision process and then uses reinforcement learning to optimize it. By using learning during search, the algorithm can automatically discover important parts of the state space and adaptively focus the search. We apply adaptive stress testing to stress test a prototype of next-generation aircraft collision avoidance system in simulated encounters, where we find and analyze the most likely paths to near mid-air collision.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
C.B. Browne, E. Powley, D. Whitehouse, S.M. Lucas, P.I. Cowling, P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, S. Colton, A survey of Monte Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4(1), 1–43 (2012)
B.J. Chludzinski, Evaluation of TCAS II version 7.1 using the FAA fast-time encounter generator model. Project Report ATC-346, Massachusetts Institute of Technology, Lincoln Laboratory (2009)
A. Couëtoux, J.B. Hoock, N. Sokolovska, O. Teytaud, N. Bonnard, Continuous upper confidence trees, in Learning and Intelligent Optimization (LION) (2011), pp 433–445
R.W. Gardner, D. Genin, R. McDowell, C. Rouff, A. Saksena, A. Schmidt, Probabilistic model checking of the next-generation airborne collision avoidance system, in Digital Avionics Systems Conference (DASC) (2016)
J.E. Holland, M.J. Kochenderfer, W.A. Olson, Optimizing the next generation collision avoidance system for safe, suitable, and acceptable operational performance. Air Traffic Control Q. 21(3), 275–297 (2013)
International Civil Aviation Organization, Surveillance, radar and collision avoidance, in International Standards and Recommended Practices, vol IV, annex 10, 4th edn (2007)
J.B. Jeannin, K. Ghorbal, Y. Kouskoulas, R. Gardner, A. Schmidt, E. Zawadzki, A. Platzer, A formally verified hybrid system for the next-generation airborne collision avoidance system, in International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS) (2015)
M.J. Kochenderfer, Decision making under uncertainty: theory and application. MIT Press (2015)
M.J. Kochenderfer, J.P. Chryssanthacopoulos, A decision-theoretic approach to develo** robust collision avoidance logic, in IEEE International Conference on Intelligent Transportation Systems (ITSC) (2010), pp 1837–1842
M.J. Kochenderfer, L.P. Espindle, J.K. Kuchar, J.D. Griffith, Correlated encounter model for cooperative aircraft in the national airspace system. Project Report ATC-344, Massachusetts Institute of Technology, Lincoln Laboratory (2008)
M.J. Kochenderfer, J.E. Holland, J.P. Chryssanthacopoulos, Next-generation airborne collision avoidance system. Lincoln Lab. J. 19(1), 17–33 (2012)
L. Kocsis, C. Szepesvári, Bandit based Monte-Carlo planning, in European Conference on Machine Learning (ECML) (2006), pp 282–293
Y. Kouskoulas, D. Genin, A. Schmidt, J. Jeannin, Formally verified safe vertical maneuvers for non-deterministic, accelerating aircraft dynamics, in 8th International Conference on Interactive Theorem Proving (2017), pp 336–353
J.K. Kuchar, A.C. Drumm, The traffic alert and collision avoidance system. Lincoln Lab. J. 16(2), 277–296 (2007)
R. Lee, M.J. Kochenderfer, O.J. Mengshoel, G.P. Brat, M.P. Owen, Adaptive stress testing of airborne collision avoidance systems, in Digital Avionics Systems Conference (DASC) (2015)
R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 1998)
C. von Essen, D. Giannakopoulou, Probabilistic verification and synthesis of the next generation airborne collision avoidance system. Int. J. Softw. Tools Technol. Transfer 18(2), 227–243 (2016)
C.J.C.H. Watkins, P. Dayan, Technical note: Q-learning. Mach. Learn. 8, 279–292 (1992)
Acknowledgements
We thank Neal Suchy at the Federal Aviation Administration (FAA); Michael Owen, Robert Klaus, and Cindy McLain at MIT Lincoln Laboratory; Joshua Silbermann, Anshu Saksena, Ryan Gardner, and Rachel Szczesiul at Johns Hopkins Applied Physics Laboratory; and others in the ACAS X team. We thank Guillaume Brat at NASA and Corina Pasareanu at Carnegie Mellon University for their invaluable feedback. This work was supported by the Safe and Autonomous Systems Operations (SASO) Project under NASA Aeronautics Research Mission Directorate (ARMD) Airspace Operations and Safety Program (AOSP).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Lee, R., Mengshoel, O.J., Kochenderfer, M.J. (2019). Adaptive Stress Testing of Safety-Critical Systems. In: Yu, H., Li, X., Murray, R., Ramesh, S., Tomlin, C. (eds) Safe, Autonomous and Intelligent Vehicles. Unmanned System Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-97301-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-97301-2_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97300-5
Online ISBN: 978-3-319-97301-2
eBook Packages: EngineeringEngineering (R0)