SpringerLink
Log in

Uncertainty Modelling in Performability Prediction for Safety-Critical Systems

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

A Correction to this article was published on 17 April 2024

This article has been updated

Abstract

Failure of safety-critical systems (SCSs) devastates human life and the environment and involves huge costs. So, quality assurance is essential for the SCSs. Performability is a combined study of performance and reliability. The performability parameters should be precise for the critical systems since each model has some limitations that cause uncertainty. This paper proposes a model for uncertainty Prediction and optimization for the performability of the SCSs. A wireless sensor network fire alarm system is taken as a case study to illustrate the concepts and demonstrate their applicability. The performability of the fire alarm system is modelled using the hyper-exponential distribution, and synthetic data are generated using this distribution. The output uncertainty is calibrated using the probabilistic neural network (PNN). Failure of safety-critical systems (SCSs) has catastrophic effects on both the environment and human life and is extremely expensive. Quality control is, therefore, crucial for SCSs. The study of performability combines dependability with performance. Given that each model has some limits that lead to uncertainty, the performability parameters should be accurate for the key systems. This article suggests a model for uncertainty predictions and performance improvement for the SCSs. A wireless sensor network fire alarm system is used as a case study to clarify the ideas and show how they apply. The hyper-exponential distribution is used to model the fire alarm system’s performance, and it is also used to create synthetic data. The probabilistic neural network (PNN) is used to calibrate the output uncertainty.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Change history

References

  1. Mourelatos, Z.P.; Liang, J.: A methodology for trading-off performance and robustness under uncertainty. J. Mech. Des. Trans. ASME 128, 856–863 (2006). https://doi.org/10.1115/1.2202883

    Article  Google Scholar 

  2. Zio, E.: Challenges in the vulnerability and risk analysis of critical infrastructures. Reliab. Eng. Syst. Saf. 152, 137–150 (2016). https://doi.org/10.1016/j.ress.2016.02.009

    Article  Google Scholar 

  3. Marie, R.A.: From perfromability to uncertainty. In: PA-Offload: performability-aware adaptive fog offloading for drone image processing, pp. 66–73. IEEE (2021)

  4. Vittal, S.; Dubi, A.: Uncertainty assessment in system performability analysis. In: Collection of Technical Papers—AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics and materials conference, vol. 8, pp. 5234–5243 (2005). https://doi.org/10.2514/6.2005-2220

  5. De Angelis, M.; Patelli, E.; Beer, M.: Uncertainty management of safety-critical systems: A solution to the back-propagation problem. In: 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP 2015. pp. 9–13 (2015)

  6. Dorn, J.; Apel, S.; Siegmund, N.: Mastering uncertainty in performance estimations of configurable software systems. In: Proceedings—2020 35th IEEE/ACM International Conference on Automated Software Engineering, ASE 2020. pp. 684–696 (2020). https://doi.org/10.1145/3324884.3416620

  7. Lin, L.; Bao, H.; Dinh, N.: Annals of nuclear energy uncertainty quantification and software risk analysis for digital twins in the nearly autonomous management and control systems: a review. Ann. Nucl. Energy 160, 108362 (2021). https://doi.org/10.1016/j.anucene.2021.108362

    Article  Google Scholar 

  8. Rottmann, M.; Reese, M.: Automated detection of label errors in semantic segmentation datasets via deep learning and uncertainty quantification, (2023)

  9. Hu, X.; Fang, G.; Yang, J.; Zhao, L.; Ge, Y.: Simplified models for uncertainty quantification of extreme events using Monte Carlo technique. Reliab. Eng. Syst. Saf. 230, 108935 (2023). https://doi.org/10.1016/J.RESS.2022.108935

    Article  Google Scholar 

  10. Jahmunah, V.; Ng, E.Y.K.; Tan, R.S.; Oh, S.L.; Acharya, U.R.: Uncertainty quantification in DenseNet model using myocardial infarction ECG signals. Comput. Methods Progr. Biomed. 229, 107308 (2023). https://doi.org/10.1016/J.CMPB.2022.107308

    Article  Google Scholar 

  11. Zhang, J.; Yin, J.; Wang, R.: Basic framework and main methods of uncertainty quantification. Math. Probl. Eng. (2020). https://doi.org/10.1155/2020/6068203

    Article  Google Scholar 

  12. Mahadevan, S.; Sarkar, S.: Uncertainty analysis methods. In: Issues in quantitative and qualitative risk modeling with application\rto import risk assessment ACERA project (0705). pp. 1–26 (2009)

  13. Mittal, H.: Uncertainty analysis of software reliability: architecture-based approach. Int. J. Softw. Eng. Res. Pract. 5, 6–11 (2015)

    Google Scholar 

  14. Kamavaram, S.K.; Virginia, W.: Uncertainty analysis in software reliability of component-based systems Uncertainty Analysis in Software Reliability of Component-Based Systems Sunil Kumar Kamavaram Department of Computer Science and Electrical Engineering. In: Uncertainty analysis in software reliability of component-based systems. West Virginia University Follow (2004)

  15. Yin, L.; Smith, M.A.J.; Trivedi, K.S.: Uncertainty analysis in reliability modeling. In: Proceedings of the Annual Reliability and Maintainability Symposium. pp. 229–234 (2001). https://doi.org/10.1109/rams.2001.902472

  16. Meidani, H.; Ghanem, R.: Uncertainty quantification for Markov chain models. Chaos Interdiscipl. J. Nonlinear. Sci. (2012). https://doi.org/10.1063/1.4757645

    Article  Google Scholar 

  17. Haverkort, B.R.; Ieee, M.: Sensitivity & uncertainty analysis of markov-reward models. IEEE Trans. Nucl. Sci. 44, 147–154 (1995)

    Google Scholar 

  18. Stavroulakis, G.; Giovanis, D.G.; Papadrakakis, M.; Papadopoulos, V.: A new perspective on the solution of uncertainty quantification and reliability analysis of large-scale problems. Comput. Methods Appl. Mech. Eng. 276, 627–658 (2014). https://doi.org/10.1016/j.cma.2014.03.009

    Article  MathSciNet  Google Scholar 

  19. Perez-Palacin, Diego; Mirandola, R.: Deling with the Uncertainity in the performance modelling of Software Systems. In: 10th international ACM Sigsoft conference on Quality of software architectures. pp. 33–42 (2014)

  20. Cheung, L.; Golubchik, L.; Medvidovic, N., Sukhatme, G.: Identifying and addressing uncertainty in architecture-level software reliability modeling. In: Proceedings—21st International Parallel and Distributed Processing Symposium, IPDPS 2007; Abstracts and CD-ROM. (2007). https://doi.org/10.1109/IPDPS.2007.370524

  21. Perez-Palacin, D.; Mirandola, R.: Uncertainties in the modeling of self-adaptive systems: A taxonomy and an example of availability evaluation. In: ICPE 2014—Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering. pp. 3–14 (2014). https://doi.org/10.1145/2568088.2568095

  22. Goševa-Popstojanova, K.; Kamavaram, S.: Assessing uncertainty in reliability of component-based software systems. In: Proceedings—International Symposium on Software Reliability Engineering, ISSRE. 2003-Janua, pp. 307–320 (2003). https://doi.org/10.1109/ISSRE.2003.1251052

  23. Dai, Y.S.; **e, M.; Long, Q.; Ng, S.H.: Uncertainty analysis in software reliability modeling by Bayesian approach with maximum-entropy principle. IEEE Trans. Softw. Eng. 33, 781–795 (2007). https://doi.org/10.1109/TSE.2007.70739

    Article  Google Scholar 

  24. Farrance, I.; Frenkel, R.: Uncertainty of measurement: a review of the rules for calculating uncertainty components through functional relationships. Clin. Biochem. Rev. 33, 49–75 (2012)

    Google Scholar 

  25. Atamturktur, S.: Verification and validation under uncertainty applied to Finite Element models of historic masonry monuments. In: Conference Proceedings of the Society for Experimental Mechanics Series. (2009)

  26. Mishra, K.; Trivedi, K.S.: Uncertainty propagation through software dependability models. In: Proceedings—International Symposium on Software Reliability Engineering, ISSRE. pp. 80–89 (2011)

  27. Zhang, J.: Modern Monte Carlo methods for efficient uncertainty quantification and propagation: a survey. Wiley Interdiscip. Rev. Comput. Stat. 13(5), e1539 (2021)

    Article  MathSciNet  Google Scholar 

  28. Durga Rao, K.; Kushwaha, H.S.; Verma, A.K.; Srividya, A.: Quantification of epistemic and aleatory uncertainties in level-1 probabilistic safety assessment studies. Reliab. Eng. Syst. Saf. 92, 947–956 (2007). https://doi.org/10.1016/j.ress.2006.07.002

    Article  Google Scholar 

  29. Souza, R.R.; Dorn, A.; Piringer, B.; Wandl-Vogt, E.: Towards a taxonomy of uncertainties: analysing sources of spatio-temporal uncertainty on the example of non-standard German corpora. Informatics (2019). https://doi.org/10.3390/informatics6030034

    Article  Google Scholar 

  30. Devaraj, A.; Mishra, K.; Trivedi, K.S.: Uncertainty propagation in analytic availability models. In: Proceedings of the IEEE Symposium on Reliable Distributed Systems. pp. 121–130 (2010)

  31. Marinho, M.; Sampaio, S.; Lima, T.; de Moura, H.: A guide to deal with uncertainties in software project management. Int. J. Comput. Sci. Inf. Technol. 6, 1–20 (2014). https://doi.org/10.5121/ijcsit.2014.6501

    Article  Google Scholar 

  32. Yanev, G.P.: Exponential and hypoexponential distributions: Some characterizations. Mathematics 8, 1–10 (2020). https://doi.org/10.3390/math8122207

    Article  Google Scholar 

  33. Singh, L.N.; Dattatreya, G.R.: Estimation of the hyperexponential density with applications in sensor networks. Int. J. Distrib. Sens. Netw. 3, 311–330 (2007). https://doi.org/10.1080/15501320701259925

    Article  Google Scholar 

  34. Riihonen, T.; Werner, S.; Wichman, R.: Hypoexponential power-delay profile and performance of multihop OFDM relay links. IEEE Trans. Wirel. Commun. 9, 3878–3888 (2010). https://doi.org/10.1109/TWC.2010.101310.100250

    Article  Google Scholar 

  35. Salah, K.; El, S.: Performance modeling and analysis of hypoexponential network servers. Telecommun. Syst. 65, 717–728 (2017). https://doi.org/10.1007/s11235-016-0262-3

    Article  Google Scholar 

  36. Kim, J.; Lin, X.; Shroff, N.B.; Sinha, P.: Minimizing delay and maximizing lifetime for wireless sensor networks with anycast. IEEE/ACM Trans. Netw. 18, 515–528 (2010). https://doi.org/10.1109/TNET.2009.2032294

    Article  Google Scholar 

  37. GLen, S.: Hyperexponential distribution, https://www.statisticshowto.com/hyperexponential-distribution/, (2021)

  38. Wang, J.; Dong, W.; Cao, Z.; Liu, Y.: On the delay performance in a large-scale wireless sensor network: measurement, analysis, and implications. IEEE/ACM Trans. Netw. 23, 186–197 (2015). https://doi.org/10.1109/TNET.2013.2296331

    Article  Google Scholar 

  39. Yao, W.; Chen, X.; Huang, Y.; Gurdal, Z.; van Tooren, M.: Sequential optimization and mixed uncertainty analysis method for reliability-based optimization. AIAA J. 51, 2266–2277 (2013). https://doi.org/10.2514/1.J052327

    Article  Google Scholar 

  40. Cerutti, F.; Kaplan, L.M.; Kimmig, A.; Şensoy, M.: Handling epistemic and aleatory uncertainties in probabilistic circuits. Springer, US (2022)

    Book  Google Scholar 

  41. Havcrkort, B.R.; Meeuwissen, A.M.H.: Sensitivity and uncertainty analysis in performability modelling. In: Proceedings of the IEEE Symposium on Reliable Distributed Systems. pp. 93–102 (1992)

  42. Park, J.H.; Lee, S.; Yun, S.; Kim, H.; Kim, W.T.: Dependable fire detection system with multifunctional artificial intelligence framework. Sensors (Switzerland) (2019). https://doi.org/10.3390/s19092025

    Article  Google Scholar 

  43. Luis, J.A.; Galán, J.A.G.; Espigado, J.A.: Low power wireless smoke alarm system in home fires. Sensors (Switzerland) 15, 20717–20729 (2015). https://doi.org/10.3390/s150820717

    Article  Google Scholar 

  44. Chien, H.-Y.; Chen, J.; Chen, Y.; Lin, P.; Chang, Y.; Chen, R.: Advances in intelligent information hiding and multimedia signal processing. Springer, Berlin (2018)

    Google Scholar 

  45. Abdar, M.; Pourpanah, F.; Hussain, S.; Rezazadegan, D.; Liu, L.; Ghavamzadeh, M.; Fieguth, P.; Cao, X.; Khosravi, A.; Acharya, U.R.; Makarenkov, V.; Nahavandi, S.: A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Inf. Fusion. 76, 243–297 (2021). https://doi.org/10.1016/j.inffus.2021.05.008

    Article  Google Scholar 

  46. Zhang, W.; Yang, X.; Deng, Y.; Li, A.: An inspired machine-learning algorithm with a hybrid whale optimization for power transformer PHM. Energies (Basel) 13, 1–17 (2020). https://doi.org/10.3390/en13123143

    Article  Google Scholar 

  47. Chung, Y.; Char, I.; Guo, H.; Schneider, J.; Neiswanger, W.: uncertainty toolbox: an open-source library for assessing, visualizing, and improving uncertainty quantification, http://arxiv.org/abs/2109.10254, (2021)

  48. Uncertainty toolbox, https://github.com/uncertainty-toolbox

  49. Kuleshov, V.; Fenner, N.; Ermon, S.: Accurate uncertainties for deep learning using calibrated regression. In: 35th International Conference on Machine Learning, ICML 2018. 6, pp. 4369–4377 (2018). https://doi.org/10.48550/arxiv.1807.00263

  50. Su, S.; Li, Y.; He, S.; Han, S.; Feng, C.; Ding, C.; Miao, F.: Uncertainty quantification of collaborative detection for self-driving. In: Proceedings of IEEE International Conference on Robotics and Automation. pp. 5588–5594 (2023). https://doi.org/10.1109/ICRA48891.2023.10160367

  51. Cheng, S.; Quilodran-Casas, C.; Ouala, S.; Farchi, A.; Liu, C.; Tandeo, P.; Fablet, R.; Lucor, D.; Iooss, B.; Brajard, J.; **ao, D.; Janjic, T.; Ding, W.; Guo, Y.; Carrassi, A.; Bocquet, M.; Arcucci, R.: Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review. IEEE/CAA J. Autom. Sinica 10, 1361–1387 (2023). https://doi.org/10.1109/JAS.2023.123537

    Article  Google Scholar 

  52. Nemani, V.; Biggio, L.; Huan, X.; Hu, Z.; Fink, O.; Tran, A.; Wang, Y.; Zhang, X.; Hu, C.: Uncertainty quantification in machine learning for engineering design and health prognostics: a tutorial. Mech. Syst. Signal Process. 205, 110796 (2023). https://doi.org/10.1016/J.YMSSP.2023.110796

    Article  Google Scholar 

  53. Seoni, S.; Jahmunah, V.; Salvi, M.; Barua, P.D.; Molinari, F.; Acharya, U.R.: Application of uncertainty quantification to artificial intelligence in healthcare: a review of last decade (2013–2023). Comput. Biol. Med. 165, 107441 (2023). https://doi.org/10.1016/J.COMPBIOMED.2023.107441

    Article  Google Scholar 

  54. Abdar, M.; Pourpanah, F.; Hussain, S.; Rezazadegan, D.; Liu, L.; Ghavamzadeh, M.; Fieguth, P.; Cao, X.; Khosravi, A.; Acharya, U.R.; Makarenkov, V.; Nahavandi, S.: A review of uncertainty quantification in deep learning: techniques, applications and challenges. Inf. Fusion 76, 243 (2021)

    Article  Google Scholar 

  55. Goel, S.; Ahamad, S., Ratneshwer: Predicting service availability in SOA-based systems: service dependency tree (SDT). In: 2021 IEEE 2nd International Conference on Technology, Engineering, Management for Societal Impact using Marketing, Entrepreneurship and Talent, TEMSMET 2021. (2021). https://doi.org/10.1109/TEMSMET53515.2021.9768694

  56. Sharma, V.S.; Trivedi, K.S.: Architecture based analysis of performance, reliability and security of software systems. In: Proceedings of the Fifth International Workshop on Software and Performance, WOSP’05. pp. 217–227 (2005)

Download references

Author information

Authors and Affiliations

  1. Department of Data Science, Noida Institute of Engineering and Technology, Greater Noida, Uttar Pradesh, India

    Shakeel Ahamad

  2. Software Quality Assurance Lab, School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi, 110067, India

    Ratneshwer Gupta

Authors
  1. Shakeel Ahamad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ratneshwer Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shakeel Ahamad.

Additional information

The original version of this article was revised: the correct author name of last author was provided.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahamad, S., Gupta, R. Uncertainty Modelling in Performability Prediction for Safety-Critical Systems. Arab J Sci Eng (2024). https://doi.org/10.1007/s13369-024-08891-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13369-024-08891-0

Keywords

Search

Navigation