SpringerLink
Log in

Revealing User Confidence in Machine Learning-Based Decision Making

  • Chapter
  • First Online:
Human and Machine Learning

Part of the book series: Human–Computer Interaction Series ((HCIS))

  • 4421 Accesses

Abstract

This chapter demonstrates the link between human cognition states and Machine Learning (ML) with a multimodal interface. A framework of informed decision making called DecisionMind is proposed to show how human’s behaviour and physiological signals are used to reveal human cognition states in ML-based decision making. The chapter takes the revealing of user confidence in ML-based decision making as an example to demonstrate the effectiveness of the proposed approach. Based on the revealing of human cognition states during ML-based decision making, the chapter presents a concept of adaptive measurable decision making to show how the revealing of human cognition states are integrated into ML-based decision making to make ML transparent. On the one hand, human cognition states could help understand to what degree humans accept innovative technologies. On the other hand, through understanding human cognition states during ML-based decision making, ML-based decision attributes/factors and even ML models can be adaptively refined in order to make ML transparent.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 43.99
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 54.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info
Hardcover Book
GBP 89.99
Price includes VAT (United Kingdom)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aimone, J.A., Houser, D., Weber, B.: Neural signatures of betrayal aversion: an fMRI study of trust. Proc. Biol. Sci. / Royal Soc. 281(1782), 20132127 (2014)

    Article  Google Scholar 

  2. Arshad, S., Wang, Y., Chen, F.: Interactive mouse stream as real-time indicator of user’s cognitive load. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1025–1030 (2015)

    Google Scholar 

  3. Azizi, H., Ajirlu, S.F.: Measurement of overall performances of decision-making units using ideal and anti-ideal decision-making units. Comput. Ind. Eng. 59(3), 411–418 (2010)

    Article  Google Scholar 

  4. Beller, J., Heesen, M., Vollrath, M.: Improving the drivercautomation interaction an approach using automation uncertainty. Hum. Factors J. Human Factors Ergon. Soc. 55(6), 1130–1141 (2013)

    Article  Google Scholar 

  5. Biewald, L.: Why human-in-the-loop computing is the future of machine learning. http://www.computerworld.com/article/3004013/robotics/why-human-in-the-loop-computing-is-the-future-of-machine-learning.html (2015). [Online; Accessed 4 June 2017]

  6. Bridgwater, A.: Machine learning needs a human-in-the-loop. https://www.forbes.com/sites/adrianbridgwater/2016/03/07/machine-learning-needs-a-human-in-the-loop/#2bbb379f4cab (2016). [Online; Accessed 4 June 2017]

  7. Camerer, C., Weber, M.: Recent developments in modeling preferences: uncertainty and ambiguity. J. Risk Uncertain. 5(4), 325–370 (1992)

    Article  Google Scholar 

  8. Chen, F., Zhou, J., Wang, Y., Yu, K., Arshad, S.Z., Khawaji, A., Conway, D.: Robust Multimodal Cognitive Load Measurement, 1st edn. Springer, Berlin (2016)

    Book  Google Scholar 

  9. Clark, H.H., Brennan, S.E.: Grounding in communication. In: Perspectives on Socially Shared Cognition, pp. 127–149. American Psychological Association (1991)

    Google Scholar 

  10. Edwards, J.A., Snyder, F.J., Allen, P.M., Makinson, K.A., Hamby, D.M.: Decision making for risk management: a comparison of graphical methods for presenting quantitative uncertainty. Risk Anal. Off. Publ. Soc. Risk Anal. 32(12), 2055–2070 (2012)

    Article  Google Scholar 

  11. Eyster, E., Weizsacker, G.: Correlation Neglect in Financial Decision-Making. DIW Berlin Discussion Paper No. 1104 (2010). https://doi.org/10.2139/ssrn.1735339

  12. Fiedler, S., Glockner, A.: The dynamics of decision making in risky choice: an eye-tracking analysis. Front. Psychol. 3 (2012)

    Google Scholar 

  13. Figner, B., Murphy, R.O.: Using skin conductance in judgment and decision making research. In: A Handbook of Process Tracing Methods for Decision Research: A critical Review and User’s Guide, pp. 163–184 (2010)

    Google Scholar 

  14. Franco-Watkins, A.M., Johnson, J.G.: Applying the decision moving window to risky choice: Comparison of eye-tracking and mouse-tracing methods. Judgm. Decis. Mak. 6(8), 740–749 (2011)

    Google Scholar 

  15. Gog, T.V., Paas, F., Marcus, N., Ayres, P., Sweller, J.: The mirror neuron system and observational learning: implications for the effectiveness of dynamic visualizations. Educ. Psychol. Rev. 21(1), 21–30 (2008)

    Google Scholar 

  16. Gütl, C., Pivec, M., Trummer, C., Garcabarrios, V.M., Mdritscher, F., Pripfl, J., Umgeher, M.: ADELE (adaptive e-learning with eye-tracking): theoretical background, system architecture and application scenarios. Eur. J. Open Distance E-Learn. 2 (2005)

    Google Scholar 

  17. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  18. Hahn, T., Notebaert, K., Anderl, C., Teckentrup, V., Kaßecker, A., Windmann, S.: How to trust a perfect stranger: predicting initial trust behavior from resting-state brain-electrical connectivity. Soc. Cogn. Affect. Neurosci. 10(6), 809–813 (2015)

    Article  Google Scholar 

  19. Heekeren, H.R., Marrett, S., Ungerleider, L.G.: The neural systems that mediate human perceptual decision making. Nature Rev. Neurosci. 9(6), 467–479 (2008)

    Article  Google Scholar 

  20. Hill, B.: Confidence and decision. Games Econ. Behav. 82(C), 675–692 (2013)

    Article  MathSciNet  Google Scholar 

  21. Hsu, M.: Neural systems responding to degrees of uncertainty in human decision-making. Science 310(5754), 1680–1683 (2005)

    Article  Google Scholar 

  22. Huettel, S.A., Stowe, C.J., Gordon, E.M., Warner, B.T., Platt, M.L.: Neural signatures of economic preferences for risk and ambiguity. Neuron 49(5), 765–775 (2006)

    Article  Google Scholar 

  23. Ibrekk, H., Morgan, M.G.: Graphical communication of uncertain quantities to nontechnical people. Risk Anal. 7(4), 519–529 (1987)

    Article  Google Scholar 

  24. Jenkins, Q., Jiang, X.: Measuring trust and application of eye tracking in human robotic interaction. In: Proceedings of the 2010 Industrial Engineering Research Conference (2010)

    Google Scholar 

  25. Kahneman, D., Tversky, A.: Prospect theory: an analysis of decision under risk. Econometrica 47(2), 263–91 (1979)

    Article  Google Scholar 

  26. Kelsey, D.: Maxmin expected utility and weight of evidence. Oxf. Econ. Pap. 46(3), 425–444 (1994)

    Article  Google Scholar 

  27. Khawaji, A., Chen, F., Zhou, J., Marcus, N.: Trust and cognitive load in the text-chat environment: the role of mouse movement. In: Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design, OzCHI ’14, pp. 324–327 (2014)

    Google Scholar 

  28. Kirschenbaum, S.S., Trafton, J.G., Schunn, C.D., Trickett, S.B.: Visualizing uncertainty: the impact on performance. Hum. Factors 56(3), 509–520 (2014)

    Article  Google Scholar 

  29. Köbberling, V.: Strength of preference and cardinal utility. Econ. Theory 27(2), 375–391 (2006)

    Article  MathSciNet  Google Scholar 

  30. Lee, M.D., Dry, M.J.: Decision making and confidence given uncertain advice. Cogn. Sci. 30(6), 1081–1095 (2006)

    Article  Google Scholar 

  31. Liao, H., Xu, Z., Zeng, X.J., Merigó, J.M.: Qualitative decision making with correlation coefficients of hesitant fuzzy linguistic term sets. Knowl.-Based Syst. 76, 127–138 (2015)

    Article  Google Scholar 

  32. Moll, J., De Oliveira-Souza, R., Zahn, R.: The neural basis of moral cognition: sentiments, concepts, and values. Ann. N. Y. Acad. Sci. 1124, 161–180 (2008)

    Article  Google Scholar 

  33. Moran, R., Teodorescu, A.R., Usher, M.: Post choice information integration as a causal determinant of confidence: novel data and a computational account. Cogn. Psychol. 78, 99–147 (2015)

    Article  Google Scholar 

  34. Morgado, P., Sousa, N., Cerqueira, J.J.: The impact of stress in decision making in the context of uncertainty. J. Neurosci. Res. 93(6), 839–847 (2015)

    Article  Google Scholar 

  35. Pallier, G., Wilkinson, R., Danthiir, V., Kleitman, S., Knezevic, G., Stankov, L., Roberts, R.D.: The role of individual differences in the accuracy of confidence judgments. J. Gen. Psychol. 129(3), 257–299 (2002)

    Article  Google Scholar 

  36. Peper, E., Harvey, R., Lin, I.M., Tylova, H., Moss, D.: Is there more to blood volume pulse than heart rate variability respiratory sinus arrhythmia, and cardiorespiratory synchrony? Biofeedback 35(2), 54–61 (2007)

    Google Scholar 

  37. Platt, M.L., Huettel, S.A.: Risky business: the neuroeconomics of decision making under uncertainty. Nature Neurosci. 11(4), 398–403 (2008)

    Article  Google Scholar 

  38. Preuschoff, K., ’t Hart, B.M., Einhäuser, W.: Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front. Neurosci. 5, 115 (2011)

    Google Scholar 

  39. Ray, P.K., Sahu, S.: Productivity measurement through multi-criteria decision making. Eng. Costs Prod. Econ. 20(2), 151–163 (1990)

    Article  Google Scholar 

  40. Rothbard, M.N.: Toward a reconstruction of utility and welfare economics. Cent. Lib. Stud. (1977)

    Google Scholar 

  41. Scantamburlo, T.: Machine learning in decisional process: a philosophical perspective. ACM SIGCAS Comput. Soc. 45(3), 218–224 (2015)

    Article  Google Scholar 

  42. Smith, L.J., Anand, P., Benattayallah, A., Hodgson, T.L.: An fMRI investigation of moral cognition in healthcare decision making. J. Neurosci. Psychol. Econ. 8(2), 116–133 (2015)

    Article  Google Scholar 

  43. Smith, P.J., Geddes, N.D., Beatty, R.: Human-centered design of decision-support systems. In: Sears, A., Jacko, J.A. (eds.) Human-Computer Interaction: Design Issues, Solutions, and Applications. CRC Press (2009)

    Google Scholar 

  44. Stickel, C., Ebner, M., Steinbach-Nordmann, S., Searle, G., Holzinger, A.: Emotion detection: application of the valence arousal space for rapid biological usability testing to enhance universal access. In: Stephanidis, C. (ed.) Universal Access in Human-Computer Interaction. Addressing Diversity. Lecture Notes in Computer Science, vol. 5614, pp. 615–624. Springer, Berlin (2009)

    Chapter  Google Scholar 

  45. Taghavifard, M.T., Damghani, K.K., Moghaddam, R.T.: Decision Making Under Uncertain and Risky Situations. Enterprise Risk Management Symposium Monograph Society of Actuaries, Schaumburg, Illinois, Tech. rep (2009)

    Google Scholar 

  46. Tversky, A., Kahneman, D.: Judgment under uncertainty: heuristics and biases. Science 185(4157), 1124–1131 (1974)

    Article  Google Scholar 

  47. Wagstaff, K.: Machine learning that matters. In: Proceedings of ICML2012, pp. 529–536 (2012)

    Google Scholar 

  48. Watanabe, S.: Pattern Recognition: Human and Mechanical. John Wiley & Sons Inc, New York, USA (1985)

    Google Scholar 

  49. White, T.P., Engen, N.H., Sørensen, S., Overgaard, M., Shergill, S.S.: Uncertainty and confidence from the triple-network perspective: voxel-based meta-analyses. Brain Cogn. 85, 191–200 (2014)

    Article  Google Scholar 

  50. Ye, J.: Multicriteria decision-making method using the correlation coefficient under single-valued neutrosophic environment. Int. J. Gen. Syst. 42(4), 386–394 (2013)

    Article  MathSciNet  Google Scholar 

  51. Zhou, J., Arshad, S.Z., Wang, X., Li, Z., Feng, D., Chen, F.: End-user development for interactive data analytics: uncertainty, correlation and user confidence. IEEE Trans. Affect. Comput. (2017)

    Google Scholar 

  52. Zhou, J., Arshad, S.Z., Yu, K., Chen, F.: Correlation for user confidence in predictive decision making. In: Proceedings of the 28th Australian Conference on Computer-Human Interaction, OzCHI’16, pp. 252–256 (2016)

    Google Scholar 

  53. Zhou, J., Bridon, C., Chen, F., Khawaji, A., Wang, Y.: Be informed and be involved: effects of uncertainty and correlation on user’s confidence in decision making. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, pp. 923–928 (2015)

    Google Scholar 

  54. Zhou, J., Chen, F.: Making machine learning useable. Int. J. Intell. Syst. Tech. Appl. 14(2), 91 (2015)

    MathSciNet  Google Scholar 

  55. Zhou, J., Jung, J.Y., Chen, F.: Dynamic workload adjustments in human-machine systems based on GSR features. In: Human-Computer Interaction – INTERACT 2015. Lecture Notes in Computer Science, pp. 550–558. Springer, Berlin (2015)

    Chapter  Google Scholar 

  56. Zhou, J., Khawaja, M.A., Li, Z., Sun, J., Wang, Y., Chen, F.: Making machine learning useable by revealing internal states update & #45; a transparent approach. Int. J. Comput. Sci. Eng. 13(4), 378–389 (2016)

    Article  Google Scholar 

  57. Zhou, J., Sun, J., Chen, F., Wang, Y., Taib, R., Khawaji, A., Li, Z.: Measurable decision making with GSR and pupillary analysis for intelligent user interface. ACM Trans. Comput. Hum. Interact. 21(6), 33:1–33:23 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by AOARD under grant No. FA2386-14-1-0022 AOARD 134131.

Author information

Authors and Affiliations

  1. DATA61, CSIRO, 13 Garden Street, Eveleigh, NSW, 2015, Australia

    Jianlong Zhou, Kun Yu & Fang Chen

Authors
  1. Jianlong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianlong Zhou .

Editor information

Editors and Affiliations

  1. DATA61, CSIRO, Eveleigh, New South Wales, Australia

    Jianlong Zhou

  2. DATA61, CSIRO, Eveleigh, New South Wales, Australia

    Fang Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Crown

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhou, J., Yu, K., Chen, F. (2018). Revealing User Confidence in Machine Learning-Based Decision Making. In: Zhou, J., Chen, F. (eds) Human and Machine Learning. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-90403-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-90403-0_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-90402-3

  • Online ISBN: 978-3-319-90403-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation