SpringerLink
Log in

Development of an eye-tracking and voice command interface to facilitate GUI operation for people with disabled upper limbs

  • Long Paper
  • Published:
Universal Access in the Information Society Aims and scope Submit manuscript

Abstract

Purpose

Eye-tracking technology can be used for interaction functions involving the manipulation of various types of web content by replacing the functions of conventional input devices, such as mouse and keyboard. However, eye-tracking technology can also cause serious errors in pointing and executing pointed objects when using a web browser. Owing to the small size of executable objects, pointer execution errors cause the unintended execution of objects and difficulty in pointing.

Methods

We propose an interface that solves these two types of pointer execution errors for people with disabled upper limbs to control a graphical user interface. The proposed interface dynamically magnifies the tracked execution objects and executes objects pointed at by eye-tracking using voice commands. We develop a Chrome extension for this interface and use the interface by embedding the program in the browser.

Results

We verify the effect of this interface by conducting experiments on the error reductions of pointer execution and on proper magnification ratio identification. The results show that the eye-voice interface lowered the error rate of pointer execution and proved effective.

Conclusion

We confirmed that an eye-tracking interface combined with voice commands is effective in executing menus and execution objects in a web browser.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Thailand)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

Data availability

Applicable.

Code availability

Applicable.

References

  1. Park, J.H., Park, S., Lee, M., Lim, S.B.: Voice activity detection algorithm using wavelet band entropy ensemble analysis in car noisy environments. J. Korea Multimed. Soc. 21(11), 1342–1352 (2018)

    Google Scholar 

  2. **, D.H.: A study on Usability Evaluation of an Eye Mouse-Based on the Function of Click. In: Double-Click, Drag, and Scroll Master’s Thesis, pp. 1–16. Seoul National University, Seoul (2016)

    Google Scholar 

  3. Murata A, Uetsugi R, Hayami T (2012) Study on cursor shape suitable for eye-gaze input system. In: 2012 Proceedings of SICE Annual Conference (SICE). pp 926–931.

  4. Choi J (2017) A digital publishing framework for crowdsourcing based adaptive eBook contents PhD thesis. Sookmyung Women’s University.

  5. Menges R, Kumar C, Müller D, Sengupta K. (2017) GazeTheWeb: a Gaze-Controlled Web Browser. In: Proceedings of the 14th Web for All Conference on the future of accessible work. pp 25–42

  6. Chetcuti A, Porter C (2016) Butterfleye: Supporting the Development of Accessible Web Applications for Users with Severe Motor-Impairment. In: Proceedings of the 30th International BCS Human Computer Interaction Conference.

  7. Kwak, S., Kim, I., Sim, D., Lee, S.H., Hwang, S.S.: A computer access system for the physically disabled using eye-tracking and speech recognition. J. HCI Soc. Korea 12(4), 5–15 (2017)

    Article  Google Scholar 

  8. Python. https://www.python.org. Accessed (24 Mar 2018)

  9. tobii_research API. http://developer.tobiipro.com. Accessed (24 Mar 2018)

  10. Google Cloud Speech API. https://cloud.google.com/speech-to-text/?hl=ko. Accessed (24 Mar 24 2018)

  11. Tobii Etyetracket X130. https://www.tobiipro.com. Accessed (24 Mar 2018)

  12. Window API. https://docs.microsoft.com/en-us/windows/win32/apiindex/api-index-portal. Accessed (24 Mar 2018)

Download references

Acknowledgements

This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (NRF-2018R1A4A1025559).

Author information

Authors and Affiliations

  1. Department of IT Engineering, Sookmyung Women’s University, Seoul, Korea

    Soon-Bum Lim

  2. Research Institute of ICT Convergence, Sookmyung Women’s University, Seoul, Korea

    Joo Hyun Park

Authors
  1. Soon-Bum Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joo Hyun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joo Hyun Park.

Ethics declarations

Conflicts of interest

On behalf of all authors, the corresponding author states that there are no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lim, SB., Park, J.H. Development of an eye-tracking and voice command interface to facilitate GUI operation for people with disabled upper limbs. Univ Access Inf Soc 23, 329–343 (2024). https://doi.org/10.1007/s10209-022-00939-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-022-00939-y

Keywords

Search

Navigation