Low-Cost Human–Machine Interface for Computer Control with Facial Landmark Detection and Voice Commands

Pablo Ramos*, Mireya Zapata, Kevin Valencia, Vanessa Vargas, Carlos Ramos-Galarza

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Nowadays, daily life involves the extensive use of computers, since human beings are immersed in a technological society. Therefore, it is mandatory to interact with computers, which represents a true disadvantage for people with upper limb disabilities. In this context, this work aims to develop an interface for emulating mouse and keyboard functions (EMKEY) by applying concepts of artificial vision and voice recognition to replace the use of hands. Pointer control is achieved by head movement, whereas voice recognition is used to perform interface functionalities, including speech-to-text transcription. To evaluate the interface’s usability and usefulness, two studies were carried out. The first study was performed with 30 participants without physical disabilities. Throughout this study, there were significant correlations found between the emulator’s usability and aspects such as adaptability, execution time, and the participant’s age. In the second study, the use of the emulator was analyzed by four participants with motor disabilities. It was found that the interface was best used by the participant with cerebral palsy, followed by the participants with upper limb paralysis, spina bifida, and muscular dystrophy. In general, the results show that the proposed interface is easy to use, practical, fairly accurate, and works on a wide range of computers.

Original languageEnglish
JournalSensors
StatePublished - 29 Dec 2022

Bibliographical note

Publisher Copyright:
© 2022 by the authors.

Funding

This work has been supported by the Universidad de las Fuerzas Armadas ESPE grant 2022-EXT-003 ESPE and by the Universidad Tecnológica Indoamérica through the project “Cyber-Physical Systems for Smart Environments-SCEIN” with funding code INV-0012-031.

Fingerprint

Dive into the research topics of 'Low-Cost Human–Machine Interface for Computer Control with Facial Landmark Detection and Voice Commands'. Together they form a unique fingerprint.

Cite this