By David Murphy

Montserrat Ramirez De Angel’s promising research efforts have seen the KAUST Ph.D. student win the recent Women in Sensors (WiSe)-YP Big Idea Pitch Competition. Ramirez De Angel—who is working toward her Ph.D. under the supervision of Professor Khaled Salama—received her “Best Pitch” recognition at the 23rd IEEE Conference on Sensors, held in Kobe, Japan, from October 20–23, 2024.

The bioengineering student took top honors in Kobe for her presentation, 'Magnetic Skin System for Speech Rehabilitation.' The pitch outlined a collaborative project between researchers from the KAUST Biological and Environmental Science and Engineering Division (BESE) and the Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, including Ramirez De Angel, Professor Salama and Eckaard le Roux,  as well as former KAUST researchers Professor Jürgen Kosel, Dr. Abdullah Almansouri and Alexander Przybysz.

The team’s research has resulted in the creation of an assistive magnetic skin system, known as AM2S-SR. The system tracks the mouth movements of individuals with voice or speech sound disorders (SSDs) for communication, speech reconstruction and prediction purposes.

“It felt great to have our work rewarded,” said Ramirez De Angel, whose research combines sensors and artificial intelligence for speech recognition and verbal communication applications. “I am truly fortunate to have the support of my adviser and all my colleagues to participate in such an event. Our technology has a promising potential to treat not only voice disorders but also other injuries and diseases.”

Enabling the power of communication

In the past, magnetic skin has been used to assist people with quadriplegia in controlling their ambulatory devices by tracking the movements of the user’s nose or eyebrows. As a result of this breakthrough, Ramirez De Angel et al. developed an idea to create highly elastic, biocompatible magnetic skin patches that would allow individuals with SSD to communicate naturally.

“Magnetic skin is a very flexible technology, since it can be modified in shape, size, thickness and concentration of the magnetic powder,” she explained. “When combined with magnetic sensors, the system can measure vibration, touch, and movement through magnetized microparticles. Therefore, we wondered if we could achieve the same positive results for people with voice disorders as was achieved for people with quadriplegia."

To achieve communication, as the user moves their lips, a wireless, wearable headset with embedded magnetic field sensors—two magnetphones placed at each side of the face—reads and processes the mouth’s movement. Using a machine learning algorithm, the sensors stream the speech data to a head unit, which analyzes and predicts the user’s intended words and letters.

The speech data is then exported either verbally to a speaker or visually to a display. In previous lab tests, the KAUST-designed technology successfully identified all the letters in the English alphabet with an impressive average success rate of 94.96%. 

At present, the system detects words and recognizable speech patterns. After detecting and predicting sentences, Ramirez De Angel and her colleagues aim to correlate voices with magnetic fields, which can then be translated into any language.

“Moving forward, we hope that the technology can be readily deployed as an assessment tool for speech therapists to diagnose voice disorders, as a therapy tool to treat these diseases or as an everyday communication tool,” she emphasized.

“I want to keep developing our technology for other purposes, such as physical rehabilitation for carpal tunnel syndrome or other related muscular diseases. Our technology has what is needed to be a great product: It is non-invasive, wearable, can be trained for specific scenarios, the AI can be as robust as desired, and it can also be tailored to the needs of the specific user or medical professional. It truly is our hope to give a ‘voice to the voiceless.’”