Starting Spring 2025 and rolling into the fall, I will be a research assistant at RPI! This research dives into vocal science, or pedagogy, aiming to help both AAC users and singers around the world.
This research focuses on using a software titled FonaDyn, of which analyses various forms of speech, singing, and data collected from an EGG device (electroglottogram) of which collects data on vocal cords. Using this software, we can analyze various aspects of the voice sample, of which both singing and speaking would provide different data. The data we are interested in is the analysis of the voice between human speech and AI speech, as well as human and AI singing. From the information we gather, in terms of AI vs. human, we are working with music software company Dreamtonics to create an AI that can both speak and sing like a human (it should be noted that we only want the good aspects of this, not to harm anyone) and then use that technology to help AAC users.
We are focused on helping AAC users since the voice box that they use all come with the same 6 stock voices. As such, they don't have their own personal voice like most people do. This technology will give AAC users the chance to personalize and have their own unique voice that otherwise would cost tens of thousands of dollars to create.
Furthermore, we are actively using this technology to help compose and create an opera, "The Other Side of Science" of which stars a non-binary non-verbal AAC-using character thats moving into a fully automated house in the middle of nowhere. This technology would help us create a voice that would allow the main character to sing.
This research is also ever evolving and more people from around the world are also assisting us in our research, including the creator of FonaDyn. As we learn more and continue our research, we will present our findings at the Voice Foundation's Symposium in Philadelphia, tentatively in 2026.
You can find a link to the New York Times article on "The Other Side of Silence" here
You can find the link to the Opera Saratoga article here