Professor Nanette Veilleux has received a collaborative grant from the National Science Foundation (NSF) to study the connection between prosody and meaning. Other researchers include syntactician Byron Ahn (Princeton), psycholinguist Stefanie Shattuck-Hufnagel (MIT), and linguist Alejna Brugos, a visiting scholar at Simmons who is also a long-time collaborator with Professor Veilleux. The ultimate goal of the project is to clarify the meaning in prosody — inflections in tone and lengthening syllables as we speak.
“Everyone thinks how you say something has meaning, but it’s hard to pin down,” explains Veilleux. “If someone is telling you something as if you don’t already know it, you pick that up from how they say it — that is prosody. Think of how frequently you infer something from the tone of someone’s voice.”
“Prosody is the music of language,” says psycholinguist Shattuck-Hufnagel. “This work explores a new system for annotating prosody—particularly the intonational aspects heard by human listeners as pitch changes.”
This new system, called PoLaR for Points, Levels and Ranges, offers a means of analyzing the detailed ways in which the pitch pattern of one spoken utterance differs from another. Shattuck-Hufnagel explains that “while the proposed work is focused on how speakers use these detailed patterns to signal particular meanings, such as the type of information known to each person in a conversation, this new tool also has a much broader potential to reveal important differences and commonalities across languages and dialects, and to extend prosodic analysis to clinical domains.”
Prosody is the music of language.
Brugos is also excited to be a part of the cross-disciplinary collaboration. “As a phonetic researcher, I look forward to using the new experimental methods and annotation tools to capture how fine-grained differences in the acoustic signal play a role in the expression and perception of prosodic categories,” she says. “I hope that our research will uncover important aspects of prosodic patterns and their variations. This knowledge has great potential to feed into research in multiple fields, including linguistics, computer-human interactions, as well as for work with speech disorders."
The results of this work may ultimately improve human-computer interfaces, such as speech synthesis and recognition. “As humans, we decode prosody pretty well, but computers use literal word for word translation,” says Veilleux. “Computers can’t distinguish between background, lead-up or relevant information. They have no way of using prosody to maintain continuity over several sentences.”
College of Organizational, Computational, and Information Sciences Dean Marie desJardins, whose research focuses on Artificial Intelligence (AI) is also aware of the possibilities of this research.
“Artificial intelligence (AI) is increasingly powerful and relevant for helping to solve complex problems,” says desJardins. “Natural language understanding is one of the grand challenges for AI systems, and human-level spoken language understanding requires understanding not just words, but the tone, cadence, and inflection of speech. This important problem is at the core of Dr. Veilleux's collaborative work."