AI developed by UC engineers offers insight into conversations using physiology alone
Could an app let you know if a first date is just not that into you?
Engineers at the University of Cincinnati say the technology might not be far off. They trained a computer — using data from wearable technology that measures respiration, heart rates and perspiration — to identify the type of conversation two people were having based on their physiological responses alone.
Researchers studied a phenomenon in which people’s heart rates, respiration and other autonomic nervous system responses become synchronized when they talk or collaborate. Known as physiological synchrony, this effect is stronger when two people engage deeply in a conversation or cooperate closely on a task.
“Physiological synchrony shows up even when people are talking over Zoom,” said study co-author Vesna Novak, an associate professor of electrical engineering in UC’s College of Engineering and Applied Science.
University of Cincinnati engineering students demonstrate how they taught a computer to distinguish types of conversations based only on physiological cues as part of a project in Associate Professor Vesna Novak's lab. Photo/Andrew Higley/UC Marketing + Brand
In experiments with human participants, the computer was able to differentiate four different conversation scenarios with as much as 75% accuracy. The study is one of the first of its kind to train artificial intelligence how to recognize aspects of a conversation based on the participants’ physiology alone.
The study was published in the journal IEEE Transactions on Affective Computing.
Lead author and UC doctoral student Iman Chatterjee said a computer could give you honest feedback about your date — or yourself.
“The computer could tell if you’re a bore,” Chatterjee said. “A modified version of our system could measure the level of interest a person is taking in the conversation, how compatible the two of you are and how engaged the other person is in the conversation.”
UC doctoral student Iman Chatterjee, lead author of the study, demonstrates how they used sensors to measure the physiology of two people having a conversation. Photo/Andrew Higley/UC Marketing + Brand
Chatterjee said physiological synchrony is likely an evolutionary adaptation. Humans evolved to share and collaborate with each other, which manifests even at a subconscious level, he said.
“It is certainly no coincidence,” he said. “We only notice physiological synchrony when we measure it, but it probably creates a better level of coordination.”
Studies have shown that physiological synchrony can predict how well two people will work together to accomplish a task. The degree of synchrony also correlates with how much empathy a patient perceives in a therapist or the level of engagement students feel with their teachers.
“You could probably use our system to determine which people in an organization work better together in a group and which are naturally antagonistic,” Chatterjee said.
This aspect of affective computing holds huge potential for providing real-time feedback for educators, therapists or even autistic people, Novak said.
“There are a lot of potential applications in this space. We’ve seen it pitched to look for implicit bias. You might not even be aware of these biases,” Novak said.
Novak studies rehabilitation robotics and wearable technology among other topics in her lab at UC.
Novak and her students were able to teach the computer how to recognize four types of conversations based on five physiological indicators: chest and nose respiration, an electrocardiogram, skin conductance and peripheral skin temperature.
Individually, Novak said, these measurements can’t say much about interpersonal relations. Each physiological signal can be statistically noisy and hard to interpret. But researchers were able to sift through the noise by applying pattern recognition algorithms.
Sixteen pairs of participants discussed possible topics that they could strongly agree on or disagree on before engaging in four different conversations:
- A positive conversation in which they happily talk about a topic in which they shared a similar opinion.
- A negative conversation in which they unhappily discuss a topic over which they disagree.
- Two conversations about an agreeable topic in which each participant takes a turn dominating the discussion.
Every three out of four times, the artificial intelligence was able to identify the type of conversation (one-sided, two-sided, positive or negative) based only on what the participants’ bodies told the machine.
The study was supported by grants from the University of Wyoming and the National Institutes of Health.
Novak said their findings raise tantalizing questions about what else computers can tell us about interpersonal relations.
“Our next step is to see how much nuance we can separate,” she said. “We’ve shown that AI has the ability to identify positive versus negative conversations, but can you separate shades of gray that humans wouldn’t discern?”
UC College of Engineering and Applied Science Associate Professor Vesna Novak studies rehabilitation engineering and human-machine interaction, among other topics. Photo/Andrew Higley/UC Marketing + Brand
Featured image at top courtest Unsplash
All other photos/Andrew Higley/UC Marketing + Brand
Digital design/Kerry Overstake/UC Digital + User Experience
Become a Bearcat
Whether you’re a first-generation student or from a family of Bearcats, UC is proud to support you at every step along your journey. We want to make sure you succeed — and feel right at home.
Related Stories
Championing science amid adversity
University of Cincinnati epidemiologist Diego Cuadros is used to telling people what they don’t want to hear. The assistant professor runs the Health Geography and Disease Modeling Lab in UC’s College of Arts and Sciences, where he studies global topics such as HIV, malaria and, this year, COVID-19. He condenses data into easy-to-follow maps that predict the future with uncanny accuracy.
Social justice through medicine
Carl Fichtenbaum, a professor in the Division of Infectious Diseases at the University of Cincinnati College of Medicine and one of the leaders in UC’s efforts to combat COVID-19, is still going to protest marches as his passion for social justice, personally and profession- ally, burns as brightly as ever.
‘Somos familia’
April 26, 2023
“Somos familia” means “we are family,” and students from throughout the Latinx community are building family, finding success and continuing a legacy at UC.
Traffic of tomorrow
June 10, 2020
More driverless cars. More networked roads. More naps. UC is helping change how we drive.
Remotely possible
When did the COVID-19 pandemic first make an impact on your life? March 10? That was the day the University of Cincinnati decided to change something it has excelled at for 200 years. Teaching. Educating. That day UC announced that all lectures in classrooms, experiments in labs or designing in studios would be suspended. Students started what was expected to be just an extended spring break, but then 12 days later all courses had gone virtual to protect the university community and stop the virus’s spread.
A milestone centennial
March 8, 2021
In 1920, UC became the first university in the nation to open cooperative education — founded at UC in 1906 — to women.