When your dignity is on the line, the simple act of speaking can be intimidating. Whether you're meeting a prospective employer, attempting to charm a date, or addressing a large group of people in public, it's natural to feel tongue-tied when a conversation or speech is clouded by fear of being judged harshly.

That kind of situational anxiety permeates the daily existence of about 15 million American adults with social phobias, according to the National Institute of Mental Health (NIMH), and can be even more debilitating for people with high-functioning autism, or Asperger's syndrome, who often have trouble maintaining eye contact, reading social cues, and following normal rhythms of conversation.

Intensive public speaking classes may be out of reach for many with social anxiety, but a new virtual conversation coaching program from researchers at the Massachusetts Institute of Technology (MIT) may be able to boost people's conversation skills enough to overcome all kinds of pitfalls — from garden-variety social awkwardness to a crippling fear of public speaking.

A Virtual Conversation Coach

The software, designed to operate on a standard laptop computer by a team led by M. Ehsan Hoque of the MIT Media Lab, is called MACH, short for "My Automated Conversation coacH."

MACH users practice conversations with a female or male 3-D animated virtual coach rendered on a computer screen, who is designed to look professional and eerily lifelike.

The software automatically analyzes the user's facial, behavioral, and speech cues picked up by the computer's webcam and microphone, focusing on specific measures like speaking volume and rate, pitch variation, eye contact, smiles, and other facial expressions, in addition to head movements like nods, shakes, and tilts.

The virtual coach mirrors the user's cues by smiling and nodding, and keeps the conversation afloat by asking and answering questions.

After the conversation ends, MACH provides video of the user's performance, accompanied by detailed feedback about the recorded conversation measures, like how well they maintained eye contact and modulated their voices, as well as how often they smiled, paused, and used filler words such as "umm," "basically," "like," and "totally."
Testing Users' Speaking Skills in Job Interviews

Hoque's team tested MACH on 90 MIT juniors, who they figured might be looking for an edge on the hypercompetitive job interview process most of them expected to face after graduating.

The participants, all of whom were native English speakers, went through two simulated job interviews with college career counselors that were one week apart.

The participants were divided into three groups based on the type of help they were given between the two interviews. One group watched a series of interview advice videos; a second group practiced an interview with MACH, but was given no feedback except the video of their performance; and a third group practiced with MACH and received full feedback.

MACH provides two forms of feedback: summary feedback (top) that captures the overall interaction, and focused feedback (bottom) that allows participants to watch their own video for nonverbal behaviors, like smiles, head movements, and intonation change. [- M. Ehsan Hoque]

An independent group of career counselors rated the students' performance during both interviews, without knowing what kind of help each one received.

The assessments showed that while the performances of those in the first two groups did not change much between the two interviews, participants in the third group were rated significantly better in their second interviews on subjective measures like "appears excited about job," "overall performance," "recommend hiring this person."

Most of the MIT students who used MACH also seemed to appreciate the feedback on their conversation skills, saying they were likely to use the program in preparation for future social encounters.

Future Applications: A Social Network Based on Speaking Skills

In a news release, Hoque explained that people with social phobias often express a need for an automated system like MACH, which can help them practice conversation skills on their own time, at their own speed.

"They desire to control the pace of the interaction, practice as many times as they wish, and own their data," he said.

In a paper on the software, which Hoque expects to present at the International Joint Conference on Pervasive and Ubiquitous Computing, known as UbiComp, this September, the researchers write that they expect to optimize the program for use on mobile devices like smartphones and tabletss as well as on different operating systems.

They also hope to train the software to focus on social situations other than job interviews, and to develop it specifically for people with communication difficulties, like those with autism spectrum disorder.

If MACH catches on, they plan to create a sort of social network based entirely on the face-to-face aspect of being social — a next-level LinkedIn — in which users can track their own communication progress and compare it against that of other users based on factors like education, work history, and location.

Source: Hoque ME, Courgeon M, Martin JC, et al. MACH: My Automated Conversation coacH. 2013.