On eLearning, Avatars, and the “Uncanny Valley”

One of the ways you can personalize an elearning course is to include a host avatar who introduces problems, questions, and provides hints as a learner progresses throughout the course. When these avatars are created from photos of real people, I often wonder why they’re smiling quite so broadly (particularly in a compliance course). While you may doubt the sincerity of your course host, you tend to accept him/her as human. But what about avatars that are created through computer-rendering wizardry?

The Polar Express Phenomenon

You may remember The Polar Express, a motion-capture film about kids wending their way on a trip to the North Pole. Meant to be a feel-good movie, it actually creeped out quite a few people because of the characters’  relatively expressionless eyes and mouths.

The Uncanny Valley

This leads to a discussion of the “Uncanny Valley.” As described by Tinwell, Grimshaw, Nabi & Williams  (“Tinwell et al.”) in their 2011 paper, the Uncanny Valley phenomenon refers to the negative reaction virtual characters can evoke when they are very life-like but just not quite lifelike enough. Citing Kang (2009), Tinwell et al. suggest that:

[T]he negative impact of the uncanny is related to how much of a threat a character is perceived to be and how much control we have over the potentially threatening or dangerous interaction.

Some avatars may be uncannier than others

As reviewed in Tinwell et al., it’s been suggested that there’s an evolutionary benefit to being able to recognize emotions on the faces of our fellow human beings. Tinwell et al. postulated that being unable to read emotions that are more intimately connected with survival skills (anger, fear, sadness and disgust) would be more disturbing (strange) than being unable to read than happiness or surprise.

An uncanniness experiment

Tinwell et al.  investigated the relationship between the type of the emotion displayed by an avatar and levels of perceived uncanniness. They compared reactions of 129 male university student with a mean age of 21.8 years (SD = 2.44 years) to:

• a human  actor displaying a range of emotions (anger,
  disgust, fear, happiness, sadness and surprise)
• a virtual character displaying facial expressions relating to the same emotions
• a virtual character displaying no facial expressions relating to emotions

All of the characters said the phrase “the cat sat on the mat,” varying the intonation of the phrase in accordance with an emotional state ((anger,
disgust, fear, happiness, sadness and surprise), with the human actor providing the voices for the virtual characters as well.  In other words, even the facially inexpressive virtual character emoted when it came to actually saying “the cat sat on the mat.”

Participants were asked to watch videos of the actor and avatars in random order and to rate characters using a 9-point scale (with 1 being non-human-like and 9, very human-like). They were also asked to rate uncanniness using a 9-point scale (with a rating of 1, being very strange, and a rating of 9, being very familiar).

Accuracy and uncanniness ratings

Participants were best able to identify anger in both the human actor and emotion-expressing avatar and  least able to recognize happiness, whether the human or the virtual expressive character was displaying this emotion. Participants also attributed emotion to the emotionless avatar (presumably based on its tone of voice).

Which emotions had the most dramatic effect on perceptions of uncanniness (i.e., creepiness)? Facially expressionless virtual characters  were considered far creepier than virtual characters that did display facial expressions, when it came to fear, sadness, surprise, and disgust. Participants rated a facially expressionless avatar saying the “cat sat on the mat” in a fearful voice high on the creepiness scale.  However, being unable to identify facial expressions associated with anger and happiness had less of an impact on perceived uncanniness/creepiness.

Some theories: Incongruence is scary

People expect tone of voice to match facial expressions and when there’s a lack of correlation, it can be perceived as frightening and even threatening. Tinwell et al. postulated that the sensation of uncanniness may act as a sign of unpredictability and danger.  Recognizing certain emotions more accurately may have more of a survival benefit. For example, recognizing your fellow cave dwellers’ fear (There’s a bear behind you!) and sadness (Poor Zorg over there just got eaten by a bear), may be more beneficial than surprise (Roots for dinner? I was counting on bear!).

The lack of correlation between being unable to view a facial expression of anger and perceptions of uncanniness was surprising. You’d expect that being able to discern anger in your fellow human beings would be an important survival skill (I am about to club you!). On the other hand, anger was one of the emotions participants were able to identify most accurately from tone of voice, which may have had an impact on perceptions of uncanniness.

Conclusions and impacts on design

We have to be cautious about drawing conclusions from this study given the subjective rating systems used and the complexities of our own emotional reactions to the emotional displays of others. Additionally, all the study participants were male, university educated, and fairly young. However, some issues are worth thinking about:

• If you are going to use a computer-generated avatar in an elearning experience, its ability to accurately display emotion may have an impact on learner reactions and this impact might be a negative one.
• The impact may be affected by context. For example, it might be more serious in an elearning experience that’s a game or heavily reliant on narrative, where the avatar actually has to express a range of emotions.
• You might be better off using a completely unrealistic cartoon than one that’s lifelike but not quite lifelike enough.

It’s worth noting that even a human actor triggered some perceptions of uncanniness. A display of happiness tended to score higher for uncanniness than other emotions, suggesting that people were reacting to perceptions of insincerity. So even when you use videos or images of real people, your audience may react negatively to certain displays of emotion when they don’t quite “feel right.” Keeping it real is a good rule of thumb even when images of people are used rather than computer-generated avatars.

Reference

Tinwell, A., Grimshaw, M. Nabi, D. A., & Williams, A. (2011). Facial expression of emotion and perception of the Uncanny Valley
in virtual characters. Computers in Human Behavior, 27, 741–749.