It was a Friday night. Myself and my two best friends were deciding for what felt like an eternity how we wanted to spend our night. Sally, I will call her, wanted to spend the night in, watching Netflix and eating pizza. Ally wanted to go out and grab some drinks at a local bar. I was indifferent and would have been satisfied with either decision. Sally said she had a long day and just wanted to enjoy some quiet time at home. Ally, on the other hand, said she thought she deserved a night out after a long and stressful week. The two went back and forth until we decided that “Netflix and chilling” was how we would be spending our Friday night. Sally was content with our decision, but she looked over to Ally and noticed she did not look too thrilled. She asked Ally if she was alright with the decision. She smiled. Sally interpreted Ally’s smile as confirmation that she was happy with the decision and so she continued on to enjoy her night. Myself, on the other hand, I did not see contentment in Ally’s overall disposition. Something about her body – the slouching and her head hanging into her chest (Wallbott, 1998) – was telling me otherwise.
There is a lot going on in this above scenario. Sally and Ally were obviously in disagreeance about how they wanted to spend their night. Although Ally’s facial expression appeared to denote that she approved the decision, her body posture clearly communicated something different. Both of these aspects of Ally’s behavior make-up her body language. Many experts, including Joe Navarro, a former FBI agent turned body language expert, define body language as “the unspoken element of communication that we use to reveal our true feelings and emotions. Our gesture, facial expressions and posture, for instance” (“Body Language,” n.d.). Whether intentional or not, Ally used two different forms of body language to communicate how she was feeling. She used a smile to communicate that she appeared to be content with the decision and a sluggish body posture to communicate that she may actually not be. Evidently, Sally was attending to Ally’s smile to uncover how she was feeling – she failed to take into consideration her body posture.
The idea of using someone’s facial expressions to recognize what emotion they are feeling is not new. Emotion researchers Dael, Mortillaro, and Scherer (2012) share that “it has long been assumed that whereas a number of facial muscle configurations are reliable indicators of specific emotions, body movements or posture provide information of only gross affect state or emotion intensity” (p.1085). Essentially, due to the fact that certain emotions are associated with specific facial expressions, we as a species tend to use these expressions to distinguish one emotion from the next. For example, a smile is a globally recognized way of expressing happiness, and a frown, sadness (Ekman & Friesen, 1969).
These tendenices can be attributed to the works of Dr. Paul Ekman, who has been deemed the father of emotions and their expressions. In 1969 and later in 1971, alongside Wallace Friesen, the two men discovered that the emotions of happiness, sadness, anger, disgust, surprise, and fear, known as the basic emotions, each had distinct facial expressions that made them differentiable from one another (Ekman & Friesen, 1969). They also found that people across cultures were able to recognize these expressions with considerable accuracy when provided images of actors expressing these emotions on their faces (Ekman & Friesen, 1971). From these studies, we learn that facial expressions are important in detecting what emotion an individual is portraying. Likewise, they are also important in communicating an emotion. As a result, it might have been fair to assume in earlier days that facial expressions were the meat and bones of detecting an emotion. Nonetheless, Dael et al. (2012) share that “this view is no longer supported by recent studies showing that variations in body movement and posture convey specific information about a person’s emotional state” (p.1086). After all, the word emotion is derived from the latin word emovere which literally means to move (- movere) out (e-). Thus, body movement must play some part in the recognition of emotions (“Emotion,” n.d.).
My skepticism in regards to Sally’s interpretation of Ally’s emotions was in part due to Ally’s face “saying” one thing, and her body “saying” another. I was put off by these differences in her facial and body expressions. Consequently, I truthfully did not know what to make of Ally’s feelings – was she upset, was she happy? Naturally, as a Psychology student, I turned to the research to find some answers. “What insightful and revolutionary answers did you find?,” you may be asking yourself. To put it simply, body expressions such as body postures and gestures, have extremely powerful roles in the recognition of emotions. What I found might sound trivial, perhaps even like common sense, but nonetheless it is an important and often overlooked idea in emotion research and literature (e.g. Schindler, Van Gool & de Gelder, 2008; Dael et al., 2012).
The Power of Incongruence
One way in which body expressions influence the recognition of emotions is illuminated through the idea of incongruence – a mismatch between the emotions displayed in one’s face and body (Meeren, van Heijnsbergen, & de Gelder, 2005). As I briefly mentioned, Ally’s facial display of contentment (i.e. her smile), did not match her sluggish body posture. Now, imagine yourself in a similar and very common situation. Say your boss reads your business proposal, smiles and appears to like what they see, but, they do not make eye contact and continuously look out the window (“Body Language,” n.d.). What do you make of this? Is the body trying to say something? Or is it affecting our processing of the facial expression? Meeren et al. (2005), emotions researchers from Tilburg and Harvard University, share that “incongruent bodily expression hampers facial expression recognition” (p. 16521). Considering all the ambiguities that can arise from a mismatch between facial and body expressions in everyday life, these three researchers were the first to study how such a mismatch between body movements and facial expression can influence the detection of an emotion an individual is trying to display from their face.
Twelve participants were brought into Meeren and colleague’s cognitive and neuroscience laboratory at Tilburg University in the Netherlands. The researchers put together different images showing an individual either portraying fear or portraying anger. However, the images were altered such that some pictures had congruent facial expressions and body postures and some had incongruent facial expressions and body postures. Congruent images included an actor whose fearful facial expression matched their fearful body posture, and likewise for anger. Incongruent images included an actor displaying an angry face, but a fearful body posture, or an actor displaying a fearful face, but an angry body posture. All faces were photoshopped onto the body of an actor. The participants were then simply asked to determine if the actor was expressing anger or fear. But there was a catch, participants had to make these judgments based on the facial expressions, even while they were viewing the whole face-body image (Meeren et al., 2005).
It turns out that participants were not too great in accurately detecting whether the actor was expressing fear or anger when the facial expressions and body posture were incongruent. In fact, they were only right sixty-seven percent of the time (Meeren et al., 2005).
“When observers have to make judgments about a facial expression,” Meeren et al. (2005) observed, “their perception is biased toward the emotional expression conveyed by the body” (p.16521). The idea here is that our overall body is like a sealed container, and our body language, especially our postures and gestures can “leak out” our true feelings (e.g. Ekman & Friesen, 1969). Thus, we become more attuned to these body expressions because they perhaps share an individual’s true feelings. Relating this back to my experience with Ally, it is now clear that I was so focused on Ally’s body expression that it inhibited me from interpreting any emotion from her face. Does this mean I was wrong in suspecting that Ally might have been upset?
Definitely not! If we think of our facial expressions as “emotion seeds,” Aviezer et al. (2008) explain, “that lie dormant in isolated faces, they can be activated by appropriate context….These seeds ‘sprout’ and may override the original expression of the target face” (p.731). This metaphor suggests that context, especially body movement, allows the facial expression to “grow” or “morph” into what the context is suggesting.
In their research, Aviezer et al. (2008) conducted three studies to show how our perception of someone’s emotion can change in relation to the context in which the facial expression is elicited. In one of their studies, undergraduate students at the University of Toronto were presented with images of actors who expressed disgust on their faces. However, this facial display of disgust was then paired with different contexts. Interestingly, the differing contexts were actually different body postures. Thus, the researchers were extending work on previous studies that looked at incongruency in facial and body expressions. Disgust faces were either paired with fear, anger, sadness, or disgust body expressions. Again, participants were simply asked to indicate what emotion the facial expressions elicited, choosing between the basic emotions of sadness, anger, fear, disgust, happiness, and surprise.
Consistent with previous work, when a disgust facial expression was matched with a contextual display of disgust (i.e. the body showed disgust), participants accurately identified this emotion ninety-one percent of the time. However, when the contextual display showed anger, fear, or sadness, accuracy of judging the face as disgusted decreased. “This systematic decline in accuracy indicates that participants did not simply ignore the faces” (Aviezer et al., 2008, p. 726), the researchers explain. Moreover, perhaps most interesting of the results was that when the context (i.e. the body) showed anger, eighty-seven percent of participants identified the facial expression as eliciting anger rather than disgust. The authors continue to explain that “facial expressions can be perceived as conveying strikingly different emotions depending on the bodily context in which they appear” (Aviezer et al., 2008, p. 731). Although Ally’s facial and body expressions showed two extreme ends of emotions, this work suggests that her body provided some context for me to make sense of her facial expression. She was smiling, but to me her body suggested she was upset, and I used that information to re-interpret the emotion she was displaying on her face.
Let us take a pause and recall what we have learned thus far regarding how powerful body expressions can be in the recognition of emotion. Essentially, body expressions can “hamper” (Meeren et al., 2005, p. 16521) our perceptions of an emotion being expressed through the face because we typically attend to these contextual cues to make sense of the emotion. This is especially prevalent when body and facial expressions are incongruent. The next natural question becomes, why are we so influenced by incongruency in body and facial expressions? Is there something that is happening in the brain when facial and body expressions do not line up?
Rajhans, Jessen, Missana, and Grossmann (2016) sought out to determine exactly that. They were interested in learning what brain areas become activated when infants partake in emotion perception – that is, when attached to neural imaging machines, what brain areas light up when infants are trying to recognize an emotion from both the body and face? They were specifically interested in learning how incongruent body expressions impact the processing of facial expressions in the brain. Their work has demonstrated that this incongruency effect can be seen in the neural network of infants as young as 8-months-old, suggesting that this bias is part of our development.
Thirty-two eight-month old babies were shown avatars displaying either a fearful or happy emotional body expression. The avatars were followed either by a happy or fearful facial expression. Like previous body expression studies, there was a twist – some avatars were followed by incongruent facial expressions. This resulted in four possible combinations in which the emotional displays could have been presented: a happy body expression followed by a happy face, a fearful body expression followed by a fearful face, a happy body expression followed by a fearful face, and a fearful body expression followed by a happy face. The first two displays show congruence, meaning that the display of body and facial expressions match up in the emotion they are trying to express, and the latter two displays show incongruence. To measure the infant’s brain activity during these presentations, the researchers attached electrodes to an elastic cap placed around the infants’ heads, and used electroencephalography methods to record activation of brain regions (Rajhans et al., 2016).
The procedures Rajhans and colleagues used to get to their results are rather complex, but nonetheless their work illuminates some important information regarding how body expression influences the processing of facial expressions, especially when incongruent. They explain that “viewing conflicting body expressions hampers the neural discrimination of facial expressions in infants…suggest[ing] that early in development body expressions provide an important context in which facial expressions are processed” (Rajhans et al., 2016, p.119).
Thus, these converging lines of research tell us that one’s body movements can influence our perceptions of one’s emotions, especially if the body expression does not match the facial expression. With this knowledge, I now understand why I was heavily influenced by Ally’s slouchy and passive body posture. Nonetheless, I still do not know if my interpretation of her emotions were correct. What could have Ally done to make it easier for myself and Sally to detect her true emotions?