RIT Computer Science Capstone Report 20175
The Analysis of Emotional Suppression and
Cognitive Reappraisal
Nithin Reddy, Ifeoma Nwogu
Department of Computer Science
Golisano College of Computing and Information Sciences
Rochester Institute of Technology
Rochester, NY 14586
nk1214@cs.rit.edu, ion@cs.rit.edu
Abstract—Emotion Regulation is just one of the many activities,
we as humans perform to interact amicably amongst
ourselves. The attempts we do to influence our emotions and
control how we express and experience the emotions can be
categorized under emotion regulation. This study aims to analyze
two particular emotion regulation strategies, namely expressive
suppression and cognitive reappraisal. Firstly the data collection
process is explained in detail, along with the stimuli selection
process and general set up for each sensor equipment. We aim
to understand the activity within the brain while it performs
the above mentioned strategies by recording EEG signals along
with the galvanic skin response to more closely observe the
physiological manifestations. The study is augmented with the
facial expressions collected using a webcam to better analyze the
different states.
Index Terms—Emotion Suppression; Cognitive Reappraisal;
EEG; GSR; Facial Expressions; EEG Representations
I. INTRODUCTION
Emotions can be viewed as a set of coordinated responses
following an interaction between an organism and the
environment. As humans, it is very crucial to have a control
over our emotions for successful adaptive functioning.
Emotion Regulation is when an individual tries to subdue
his emotions and controls how he wishes to express them.
There are several strategies adopted by individuals to regulate
their emotions, the two most widely adopted strategies are
Cognitive Reappraisal and Expressive Suppression.
Cognitive Reappraisal is when we interpret a particular
stimuli from a different perspective than usual and potentially
alter the emotional impact it has on us. A study shows that
participants who witnessed an upsetting surgical procedure
had lower physiological responses when the process was
explained analytically. Expressive Suppression is inhibiting
the emotion felt by not expressing it outwardly. It has been
observed that reappraisal reduces the emotion experience
while suppression does not necessarily alter the emotions
experienced [1].
In this project we try to prove this theory backed with
the data obtained from a more scientific and systematically
designed experiment. We setup an environment where we
try to elicit two emotions namely, amusement and disgust,
and collect data from the participant who would be equipped
with an EEG headset and GSR equipment strapped onto their
hands. The participant would be informed prior to watching
a video to either suppress their emotions or reappraise them.
The data collected is preprocessed and analyzed to support
the theory. The rest of the paper is followed by the detailed
explanation of the environmental setup for the experiment,
detailed description of the EEG and GSR equipment, and the
data those equipments collected along with their preprocessing
methodologies.
II. EXPERIMENT SETUP
A. Equipment and Setup
The experiment was performed in a well light room
with controlled illumination. A 16 electrode Emotiv Epoc
Plus device was used to collect the EEG signals and a B4
Shimmer was used to collect the Galvanic Skin Responses.
The participant would be facing the monitor, where the video
stimulus would be presented and the participant’s frontal face
reactions would be recorded using a consumer grade web
cam corder as shown in Figure 1.
The facial features and the GSR data were collected through
the Imotions software which would synchronize the data
automatically, while the EEG data was collected from the
Emotiv Pro software to enable us access to the raw data the
EEG equipment collected.
Fig. 1. The GSR and EEG equipment
Rochester Institute of Technology 1 j P a g e
RIT Computer Science Capstone Report 20175
B. Stimuli Selection
The purpose of the stimuli is to elicit the emotions of
amusement and disgust in the participant. The selection was
a very critical process, as the stimuli selected should be a
video with which the participants shouldn’t be familiar and
also elicits the desired emotions. Several amusing and disgust
videos were initially considered and shown to several people
who would not be a part of the data collection process.
Amongst the initial videos, we selected the those which had
better ratings.
A total of 6 videos were selected, three of them which
would bring out amusement in the viewer and three which
would evoke disgust. The final stimulus which would be used
in the experiment has these 6 one minute videos separated
with 30 second calming videos. This was done to ensure two
things, one so that we ensured we had a good separation
between the signals for different emotional states and two to
allow the sensors to calibrate with the user. The link to the
list of considered video and the final stimulus are provided in
the appendix.
C. Experiment Protocol
The data was collected from a total of 30 participants ( 15
male , 15 female) who were within the age group of 17 –
30. Prior to the experiment, the participant would be clearly
explained about the experiment and would be asked to sign
the consent form. Next, they would be given instructions
regarding the experiment protocol. They were asked to always
look at the video and not to take their eyes away from the
screen. After we position the participant to get maximum
frames in the webcam, we equip the participant with both
the sensors. The paper will talk about the sensor placement
later while we discuss each sensor individually. Once the
participant is confident about the process, the experiment is
started when the participant clicks the record button.
The stimulus starts with a calming video which we would
use as a baseline. Following the calming part in the video, is
the first disgust video. The participants are informed within
the video that they are allowed to freely express their emotion
for this part. This is then followed by a calming video and
the instructions that inform the participants to suppress their
emotion for the second disgust video. This is then followed
by another calming video with instructions to inform the user
to think about the video in such a way that their perception
would negate the emotion they experience. This entire process
is repeated again for the set of amusing videos as well.
Fig. 2. A cycle of the stimuli
III. GALVANIC SKIN RESPONSE
Our body has about three million sweat glands. The density
of sweat glands varies markedly across the body, the highest
being on the forehead and cheeks, the palms and fingers
as well as on the sole of the feet. Whenever sweat glands
are triggered and become more active, they secrete moisture
through pores towards the skin surface. By changing the
balance of positive and negative ions in the secreted fluid,
electrical current flows more readily, resulting in measurable
changes in skin conductance (increased skin conductance =
decreased skin resistance). This change in skin conductance
is generally termed Galvanic Skin Response (GSR).
The reason why we have chosen to take the GSR readings
because it enables us to measure behavior which is not
under our conscious control. The skin conductivity, the GSR
equipment measures, is maintained by our body at an entirely
subconscious level, which is exactly what makes it a perfect
choice for the measurement of emotional arousal. Combined
with the EEG data, we can look into both the physiological
and psychological responses of a participant. Sweat secretion
is completely under the control of the autonomic nervous
system. We can see the different activities under the autonomic
nervous system in Figure 3.
Fig. 3. The autonomic nervous system
A. GSR Sensors and Placement
The working of a GSR equipment is quite simple. The GSR
measurements can be recorded non-invasively by placing
two electrodes on the skin with very little preperation time.
Participants usually are quite comfortable with a GSR when
compared to other sensors like the EEG – as their setup is
more tedious and time consuming. A GSR sensor is usually
a site of area 1 cm2 made up of a mixture of either silver or
silver chloride which are attached to a velcro. As we can see
in Figure 1, a third sensor is also present in our equipment
which allows us to measure the heart beat of the user at a
similar sampling rate.
It has been observed that the density of the sweat glands
are very high in the hand palms, foot soles and fingers
in a human body, making them favorable locations for
the GSR sensor placement. For this experiment, the GSR
Rochester Institute of Technology 2 j P a g e
RIT Computer Science Capstone Report 20175
sensors were strapped to the non dominant hand of the
participant allowing the dominant hand to be used for other
purposes. The sensor placement as shown in Figure 3, the two
sensors are placed in Positions A and B. The third heart beat
sensor was placed on the ring finger of the non dominant hand.
Fig. 4. The GSR sensor placement
B. Raw GSR Data
We can analyze a raw GSR signal using two parameter,
namely SCL and SCR.
Skin Conductance Level (SCL)
The tonic level, known as skin conductance level (SCL),
is the component which varies very gradually within
seconds. The rise and fall of the SCL depends on an
individual participant’s hydration, skin dryness and autonomic
regulation. The SCL is usually different for different
individuals and hence it is usually not considered to be a
useful metric.
Skin Conductance Response (SCR)
The phasic response is usually the SCR, and it is the layer
on top of the tonic leve. The SCR shows comparatively faster
alterations and the variations are called the ”GSR Peaks”. The
SCR is sensitive to emotional stimulus and can be usually
seen after a lag period of 1-5 seconds. This part of the GSR
is what will be used for analysis.
C. Preprocessing of GSR Data
The initial preprocessing for the GSR raw data includes
down sampling the data. For our data we decided to retain all
the samples because we could afford the computation, but it
Fig. 5. The GSR raw data components
is important to note that down sampling does not cause much
information loss. This step is done because the GSR data is
more often than not sampled at rates much higher than what
is required.The raw GSR signals for three different users have
been shown in Figure 4. We can clearly observe that different
users react very differently to the presented stimuli.
Fig. 6. The GSR raw signal for three different users
Filtering
The next step would be filtering the data. Filtering the raw
data is necessary to remove the tonic level of the data, as this
does not tell us anything about the arousal caused due to the
stimulus. We pass the GSR raw signal data through a median
filter to remove the tonic part of the data and keep only the
smoothened GSR signal containing only the phasic response.
We apply a median filter by going through the entire dataset
sample by sample and subtracting the median computer over
a time window of 4 seconds above and below that sample.
The resulting signal can be observed in Figure 5.
Peak Detection
The next step would be to detect peaks in the signal. Peak
detection is a very step in the analysis part. To understand
peak detection it is very important to know a few other
terminologies. Each GSR signal has a latency, this is the
Rochester Institute of Technology 3 j P a g e
RIT Computer Science Capstone Report 20175
Fig. 7. The GSR Median Filtered signal for three different users
gap between the presentation of the stimuli and the onset of
the response. Typically usually comes 1-5 seconds after the
stimuli has been presented. Onset is the voltage at which the
GSR rapidly rises to reach its peak amplitude. Rise Time is
the time taken to reach the peak amplitude from its onset.
Recovery Time is the time the signal takes to reach back to
onset from its peak amplitude. It was observed that this signal
behaves characteristically like the discharge of a capacitance.
The recovery times are usually longer than the rise times.
Hence for peak detection, we find the peak onsets and their
subsequent offsets and calculate the number of peaks in a 2
second window.
D. Feature Extractions from the GSR Signal
We extract a number of features for a GSR signal. We
first divide the entire dataset into four parts based on the
stimuli presented. We slide a two second window over the
GSR signals with an overlapping window of 1 second to
computer the number of peaks, Maximum Peak amplitude,
Minimum Peak Amplitude, Standard Deviation, Variance and
the mean of the GSR amplitudes. Int total for every two
second window, we compute 6 features.
IV. ELECTROENCEPHALOGRAPHY – EEG
Brain is the most complex working system in the world. It
carries out several daily task almost without us even realizing.
The curiosity to try and understand the working of the brain
has led to the development of Electroencephalography, the
brain imaging technique. EEG, quite literally, means the
writings of the electrical activity within the brain.
EEG has been around for quite some time now. Several
sophisticated and advanced EEG’s have been developed
which use anywhere between 10 – 500 electrodes to measure
the activity from the scalps. But the reason to select an EEG
among the various other instruments is because:
EEG signals record the neural activity with a very high
time resolution. The several activities our brain happens
faster than milliseconds, but an EEG can record the
electrical activities caused by these physiological actions.
As mentioned, every activity our brain performs, is a
cause of the electrical synapses between the neurons. So
the EEG can quite literally map the activity as it happens
within the brain.
The EEG equipment is cheap compared to the other
equipements, lightweight and be easily transported from
one place to another.
EEG is the only equipment which can measure very
subtle proccess the brain does, like inhibition or
meditiation.
The neuron emits a small undetectable electrical activity
when seen as a stand alone. But when your performs a task,
billions of such neurons fire which allows us to measure the
resultant electrical activity. The firing of the neurons have a
very complex pattern which can be classified and attributed to
different activities the brain performs. The diiferent frequency
bands the EEG can detect are:
Delta Band (1 – 4 Hz)
At this band we only see the slow and large amplitude
signals. It is usually found during the non-REM sleep.
Theta Band (4 – 8 Hz)
Anythin related to brain processing is usually found in the
theta band. Increasing task difficulty or memory workload
can result in a more dominant signal.
Alpha Band (8 – 12 Hz)
This band is usually seen for any sensory, memory
or motor related activities. It is quite prominent when
recorded while relaxing with eyes closed.
Beta Band (12 – 25 Hz)
Any thinking or planned physical movements is reflected
quite strongly in this band. The band is more visible
when there are physical body movements or thinking or
concentrating.
Gamma Band (25 and higher Hz)
The gamma band frequencies are kind of a gray
area. Researches havent fully understood what these
frequencies reflect or the source which causes them.
A. EEG Data Collection
EEG data collection is a very delicate process. The
participants were instructed to wash and dry their hair before
arriving for their data collection experiment. The electrodes
were wetted using a saline solution for establishing a clean
connection with the scalp. It was made sure that the Emotiv
Headset displayed 100% contact before the experiment was
started.
Rochester Institute of Technology 4 j P a g e
RIT Computer Science Capstone Report 20175
Electrode Placement
For this experiment, the 10-20 electrode placement system
was adopted. The 10-20 system requires the sensors be
arranged at 10% and 20% points on the lines lines of the
latitude and longitude.
To understand this system, we need to first know a few
points of references.
1) Nasion (Nz):
This is the point between the eyes just at the beginning
of the nose.
2) Inio (Iz):
This is the area at the back of a head where there is a
slight bump
3) Preauricular Points:
These are the points behind your ears where the jaw
is connected and you can feel the jaw movement with
your fingers when you open and close your mouth.
The 10 – 20 system is a well accepted electrode placement
system which gives a rough approximation of where the
sensors should be placed. The position are usually a letter
or two followed by a number. The letter indicates which
brain region the sensor would be placed on and the number
indicates the distance from the midline. The brain regions are
Fp = front polar, F = Frontal, C = Central, P = parietal, O
= occipital and T = temporal. The odd numbers are usually
placed on the left side of the brain while the even numbers
are placed on the right side of the brain. For example, F3 is
placed on the left side of the frontal brain region, and T3 is
placed on the left temporal region.
B. EEG Artifacts
Our brain processes a lot of activities simultaneously. It
is only natural that we see all these activities in our EEG
recordings. In order to get the best analysis, it is very important
that we get rid of the major known artifacts in the brain signals.
Some of the most common and frequently occurring artifacts
are as follows:
1) Any movement of our body is recorded in the EEG data.
The closer the part to the brain, the higher the impact it
has on the data.
2) Moving the eye up and down also affects the EEG
signal. The signals are either sinusoidal or box shaped
depending on the type of eye movement.
3) Just like the eye motion, even the blinking of the eye
causes disruptions in the EEG signal.
4) Adjusting the headset during the experiment can have
disastrous affect on the EEG signals. Sometimes if the
electrode region has been shifted we might end up
collecting wrongly placed data.
5) Line noise is one of the most commonly occurring type
of artifact in the signal.
6) It is also very important that the participant is advised
not too move their head too much. It is observed that
this causes quite a disruption in the EEG signal and
reason is believed to be that the head movements affect
the water distribution within the brain and that might
have a hand in affecting the electrical properties.[1]
C. EEG Data Cleaning
The EEG Data cleaning was kept to a minimum. The
paper focuses on using a unique representation of the EEG
data which will be discussed in the later sections. The
basic preprocessing steps taken have been explained in
the following subsection in the order in which they were
performed:
1) Bad Channel Removal: After importing the data the
first thing to do is to have a look at the plot manually. Some
channels have data that are either all zero or completely
garbage. It is better to remove these channels as they might
affect the ICA (explained later).
2) Band Pass Filtering: For our analysis we have cut off
the signal at a lower boundary of 1Hz and an upper boundary
of 40Hz. We were mostly interested in the alpha, beta and
theta bands and hence keeping the other frequencies would
not be useful.
3) Epoch the Data: It is very important to know when the
stimuli is presented and accordingly label the data for that
stimuli. We use a 3 second epoch with a 2 second overlapping
window. 3 seconds is usually considered a long duration but
we see that it works best in our case.
4) Independent Component Analysis: We use ICA mainly
for artifact removal. For intuitive purposes, think of our EEG
signal as a mixture of the signals which are caused by our
stimuli, breathing, eye blinking, muscle movement and line
noise. Using ICA the signal into several different signals
each representing one of the previously mentioned tasks. We
will remove the most commonly observed components like
eye blinking and muscle movement and recombine the other
components to recreate the signal.
We use the FastICA method to separate the components.
The two main steps in ICA would be data whitening
and component extraction . Data whitening usually means
removing the correlations between the channels and force the
channels to be uncorrelated. Extracting the components is
usually achieved by rotating the data and trying to minimize
the Gaussian properties in the projections.
5) Fast Fourier Transform: FFT is basically used to
transform the data from time domain to frequency domain.
As we mentioned previously, the billions of neurons fire at a
particular rhythm while performing an activity. We hope to
Rochester Institute of Technology 5 j P a g e
RIT Computer Science Capstone Report 20175
see which frequencies have the highest power spectrum for
the different emotional states. We particularly analyze just
three bands namely, theta (4 – 8 Hz), alpha (8 – 12 Hz) and
beta (12 – 25 Hz) bands.
D. Learning Representations
It is a common procedure to consider the EEG signals over
a period of time and compute different aggregations and use
them as feature vectors for the classifier. While this seemed
to work just fine, this paper looks at another way which
will even take in to consideration the topology of the EEG
electrodes by storing the features in multi dimensional tensors.
The EEG data we collect, is a time series data aggregated
from all the 14 electrodes placed over the participants head.
FFT is usually performed to study the underlying frequency
responses of the neurons collectively for a particular stimuli.
Instead of creating aggregations, which might miss out on
some key time series features, this method tries to create a
movie which shows how the brain responds to a stimuli with
respect to the frequency spectrum. The first step is to collect
the 3D coordinates of the electrodes and convert them to 2D
coordinates but maintain the relative distances between the
different electrode locations. This is done using the Azimuthal
Equidistant Projection (AEP). Once we have all the electrode
locations in place, we calculate the frequency response for a
particular time window. Since we are considering only three
frequency bands we will have three topography activity maps
which can be combined to form the RGB bands for our images.
The idea behind this approach is to convert the normal
EEG data into an intuitively put movie kind of data to
feed possibly into a recurrent neural network to learn the
evolutions and response to different stimuli.
V. CONCLUSION
The conclusion goes here.
ACKNOWLEDGMENT
The authors would like to thank…
References
[1] S. Zhang, C. Zhu, J. K. O. Sin, and P. K. T. Mok, “A novel ultrathin
elevated channel low-temperature poly-Si TFT,” IEEE Electron Device
Lett., vol. 20, pp. 569–571, Nov. 1999.
Rochester Institute of Technology 6 j P a g e