Abstract
Although working memory (WM) is part of many cognitive functions, its mechanism is still elusive. A popular way to assess WM function is using the N-Back task: subjects are asked to report whether the current stimulus matches one presented N stimuli earlier. While the basic structure of the N-back task is consistent across literature, the type of stimulus used and task features vary considerably in terms of duration, ISI, feedback, and response contingencies. Little is known about the effect of these variations on the neural and behavioral correlates of WM. To investigate the possible effect of these factors on the neural and behavioral outcomes, 9 healthy UCR undergraduates were recruited to perform different versions of the N-back task while simultaneously recording their EEGs: Three different experimental set-ups with distinct stimulus durations, ISI, feedback response contingencies, and three different stimuli types (colored circles, pictures, and syllables), crossed into nine different task/stimuli combinations. We also compared the P300 event-related potential (ERP) component with previous results from a similar picture N-back task and a syllable N-back task with different lab settings. Preliminary results suggest that neural signatures and behavioral data differ for different versions of the N-Back task. Within each set-up, task type seems to play an independent role in modulating the P300 morphology, accuracy and reaction time (RT). These results show that care must be taken when comparing N-back study outcomes particularly when it comes to task paradigms.
Keywords: Working memory, N-back, P300, Task/ Stimulus differences
Introduction
Working memory (WM) is a cognitive construct that enables storage and manipulation of a limited amount of information for a brief period of time. The fact that WM acts as an interface between perception, long-term memory, and action, makes it a crucial cognitive ability that supports thought processes, learning, problem-solving and coping with daily life. Therefore, WM interventions have become increasingly popular for patients as well as healthy adults (Baddeley, 1992; Baddeley 2003; Miyake and Shah, 1999; Zheng et al., 2012). Although there are various methods to engage and investigate WM, one of the frequently used measures in the literature is the N-back task (Owen et al., 2005; Jaeggi et al., 2008). In a sequential stimulation scenario, participants are asked to decide if the stimulus they are presented with matches the one “N” trials back. For example, in a 2-back task, participants should continuously maintain the last 2 items in the sequence, updating the memory when new items emerge and drop out the least recent one.
In recent years, variants of N-back task along with classical behavioral measures and non-invasive methods such as electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) have been used in measuring and training WM performance (Brouwer et al., 2012; Owen et al., 2005; Esposito et al., 2009). These studies have used different task features, such as stimulus duration, inter stimulus interval (ISI), stimulus type, task load and response contingencies. Additionally, they have used distinct individual (participant) features such as adaptive procedures, providing feedback, including motivational and enjoyment factors in the task and controlling subjects’ strategy selection (Mohammad et al., 2017). Despites this frequent use of N-back task, much is unknown about how using variations of the task could affect neural and behavioral outcomes and yet we encounter comparisons of results across studies (often lumped together in meta-analysis) without any attention to potential role of the differences. In this study, we first discuss the role of each task feature on behavioral and neural measures and then by introducing our task and comparing our results of p300 to two previous studies, we will try to shed more light on this issue.
Load Effect
One merit of N-back task over other WM measures such as memory span tasks is that it affords convenient manipulation of load and task difficulty by manipulating the value of N since load of information maintenance and manipulation increase systematically with increasing the value of the N (Conway et al., 2005; Conway, Kane, & Engle, 2003). It has been suggested that increase of task difficulty is expressed in electrophysiological and behavioral measures of subject’s performance. Therefore, resulting in higher reaction time (RT) and lower accuracy. Besides this negative correlation between accuracy and RT, Jaeggi et al. (2010) reported several dissociations between RT and accuracy. In other words, they found that in their visuospatial N-back task RT but not accuracy was associated with reading span and digit span performance. Also, in their visuospatial, auditory and dual n-back task, accuracy but not RT was associated with higher fluid intelligence measures.
Moreover, EEG studies seem to confirm effect of cognitive load since P300 amplitude decreased with increased task difficulty (Pratt et al., 2011; Watter et al., 2001). Also, frontal midline theta rhythm (4–7 Hz) increases and posterior alpha band power (7.5–12 Hz) decreases in magnitude as memory load increases (Schmidt et al., 2009; Miller et al., 2009; Chen et al., 2008; Scharinger et al., 2015; Lei and Roetting, 2011). Neuroimaging studies provide further supporting evidence by showing activation increments in cortical areas associated with the increased processing load (Drobyshevsky, Baumann, & Schneider, 2006).
Stimulus Type
Another common manipulation in cognitive tasks is changing the stimulus type. An additional appeal of the N-back task is that it can be used equally well with different kinds of stimuli. The original N-back task that was proposed by Kirchner (1958) utilized visuo-spatial stimuli with four different memory loads. Later Mackworth (1959) used a visual letter version with six N-back loads. Across studies of N-back tasks, numerous distinct types of stimuli have been used by implementing different input modalities (i.e. visual, auditory) and within each modality many different stimuli have been utilized such as letters, words, numbers, colors, locations and pictures (Burki et al., 2014; Clark et al., 2017; Jaeggi et al., 2008; Jaeggi et al., 2010; Kundu et al., 2013).
Christensen & Wright (2010) assessed effect of stimuli with varying linguistic load in neurologically intact and individuals with aphasia. They used three different N-back tasks with various linguistic loads as the extent to which a stimulus could rapidly be verbalized. The stimuli included pictures of fruits as the category that is easiest to name, single color fribbles (objects similar to real-world stimuli that subjects do not have previous experience with) as novel objects which believed to have semi-linguistic effect and single color blocks with no linguistic load. Their results suggested that participants’ performance (response accuracy) in both groups were significantly higher when the N-back stimuli had higher linguistic load compared to semi-linguistic and non-linguistic stimuli. However, they did not report any reaction time results as an index of processing time, since participants were not required to respond with any rapidity.
In a fMRI study, Nystrom et al (1999) used three N-back experiments with different stimulus types, contrasting the effect of verbal and spatial stimuli. In this study letters were used as verbal and non-spatial stimuli, locations were used as nonverbal and spatial stimuli and finally abstract shapes were used as non-verbal and non-spatial stimuli. Contrasting shape and letter conditions showed that response time did not significantly differ between these two stimuli types. However, subjects made more overall errors with shapes compared to letters. Similarly, in the letter-location contrast there did not find any significant reaction time differences, however, error rates were higher with locations than with letters. In case of shape-location contrast, reaction time for shape were slower than RT for location, however subjects’ error rates did not significantly differ between shapes and locations. This illustrates that different stimulus types can yield different behavioral results.
Kennedy et al. (2014) used pictures and words with three levels of emotionality (positive, negative and neutral). According to their results, P300 amplitude for picture stimuli was higher in all three emotional states compared to the corresponding word stimuli.
Stimulus Duration
Another factor in experimental design of N-back tasks is timing of stimulus presentation (duration). By using a forced-choice recognition task with a 4×4 matrix stimulus Adamowicz (1976) showed that memory performance improves as stimulus duration is increased. In his experiment, he investigated this effect on young and advanced-age groups by changing the stimulus duration from 3s to 6s to 12s and found that recall accuracy increased in both age groups. Furthermore, in 12-s condition no significant difference was seen between young and old groups, even though generally speaking it takes older individual longer to encode visual information. In a recent study Kunimi (2016) examined the effect of the same phenomenon on visuospatial information retention by increasing stimulus presentation period from 500ms to 5000ms (500ms, 1000ms,2000ms and 5000ms). He used a 4×4 non-verbal matrix to prevent phonological rehearsal and found that increasing the stimulus duration improves memory performance of advanced-age group relative to younger individuals. Correspondingly, lowest and highest memory performance between the age groups were found in the shortest and longest stimulus duration. In addition, performance discrepancy between age groups decreased as the stimulus duration increased. Also, Froeberg (1907) observed that visual stimuli that are longer in duration elicited faster RT. Welford (1980) and Hsieh (2002) showed that RT was faster where there were similar stimuli presented together than when different types of stimuli appeared in mixed order.
Although, no studies have been conducted to evaluate the effect of visual stimulus duration on P300, evidence in auditory tasks exist that increasing stimulus duration increases P300 amplitude and decreases its latency (Polich, 1989; Neshige et al., 1988).
Inter Stimulus Interval (ISI)
Inter Stimulus Interval (ISI) has been considered as experimental design feature of the N-back task. This factor affects the brain response, in particular ERP signatures using P300 component. Struber & Polich (2002) showed that P300 magnitude is affected by ISI. Relatively short ISI (2-3 s) have a smaller P300 amplitude that those obtained with longer ISI (6-10 s). The authors interpreted these findings as results of ‘recovery cycle’ that is strong enough to attenuate or eliminate target stimulus probability and task difficulty influences on P300 amplitude, probably because increased ISI facilitates the allocation of attentional resources to task performance when the ISI is 4-10 s longer. Similar findings have been shown by Bandettini & Cox (2000) using functional magnetic resonance imaging (fMRI) revealing that a shorter ISI (less than 8 seconds) is not sufficiently for the hemodynamic response to fully return to baseline compared to a longer ISI (around 8 seconds) that showed a stable pattern. Luck (2005), indicated that because of overlapping neural activity between previous and subsequent stimuli, distortion of event-related components could happen and suggested using a between stimuli temporal jitter.
Response Type
Other factor that could be manipulated in N-back task is response contingencies. In most studies participants are asked to respond to targets (relevant stimulus) with pressing a button and not to respond to non-targets (distractor stimulus). However, in several studies participants are asked to respond to both targets and non-targets by pressing two different buttons (Perlstein et al., 2003; Harvey et al., 2005; Miller et al., 2009). Salisbury et al. (2001) investigated the effect of button press with an oddball task. During the experiment participants were required to either press a button when encountered the oddball or count silently. Results showed that P300 amplitude was smaller in button-pressing task compared to silent-counting.
Feedback
N-back tasks usually include written or auditory feedback to inform the participants about their performance. The feedback could be immediate appearing after each response or delayed (appearing at the end of the block). Studies may provide feedback only during practice trials, only during experiment trials or both. It has been shown that feedback can affect behavioral and brain activity responses of individuals that are performing a cognitive training. Schiebener & Brand (2015) discussed that individuals that received feedback about the consequences of their actions were affected in their decision-making process and used this feedback to adapt, revise or verify their strategies accordingly. Furthermore, feedback (reward/punishment) produced a situation in which subjects adjusted their speed and accuracy to optimize their rewards (Simen, Buck, Holmes, Hu, & Cohen, 2009). Moreover, feedback can be positive or negative. In one hand, it has been shown that negative feedback is most effective when used to increase the performance by emphasizing the difference between the potential goal and the real performance (Cianci, Klein, & Seijts, 2010). In the other hand, positive feedback can have positive effects since it provides participant satisfaction, motivation and sense of competence. (Henderlong & Lepper, 2002). It has been shown that positive feedback yield larger P300 compared to negative feedback. Also, feedback with more informational value elicit larger P300 (Johnson and Donchin, 1985; Johnson and Donchin, 1978).
In this comparative study, we aim to shed more light on effects of N-back task design features, namely stimulus type, stimulus duration, ISI, response contingencies and feedback on neural and behavioral correlates by combining three different experimental set-ups (stimulus durations, ISI, feedback response contingencies and three different stimuli types (colored circles, pictures, and syllables). Each subject performed one of this nine combination in two consecutive days and EEG data was recorded. Neural measures (P300 latency and amplitude) and behavioral measures (RT and accuracy) were used to examine the outcomes. Moreover, we compared the neural results to two similar studies (picture and syllable N-back task) with matching task paradigms.
Materials and Methods
Participants
Nine (aged 19–24, female) healthy, students from university of California Riverside (UCR) performed the experiment in two consecutive days. All participants had normal or corrected-to-normal vision and did not have a history of neurological or psychological disorder. The experimental protocol was approved by the Institutional Review Board of UCR, and all participants gave informed consent prior to beginning the experiment. They earned credits toward their course plus were paid $10 for their participation.
Task and Procedure
For this study, we used three different versions of the N-back task with distinct stimulus duration, ISI, response contingency and feedback. Task1 had a stimulus duration of 400ms and ISI of 1600ms. In this task, participants were instructed to press one button for targets and another button for non-targets. Response registration was only possible during ISI in which participants were provided with a white fixation cross. In this task participants were only provided with response registration feedback. Upon button press, the color of the fixation cross changed to blue, indicating that the response was registered. In case of missed trials, the color of the fixation cross changed to red indicating that the participant should have responded.
Task2 had a stimulus duration 1000 ms and ISI of 2000 ms. Participants were instructed to press a button only in target trials. Similar to Task1, response registration was only possible during ISI and button press feedback was provided. Task3 had a stimulus duration of 2500ms and ISI of 500ms. In this version of the N-back task, participants were instructed to respond only to target trials and they could register the response during stimulus presentation. Correct/incorrect feedback in terms of green and red circles were provided.
Additionally, each task could present one of three different stimulus types: syllables (i.e. so, do, up), pictures (i.e. apple, fish, bag) and colored circles (i.e. red, green, blue). This comes up to 9 different conditions crossing the three task types with the three types of stimulation. Each participant performed two of these nine possible combinations as shown in Fig.1. The subjects completed two sessions and each session consisted of 11 blocks. Three of these blocks were used as practice: 1-back, 2-back and 3-back. During the experiment, only 2-back and 3-back blocks were used. Each experiment block consisted of N+40 trials (i.e. in 2-back we had 42 trials). The experiment was conducted using the Psychtoolbox extension (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997) as implemented in Matlab (Mathworks).
EEG Recording
Participants were tested in an electrically and sound-attenuated room. EEG was recorded using Biosemi Active Two system from 32 active electrode sites positioned according to the 10/20 system (Jasper, 1958). In addition, six external electrodes were used for mastoids and electro-oculogram (EOG).
Fig.1. Experimental design
Subjects were instructed to sit relaxed and follow task instruction that was implemented inside the experiment. Also, they were asked to minimize their body movements as well as eye movements and blinks. The total test time per day for each subject was approximately 90 min.
EEG Data Processing
Data processing was conducted using EEGLAB (Delorme and Makeig, 2004). The data was resampled to 512 Hz and filtered using a high-pass (0.1Hz) and low-pass (40 Hz) Butterworth filter. All electrodes re-referenced to average mastoid and manual inspection was first performed to locate and remove clearly visible disturbances in the data. Epochs were created from -1000 ms before to 2000 ms after each stimulus onset and pre-onset value was used to do the baseline correction. Independent components analysis (ICA) was used to extract out eye-blink and eye movements within the data. Finally, epochs were averaged for each task type and stimulus type.
Statistical Analysis
Behavioral data
We used a repeated measure ANOVA to investigate the effect of RT and accuracy with factor design of 3 ×3 as follows:
task type (task1, task2, task3) ×stimulus type (syllable, picture, colored circle). Only correct responses and corresponding times for target trials were used in the analysis.
Event-related potentials
Electrophysiological analyses examined P300 in the stimulus-locked waveform. Only correct target trials were used in the analysis. The P300 was identified as the large positive component that occurred between 250 and 600 ms in Pz electrode (Polich, 2007). The ERP mean amplitude and latency measures for P300 then were entered separately to a 3×3 ANOVA that examined task type (task1, task2, task3) ×stimulus type (syllable, picture, colored circle).
Results
Behavioral
For accuracy, a main effect of task type was found [F(2,27) = 16.42, p < 0.001; see Fig.2], showing that regardless of stimulus type accuracy in task 3 is higher than task1 and task2. No significant main effect of stimulus type was observed. Moreover, the interaction between task type and stimulus type was not significant.
Fig.2. Behavioral results. (A) Reaction times (B) Accuracy
For reaction times a main effect of task type was found [F(2,27) = 16.42, p < 0.001] showing the highest RT for task3 and lowest RT for task2. Again, no main effect of stimulus type or interaction effect was found. Although not significant, color circle stimulus seems to elicit the fastest RT and highest accuracy in each task compared to other stimuli. Also, in task1 we can see that RT decreases and accuracy increases for syllables, pictures and circles respectively.
Fig.3. Mean amplitude and latency results. Syllable, picture and color circles stimuli are shown in red, green and blue respectively.
Event-Related Potential
P300
There was only a marginal effect of stimulus type on P300 latency [F(2,27) = 2.936, p = 0.06; see Fig.3], however, neither task type or interaction of task and stimulus types showed any significant effect on latency. Also, no significant effect of task type or stimulus type (or interaction) was found on P300 amplitude. P300 component is shown in Fig.4. Although not significant, color circles seem to elicit the lowest P300 latency and pictures regardless of the task type.
Fig.4. P300 for Pz electrode. (A) Task1 (B) Task2 (C) Task3 for syllables, pictures and color circles from left to right respectively. Yellow window shows the interval that we used for measuring P300 mean amplitude and latency.
Fig.5 shows ERP results of two other studies with matched paradigms. Although both task type and stimulus type have a corresponding match in our experiment, it is obvious that P300 morphology is not quite the same.
Fig.5. ERP results for four participants. (A) Syllable N-back task (B) Picture N-back task with task paradigm that is matched to our paradigms of task1 and task2 respectively.
Discussion
The current study attempted to test the effect of task type and stimulus type on behavioral and neural results of N-back task. The results demonstrated that there is a significant task type effect on both RT and accuracy, indicating that Task 3 has the highest accuracy and highest reaction time among all three tasks. Task3 has the highest stimulus duration (2500 ms) and lowest ISI (500 ms) among all three tasks. Moreover, it requires only response to targets and provides the participant with the most useful information (correct/incorrect feedback). This result is in line with the previous finding that longer stimulus presentation will result in higher accuracy (Kunimi, 2016). There is no significant effect of stimulus type on behavioral performance, however, it seems that regardless of the task type, color circle stimuli elicit the highest accuracy and lowest RT compared to other stimuli. This finding challenges Christensen & Wright (2010) and the similar studies since we were expecting to have the highest performance for the stimuli with higher linguistic effect.
Although there is no significant effect of task type on P300 amplitude or latency, preliminary results show a similar trend between task 1 and task 3. Task1 has the lowest stimulus duration among all three tasks (400 ms) with an ISI of 1600 ms. This result suggests that neither feedback or button press has no observable effect on modulation of P300 amplitude or latency since the two task differ greatly in their response contingency and feedback type. Finally, stimulus type seems to have a marginal main effect on P300 latency, indicating that color circle stimulus elicits the shortest component latency, regardless of the task type.
Moreover, by comparing our results to ERPs of the two tasks that have identical task paradigms of task1 and task2, it seems that other factors besides task type and stimulus type could affect P300 morphology such as individual differences of participants, differences between labs in terms of used EEG systems, experimenters and the approach to collect the data.
References
Adamowicz, J. K. (1976). Visual short-term memory and aging. Journal of Gerontology, 31(1), 39-46.
Baddeley, A. (1992). Working memory. Science, 255(5044), 556-559.
Baddeley, A. (2003). Working memory: looking back and looking forward. Nature reviews neuroscience, 4(10), 829.
Bandettini, P. A., & Cox, R. W. (2000). Event‐related fMRI contrast when using constant interstimulus interval: theory and experiment. Magnetic Resonance in Medicine, 43(4), 540-548.
Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436.
Brouwer, A. M., Hogervorst, M. A., Van Erp, J. B., Heffelaar, T., Zimmerman, P. H., & Oostenveld, R. (2012). Estimating workload using EEG spectral power and ERPs in the n-back task. Journal of neural engineering, 9(4), 045008.
Bürki, C. N., Ludwig, C., Chicherio, C., & De Ribaupierre, A. (2014). Individual differences in cognitive plasticity: An investigation of training curves in younger and older adults. Psychological Research, 78(6), 821–835.
Chen, Y. N., Mitra, S., & Schlaghecken, F. (2008). Sub-processes of working memory in the N-back task: An investigation using ERPs. Clinical Neurophysiology, 119(7), 1546-1559.
Christensen, S. C., & Wright, H. H. (2010). Verbal and non-verbal working memory in aphasia: What three n-back tasks reveal. Aphasiology, 24(6-8), 752-762.
Cianci, A. M., Klein, H. J., & Seijts, G. H. (2010). The effect of negative feedback on tension and subsequent performance: The main and interactive effects of goal content and conscientiousness. Journal of Applied Psychology, 95, 618–630.
Clark, C. M., Lawlor-Savage, L., & Goghari, V. M. (2017). Functional brain activation associated with working memory training and transfer. Behavioural brain research, 334, 34-49.
Conway, A. R., Kane, M. J., & Engle, R. W. (2003). Working memory capacity and its relation to general intelligence. Trends in cognitive sciences, 7(12), 547-552.
Conway, A. R., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user’s guide. Psychonomic bulletin & review, 12(5), 769-786.
Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of neuroscience methods, 134(1), 9-21.
Drobyshevsky, A., Baumann, S. B., & Schneider, W. (2006). A rapid fMRI task battery for mapping of visual, motor, cognitive, and emotional function. Neuroimage, 31(2), 732-744.
Esposito, F., Aragri, A., Piccoli, T., Tedeschi, G., Goebel, R., & Di Salle, F. (2009). Distributed analysis of simultaneous EEG-fMRI time-series: modeling and interpretation issues. Magnetic resonance imaging, 27(8), 1120-1130.
Froeberg, S. (1907). The relation between the magnitude of stimulus and the time of reaction. Archives of Psychology, No. 8.
Harvey, P. O., Fossati, P., Pochon, J. B., Levy, R., LeBastard, G., Lehéricy, S., … & Dubois, B. (2005). Cognitive control and brain resources in major depression: an fMRI study using the n-back task. Neuroimage, 26(3), 860-869.
Henderlong, J., & Lepper, M. R. (2002). The effects of praise on children’s intrinsic motivation: A review and synthesis. Psychological Bulletin, 128, 774–795.
Hsieh, S. (2002). Tasking shifting in dual-task settings. Perceptual and Motor Skills 94(2): 407.
Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Perrig, W. J. (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences, 105(19), 6829-6833.
Jaeggi, S. M., Studer-Luethi, B., Buschkuehl, M., Su, Y. F., Jonides, J., & Perrig, W. J. (2010). The relationship between n-back performance and matrix reasoning—implications for training and transfer. Intelligence, 38(6), 625-635.
Jasper, H. H. (1958). The ten-twenty electrode system of the International Federation. Electroencephalogr. Clin. Neurophysiol.10, 371–375.
Johnson, R., & Donchin, E. (1978). On how P300 amplitude varies with the utility of the eliciting stimuli. Electroencephalography and Clinical Neurophysiology, 44(4), 424-437.
Johnson, R., & Donchin, E. (1985). Second thoughts: Multiple P300s elicited by a single stimulus. Psychophysiology, 22(2), 182-194.
Kennedy, L., Dorrance, S., Stoneham, T., Bryant, M., Boyd, K., Flippen, K., & Nichols, D. F. Event-Related Potentials in Humans for Emotional Words versus Pictures.
Kirchner, W. K. (1958). Age differences in short-term retention of rapidly changing information. Journal of experimental psychology, 55(4), 352.
Kleiner, M., Brainard, D., Pelli, D., Ingling, A., Murray, R., & Broussard,C.(2007).What’s new in psychtoolbox-3? Perception, 36, 1–1.
Kundu, B., Sutterer, D. W., Emrich, S. M., & Postle, B. R. (2013). Strengthened effective connectivity underlies transfer of working memory training to tests of short-term memory and attention. Journal of Neuroscience, 33(20), 8705–8715.
Kunimi, M. (2016). Effects of age, gender, and stimulus presentation period on visual short-term memory. Journal of women & aging, 28(1), 24-33.
Lei, S., & Roetting, M. (2011). Influence of task combination on EEG spectrum modulation for driver workload estimation. Human factors, 53(2), 168-179.
Luck, S. J. (2014). An introduction to the event-related potential technique. MIT press.
Mackworth, J. F. (1959). Paced memorizing in a continuous task. Journal of experimental psychology, 58(3), 206.
Miller, K. M., Price, C. C., Okun, M. S., Montijo, H., & Bowers, D. (2009). Is the n-back task a valid neuropsychological measure for assessing working memory? Archives of Clinical Neuropsychology, 24(7), 711-717.
Miyake, A., & Shah, P. (Eds.). (1999). Models of working memory: Mechanisms of active maintenance and executive control. Cambridge University Press.
Mohammed, S., Flores, L., Deveau, J., Hoffing, R. C., Phung, C., Parlett, C. M., … & Zordan, V. (2017). The benefits and challenges of implementing motivational features to boost cognitive training outcome. Journal of Cognitive Enhancement, 1(4), 491-507.
Neshige, R., Barrett, G., & Shibasaki, H. (1988). Auditory long latency event-related potentials in Alzheimer’s disease and multi-infarct dementia. Journal of Neurology, Neurosurgery & Psychiatry, 51(9), 1120-1125.
Nystrom, L. E., Braver, T. S., Sabb, F. W., Delgado, M. R., Noll, D. C., & Cohen, J. D. (2000). Working memory for letters, shapes, and locations: fMRI evidence against stimulus-based regional organization in human prefrontal cortex. Neuroimage, 11(5), 424-446.
Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005). N‐back working memory paradigm: A meta‐analysis of normative functional neuroimaging studies. Human brain mapping, 25(1), 46-59.
Perlstein, W. M., Dixit, N. K., Carter, C. S., Noll, D. C., & Cohen, J. D. (2003). Prefrontal cortex dysfunction mediates deficits in working memory and prepotent responding in schizophrenia. Biological psychiatry, 53(1), 25-38.
Polich, J. (1989). Frequency, intensity, and duration as determinants of P300 from auditory stimuli. Journal of clinical neurophysiology: official publication of the American Electroencephalographic Society, 6(3), 277-286.
Polich, J. (2007). Updating P300: an integrative theory of P3a and P3b. Clinical neurophysiology, 118(10), 2128-2148.
Pratt, N., Willoughby, A., & Swick, D. (2011). Effects of working memory load on visual selective attention: behavioral and electrophysiological evidence. Frontiers in human neuroscience, 5, 57.
Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442.
Salisbury, D. F., Rutherford, B., Shenton, M. E., & McCarley, R. W. (2001). Button-pressing affects P300 amplitude and scalp topography. Clinical Neurophysiology, 112(9), 1676-1684.
Scharinger, C., Soutschek, A., Schubert, T., & Gerjets, P. (2015). When flanker meets the n‐back: What EEG and pupil dilation data reveal about the interplay between the two central‐executive working memory functions inhibition and updating. Psychophysiology, 52(10), 1293-1304.
Schiebener, J., & Brand, M. (2015). Decision making under objective risk conditions–a review of cognitive and emotional correlates, strategies, feedback processing, and external influences. Neuropsychology review, 25(2), 171-198.
Schmidt, E. A., Schrauf, M., Simon, M., Fritzsche, M., Buchner, A., & Kincses, W. E. (2009). Drivers’ misjudgement of vigilance state during prolonged monotonous daytime driving. Accident Analysis & Prevention, 41(5), 1087-1093.
Simen, P. D. C., Buck, C., Holmes, P. Hu, P., and Cohen. J. D. (2009). Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions. Journal of Experimental Psychology: Human Perception and Performance 35(6): 1865-1898.
Strüber, D., & Polich, J. (2002). P300 and slow wave from oddball and single-stimulus visual tasks: inter-stimulus interval effects. International Journal of psychophysiology, 45(3), 187-196.
Watter, S., Geffen, G. M., & Geffen, L. B. (2001). The n-back as a dual-task: P300 morphology under divided attention. Psychophysiology, 38(6), 998-1003.
Welford, A. T. 1980. Choice reaction time: Basic concepts. In A. T. Welford (Ed.), Reaction Times. Academic Press, New York, pp. 73-128.
Zheng, X., Swanson, H. L., & Marcoulides, G. A. (2011). Working memory components as predictors of children’s mathematical word problem solving. Journal of experimental child psychology, 110(4), 481-498.