Test–retest reliability of cognitive EEG

doi:10.1016/S1388-2457(99)00258-8

Clinical Neurophysiology

Volume 111, Issue 3, 1 March 2000, Pages 457-463

https://doi.org/10.1016/S1388-2457(99)00258-8 Get rights and content

Abstract

Objective: Task-related EEG is sensitive to changes in cognitive state produced by increased task difficulty and by transient impairment. If task-related EEG has high test–retest reliability, it could be used as part of a clinical test to assess changes in cognitive function. The aim of this study was to determine the reliability of the EEG recorded during the performance of a working memory (WM) task and a psychomotor vigilance task (PVT).

Methods: EEG was recorded while subjects rested quietly and while they performed the tasks. Within session (test–retest interval of ∼1 h) and between session (test–retest interval of ∼7 days) reliability was calculated for four EEG components: frontal midline theta at Fz, posterior theta at Pz, and slow and fast alpha at Pz.

Results: Task-related EEG was highly reliable within and between sessions (r>0.9 for all components in WM task, and r>0.8 for all components in the PVT). Resting EEG also showed high reliability, although the magnitude of the correlation was somewhat smaller than that of the task-related EEG (r>0.7 for all 4 components).

Conclusions: These results suggest that under appropriate conditions, task-related EEG has sufficient retest reliability for use in assessing clinical changes in cognitive status.

Introduction

Certain components of the ongoing EEG, recorded while subjects perform a cognitive task, are sensitive to task difficulty and to changes in an individual's cognitive state. For example, frontal midline theta increases with increased task difficulty, whereas signals in the alpha band decrease as task difficulty increases (Gevins et al., 1979, Gevins et al., 1997, Gevins et al., 1998, McCallum et al., 1988, Miyata et al., 1990, Yamamoto and Matsuoka, 1990, Gundel and Wilson, 1992, Smith et al., 1999). Transient mental impairment, such as that associated with alcohol intoxication or fatigue also affects EEG signals. Acute alcohol intoxication increases the amplitude of slow alpha (Davis et al., 1941, Lukas et al., 1986, Cohen et al., 1993), whereas fatigue is associated with increased diffuse theta (Davis et al., 1937, Gevins et al., 1977, Matoušek and Petersen, 1983, Makeig and Jung, 1995) and decreased fast posterior alpha (Gevins et al., 1977, Gevins and Smith, 1999). These findings have led to the suggestion that task-related EEG measures could be used to assess temporary changes in cognitive status due to such factors as fatigue, intoxication, illness, injury, or drug consumption (Gevins and Smith, 1999).

A necessary first step in the development of an EEG-based method to assess changes in cognitive function is to demonstrate that such measures have high test–retest reliability. A number of studies have shown that the EEG is relatively stable when measured during resting states or during the performance of low load cognitive tasks, such as oddball stimulus detection. For example, in a young, healthy adult population, Burgress and Gruzelier (1993) reported average reliabilities of 0.81 and 0.86 for theta and alpha features in resting, eyes open EEG with a test–retest interval of about 1 h. Salinsky et al. (1991) reported high correlations (above 0.9) during the performance of an auditory oddball task with a test–retest interval of 5 min and somewhat lower correlations (of about 0.8) for an interval of 12–16 weeks. Pollock et al. (1991) found correlations between 0.8 and 0.84 for theta and alpha features in resting, eyes open and eyes closed data from a middle-aged population with a test–retest interval of 4.5 months.

The purpose of the current study was to assess the reliability of task-recorded EEG during the performance of tasks that could be used as part of a test battery to assess changes in cognitive state. Two tasks were used: a psychomotor vigilance task (PVT, Dinges and Powell, 1985) and a working memory (WM) task. The PVT requires subjects to remain vigilant and make a simple response to the onset of an infrequent stimulus. Performance in this task has been shown to be highly sensitive to lapses in alertness due to sleepiness (Lisper and Kjellerberg, 1972, Kribbs et al., 1993, Rosekind et al., 1994, Glenville et al., 1978, Dinges et al., 1987, Dinges et al., 1990, Kribbs and Dinges, 1994). The WM task requires subjects to attend to stimuli and make matching judgments on each stimulus relative to stimuli presented on previous trials. Two difficulty levels were used that differed in the amount of information subjects were required to remember. This task has been shown to induce changes in theta and alpha signals proportional to task-difficulty (Gevins et al., 1997, Gevins et al., 1998). For comparison purposes, we also assessed the reliability of EEG recorded while subjects rested quietly with their eyes open.

Section snippets

Subjects

Twenty healthy adults (18–29 years, mean age 22.9±3.3 years; 11 females) were paid to participate in the study. The use of human subjects in this study was approved by institutional review and all participation was fully informed and voluntary.

Tasks

Subjects performed two difficulty levels of a continuous performance WM task (Fig. 1) that required them to compare the spatial location of a current stimulus with the location of one presented previously (for full details, see Gevins et al., 1996).

WM task

Mean accuracy and reaction time (RT) in the WM task are shown in Table 1. Both accuracy (assessed using d′ scores) and RT (evaluated after normalizing with a log₁₀ transform) were affected by task difficulty. Responses were significantly faster and more accurate in the easy task than in the difficult task (d′: F(1,19)=12.62, P<0.01; RT: F(1,19)=34.08, P<0.001). Accuracy did not differ within or between sessions. RT showed a significant main effect of block (F(1,19)=5.07, P<0.05), and a

Discussion

This study showed that the power spectral density of four EEG components recorded during performance of PVT and WM tasks from alert, healthy young adults, well practiced in the task performance, was highly reliable both within and between sessions. Average within session reliability, in which the test–retest interval was approximately 1 h, was above 0.9 in the task data and above 0.8 in the resting data. Similar reliability was observed between sessions, in which the test–retest interval was

Acknowledgements

Supported by the National Aeronautics and Space Administration and the National Institute of Mental Health. We thank Devon Boyle and Caroline Prioleau for help with data collection and analyses. We are grateful to Dr. David Dinges for supplying us with the psychomotor vigilance task.

References (29)

H.L. Cohen et al.
The effects of ethanol on EEG activity in males at risk for alcoholism. Electroenceph clin Neurophysiol
(1993)
A.S. Gevins et al.
Computer rejection of EEG artifact
II: contamination by drowsiness. Electroenceph clin Neurophysiol
(1977)
A.S. Gevins et al.
High resolution evoked potential imaging of the cortical dynamics of human working memory. Electroenceph clin Neurophysiol
(1996)
S.E. Lukas et al.
EEG alpha activity increases during transient episodes of ethanol-induced euphoria
Pharmacol Biochem Behav
(1986)
M. Matoušek et al.
A method for assessing alertness fluctuations in vigilance and the EEG spectrum. Electroenceph clin Neurophysiol
(1983)
W.C. McCallum et al.
Brain slow potential and ERP changes associated with operator load in a visual tracking task
Electroenceph clin Neurophysiol
(1988)
V.E. Pollock et al.
Reliability of topographic quantitative EEG amplitude in healthy late-middle-aged and elderly subjects
Electroenceph clin Neurophysiol
(1991)
M.E. Smith et al.
Neurophysiological indices of strategy development and skill acquisition
Cogn Brain Res
(1999)
A. Burgress et al.
Individual reliability of amplitude distribution in topographical mapping of EEG
Electroenceph clin Neurophysiol
(1993)
H. Davis et al.
Human brain potentials during the onset of sleep
J Neurophysiol
(1937)

P.A. Davis et al.

The effects of alcohol upon the electroencephalogram (brain waves). Q J Stud Alc

(1941)

D.F. Dinges et al.

Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations

Behav Res Methods Instrum Comput

(1985)

D.F. Dinges et al.

Temporal placement of a nap for alertness: contributions of circadian phase and prior wakefulness

Sleep

(1987)

D.F. Dinges et al.

Fatigue-related reaction time performance in long-haul flight crews

Sleep Res

(1990)

Cited by (146)

Limitations of two time point data for understanding individual differences in longitudinal modeling — What can difference reveal about change?
2024, Developmental Cognitive Neuroscience
Emerging neuroimaging studies investigating changes in the brain aim to collect sufficient data points to examine trajectories of change across key developmental periods. Yet, current studies are often constrained by the number of time points available now. We demonstrate that these constraints should be taken seriously and that studies with two time points should focus on particular questions (e.g., group-level or intervention effects), while complex questions of individual differences and investigations into causes and consequences of those differences should be deferred until additional time points can be incorporated into models of change. We generated underlying longitudinal data and fit models with 2, 3, 4, and 5 time points across 1000 samples. While fixed effects could be recovered on average even with few time points, recovery of individual differences was particularly poor for the two time point model, correlating at r = 0.41 with the true individual parameters - meaning these scores share only 16.8% of variance As expected, models with more time points recovered the growth parameter more accurately; yet parameter recovery for the three time point model was still low, correlating around r = 0.57. We argue that preliminary analyses on early subsets of time points in longitudinal analyses should focus on these average or group-level effects and that individual difference questions should be addressed in samples that maximize the number of time points available. We conclude with recommendations for researchers using early time point models, including ideas for preregistration, careful interpretation of 2 time point results, and treating longitudinal analyses as dynamic, where early findings are updated as additional information becomes available.
Reliability and generalizability of neural speech tracking in younger and older adults
2024, Neurobiology of Aging
Neural tracking of spoken speech is considered a potential clinical biomarker for speech-processing difficulties, but the reliability of neural speech tracking is unclear. Here, younger and older adults listened to stories in two sessions while electroencephalography was recorded to investigate the reliability and generalizability of neural speech tracking. Speech tracking amplitude was larger for older than younger adults, consistent with an age-related loss of inhibition. The reliability of neural speech tracking was moderate (ICC ∼0.5–0.75) and tended to be higher for older adults. However, reliability was lower for speech tracking than for neural responses to noise bursts (ICC >0.8), which we used as a benchmark for maximum reliability. Neural speech tracking generalized moderately across different stories (ICC ∼0.5–0.6), which appeared greatest for audiobook-like stories spoken by the same person. Hence, a variety of stories could possibly be used for clinical assessments. Overall, the current data are important for developing a biomarker of speech processing but suggest that further work is needed to increase the reliability to meet clinical standards.
The effects of distraction on younger drivers: A neurophysiological perspective
2024, Applied Ergonomics
Distracted driving remains a significant cause of traffic accidents globally, including in Australia. However, many younger drivers still admit to using a phone while driving. A simulated driving study investigated the neurophysiological effects of visual, auditory, and higher-order cognitive (i.e., requiring the use of executive functions) distraction on young drivers. In total, 24 young adults aged 18–25 years completed four 8 min simulated driving sessions while concurrently engaging in various distractor tasks. Neurophysiological arousal was measured via EEG. Additionally, subjective workload and objective driving performance were assessed. Frontal beta and gamma power exhibited their highest levels during tasks involving higher-order cognitive and visual demands. The higher-order cognitive condition was rated as the most mentally demanding. In comparison, the visual condition had the most significant impact on both the standard deviation of speed and standard deviation of lateral positioning. This study has significant implications for all road users, particularly those aged 18–25 years, and it reinforces the importance of not using a phone while driving.
Wearable-based human flow experience recognition enhanced by transfer learning methods using emotion data
2023, Computers in Biology and Medicine
Flow experience is a specific positive and affective state that occurs when humans are completely absorbed in an activity and forget everything else. This state can lead to high performance, well-being, and productivity at work. Few studies have been conducted to determine the human flow experience using physiological wearable sensor devices. Other studies rely on self-reported data.
In this article, we use physiological data collected from 25 subjects with multimodal sensing devices, in particular the Empatica E4 wristband, the Emotiv Epoc X electroencephalography (EEG) headset, and the Biosignalplux RespiBAN – in arithmetic and reading tasks to automatically discriminate between flow and non-flow states using feature engineering and deep feature learning approaches. The most meaningful wearable device for flow detection is determined by comparing the performances of each device. We also investigate the connection between emotions and flow by testing transfer learning techniques involving an emotion recognition-related task on the source domain.
The EEG sensor modalities yielded the best performances with an accuracy of 64.97%, and a macro Averaged F1 (AF1) score of 64.95%. An accuracy of 73.63% and an AF1 score of 72.70% were obtained after fusing all sensor modalities from all devices. Additionally, our proposed transfer learning approach using emotional arousal classification on the DEAP dataset led to an increase in performances with an accuracy of 75.10% and an AF1 score of 74.92%.
The results of this study suggest that effective discrimination between flow and non-flow states is possible with multimodal sensor data. The success of transfer learning using the DEAP emotion dataset as a source domain indicates that emotions and flow are connected, and emotion recognition can be used as a latent task to enhance the performance of flow recognition.
Vigilant attention mediates the association between resting EEG alpha oscillations and word learning ability
2023, NeuroImage
Individuals exhibit considerable variability in their capacity to learn and retain new information, including novel vocabulary. Prior research has established the importance of vigilance and electroencephalogram (EEG) alpha rhythm in the learning process. However, the interplay between vigilant attention, EEG alpha oscillations, and an individual's word learning ability (WLA) remains elusive. To address this knowledge gap, here we conducted two experiments with a total of 140 young and middle-aged adults who underwent resting EEG recordings prior to completing a paired-associate word learning task and a psychomotor vigilance test (PVT). The results of both experiments consistently revealed significant positive correlations between WLA and resting EEG alpha oscillations in the occipital and frontal regions. Furthermore, the association between resting EEG alpha oscillations and WLA was mediated by vigilant attention, as measured by the PVT. These findings provide compelling evidence supporting the crucial role of vigilant attention in linking EEG alpha oscillations to an individual's learning ability.
Stability, change, and reliable individual differences in electroencephalography measures: A lifespan perspective on progress and opportunities
2023, NeuroImage
Electroencephalographic (EEG) methods have great potential to serve both basic and clinical science approaches to understand individual differences in human neural function. Importantly, the psychometric properties of EEG data, such as internal consistency and test-retest reliability, constrain their ability to differentiate individuals successfully. Rapid and recent technological and computational advancements in EEG research make it timely to revisit the topic of psychometric reliability in the context of individual difference analyses. Moreover, pediatric and clinical samples provide some of the most salient and urgent opportunities to apply individual difference approaches, but the changes these populations experience over time also provide unique challenges from a psychometric perspective. Here we take a developmental neuroscience perspective to consider progress and new opportunities for parsing the reliability and stability of individual differences in EEG measurements across the lifespan. We first conceptually map the different profiles of measurement reliability expected for different types of individual difference analyses over the lifespan. Next, we summarize and evaluate the state of the field's empirical knowledge and need for testing measurement reliability, both internal consistency and test-retest reliability, across EEG measures of power, event-related potentials, nonlinearity, and functional connectivity across ages. Finally, we highlight how standardized pre-processing software for EEG denoising and empirical metrics of individual data quality may be used to further improve EEG-based individual differences research moving forward. We also include recommendations and resources throughout that individual researchers can implement to improve the utility and reproducibility of individual differences analyses with EEG across the lifespan.

View all citing articles on Scopus

View full text

Test–retest reliability of cognitive EEG

Abstract

Introduction

Section snippets

Subjects

Tasks

WM task

Discussion

Acknowledgements

The effects of ethanol on EEG activity in males at risk for alcoholism. Electroenceph clin Neurophysiol

II: contamination by drowsiness. Electroenceph clin Neurophysiol

High resolution evoked potential imaging of the cortical dynamics of human working memory. Electroenceph clin Neurophysiol

Pharmacol Biochem Behav

A method for assessing alertness fluctuations in vigilance and the EEG spectrum. Electroenceph clin Neurophysiol

Electroenceph clin Neurophysiol

Electroenceph clin Neurophysiol

Cogn Brain Res

Individual reliability of amplitude distribution in topographical mapping of EEG

Electroenceph clin Neurophysiol

Human brain potentials during the onset of sleep

J Neurophysiol

The effects of alcohol upon the electroencephalogram (brain waves). Q J Stud Alc

Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations

Behav Res Methods Instrum Comput

Temporal placement of a nap for alertness: contributions of circadian phase and prior wakefulness

Sleep

Fatigue-related reaction time performance in long-haul flight crews

Sleep Res