Face stimuli effectively prevent brain–computer interface inefficiency in patients with neurodegenerative disease
Highlights
► This study shows how to effectively overcome brain computer interface (BCI) inefficiency in patients with neurodegenerative disease. ► Online performance was significantly increased in healthy participants (N = 16) and those with neurodegenerative disease (N = 9) when using faces as stimulus material in an ERP–BCI paradigm. ► Importantly, two patients who were highly inefficient with the classic BCI paradigm spelled at high accuracy levels with the face flashing paradigm.
Introduction
Brain computer interfaces (BCI) based on event-related potentials (ERP) provide a communication channel independent from muscular control, thus, potentially suited for patients with neurodegenerative diseases or severe motor impairment due to other causes such as brainstem stroke (Farwell and Donchin, 1988, Sellers and Donchin, 2006, Nijboer et al., 2008; for review Kleih et al., 2011, Mak et al., 2011). Such ERP–BCIs utilize a so-called oddball paradigm, i.e. presenting a rare target stimulus within a set of irrelevant stimuli. Commonly, users are presented with a matrix consisting of characters that are highlighted (flashed) in random order (see Fig. 1A). Communication is established by focusing attention on the intended character and counting the number of flashes. The attended target stimuli elicit ERPs (for review, Polich, 2007) in the electroencephalogram (EEG) that can be classified and hence indicate the intended character for communication (Farwell and Donchin, 1988).
Although being fast and reliable in healthy participants, studies on ERP–BCI use by impaired participants reveal large inter-individual variations in achieved BCI performance (Nijboer et al., 2008, Kübler and Birbaumer, 2008; for review, Mak et al., 2011). Reliable communication does not require perfect spelling performance since human communication partners or advanced predictive text algorithms are able to correct for spelling errors and extract informational content. Yet, BCI performance of patients is often even below a minimum level of accuracy required for basic communication (e.g., 70% accuracy as suggested by Kübler et al., 2001). The term “BCI illiteracy” has often been used to describe non-successful BCI use (e.g. Kübler and Müller, 2007, Vidaurre and Blankertz, 2010, Blankertz et al., 2010), but was suggested to be replaced by BCI inefficiency to better stress that the inability may be inherent in the system, not in the user (Kübler et al., 2011). The major goal of ERP–BCI research can thus be described as a 2-step process, (1) establishing a sufficient accuracy level for communication and if successful (2) increasing spelling speed without decreasing accuracy, i.e. increasing communication bit rate (correctly spelled characters per time unit).
Recently we addressed this issue by developing a new paradigm which aims at increasing the signal-to-noise ratio (SNR) within the entire classification time window by eliciting additional target specific ERPs (Kaufmann et al., 2011, see Fig. 1B). When flashing characters with transparently superimposed well-known faces, ERPs involved in face processing are elicited (N170, N400f) and were found to significantly increase classification accuracy in an offline setting. For example when well-known faces were used as stimulus material (face flashing, FF) a stable level of 100% offline accuracy was achieved in healthy participants with significantly fewer sequences necessary for classification than with classic character flashing (character flashing, CF).
Consequently with FF, it was possible to achieve high accuracy levels in an online setting using single trial classification in seven healthy participants (Zhang et al., 2012). However, until now it has not been known how the FF paradigm affects BCI inefficiency in severely motor-impaired end-users. Thus, this study systematically validated online classification accuracy in both healthy and motor-impaired BCI users.
Furthermore, our study explored optimization of stimulus material. Jin and colleagues (2012) investigated if face emotion and/or motion of FF stimuli may increase spelling accuracy, yet no difference was found between these stimuli. Herein we investigated the role of face familiarity. Initially we proposed use of famous faces (Kaufmann et al., 2011). Touryan and colleagues (2011) found that faces of family members elicited larger N400 potentials than celebrity faces. Such personally known faces may also increase end-user acceptance of the stimulus material due to personal meanings of the pictures (Kaufmann et al., 2011). Jin and colleagues (2012) used one face that was personally known by all participants (fellow student) whereas Zhang and colleagues (2012) used a prior unfamiliar face. In the current study, we compared these conditions (unfamiliar FF, famous FF, and personally known FF) to investigate effects of face familiarity on spelling accuracy.
Section snippets
Stimulus material
In our previous study (Kaufmann et al., 2011) we used two face flashing (FF) stimuli, i.e. famous photographs of Albert Einstein and Ernesto ‘Che’ Guevara. As we found no difference in classification accuracy between these FF conditions, we herein used the famous face that was rated as marginally more familiar, i.e. the face of Albert Einstein. Besides this famous face we included a prior unfamiliar face and personally known faces for comparison across FF stimuli. All participants sent a
Spelling performance
All participants completed all scheduled sessions, except for patients 1 and 2, who skipped the last online session due to strain (OS5, i.e. session with NoS = 1 for all three stimulus conditions). Classification accuracy estimated offline from calibration data was in line with our previous report (Kaufmann et al., 2011) in that face stimuli effectively increased offline classification accuracy (Fig. 2, CF vs. FF).
The superior efficiency of the FF paradigm emerged particularly when exposing
Discussion
Online ERP–BCI performance significantly increased when superimposing the characters of the matrix with faces. As known from several studies, patients performed significantly lower than healthy subjects in the classic character flashing condition. Not so, however, in the flashing faces condition.
Conclusion
This study compared classic target flashing against paradigms in which targets were transparently overlaid with faces. All face stimuli resulted in significantly increased online accuracy compared to classic character flashing. Patients benefited from the flashing face conditions to such an extent that their performance in single-sequence online classification did not significantly differ from that achieved by healthy users. This allowed for substantially decreasing the number of character
Acknowledgments
The authors declare no competing financial interests. This work was supported by the Deutsche Forschungsgesellschaft (DFG) RTG 1253/1 and the European ICT Programme Project FP7-224631. This paper only reflects the authors’ views and funding agencies are not liable for any use that may be made of the information contained herein.
References (25)
- et al.
Neurophysiological predictor of SMR-based BCI performance
Neuroimage
(2010) Event-related brain potentials distinguish processing stages involved in face perception and recognition
Clin Neurophysiol
(2000)- et al.
Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials
Electroencephalogr Clin Neurophysiol
(1988) - et al.
Out of the frying pan into the fire – the P300-based BCI faces real-world challenges
Prog Brain Res
(2011) - et al.
Toward enhanced P300 speller performance
J Neurosci Methods
(2008) - et al.
Brain–computer communication: self-regulation of slow cortical potentials for verbal communication
Arch Phys Med Rehabil
(2001) - et al.
Brain–computer interfaces and communication in paralysis: extinction of goal directed thinking in completely paralyzed patients?
Clin Neurophysiol
(2008) - et al.
A P300-based brain–computer interface for people with amyotrophic lateral sclerosis
Clin Neurophysiol
(2008) Updating P300: an integrative theory of P3a and P3b
Clin Neurophysiol
(2007)- et al.
A P300-based brain–computer interface: initial tests by ALS patients
Clin Neurophysiol
(2006)