Face stimuli effectively prevent brain–computer interface inefficiency in patients with neurodegenerative disease

https://doi.org/10.1016/j.clinph.2012.11.006Get rights and content

Abstract

Objectives

Recently, we proposed a new stimulation paradigm for brain computer interfaces (BCI) based on event-related potentials (ERP), i.e. flashing characters with superimposed pictures of well-known faces. This new face flashing (FF) paradigm significantly outperformed the commonly used character flashing (CF) approach, i.e. simply highlighting characters.

Methods

In the current study we assessed the impact of face stimuli on BCI inefficiency in patients with neurodegenerative disease, i.e. on their inability to communicate by means of a BCI. Healthy participants (N = 16) and those with neurodegenerative disease (N = 9) performed spelling tasks using CF and FF paradigms.

Results

Online performance with FF was significantly increased as compared to CF in both, healthy and impaired users. Importantly, two patients who were classified “highly inefficient” with the classic CF stimulation were able to spell with high accuracy using FF. Our results particularly emphasize great benefit of the FF paradigm for those users displaying low signal-to-noise ratio of the recorded ERPs in the classic stimulation approach.

Conclusion

In conclusion, we confirm previously reported results now systematically validated in an online setting and display specifically beneficial effects of FF for motor-impaired users.

Significance

The FF paradigm thus constitutes a big step forward against the BCI inefficiency phenomenon.

Highlights

► This study shows how to effectively overcome brain computer interface (BCI) inefficiency in patients with neurodegenerative disease. ► Online performance was significantly increased in healthy participants (N = 16) and those with neurodegenerative disease (N = 9) when using faces as stimulus material in an ERP–BCI paradigm. ► Importantly, two patients who were highly inefficient with the classic BCI paradigm spelled at high accuracy levels with the face flashing paradigm.

Introduction

Brain computer interfaces (BCI) based on event-related potentials (ERP) provide a communication channel independent from muscular control, thus, potentially suited for patients with neurodegenerative diseases or severe motor impairment due to other causes such as brainstem stroke (Farwell and Donchin, 1988, Sellers and Donchin, 2006, Nijboer et al., 2008; for review Kleih et al., 2011, Mak et al., 2011). Such ERP–BCIs utilize a so-called oddball paradigm, i.e. presenting a rare target stimulus within a set of irrelevant stimuli. Commonly, users are presented with a matrix consisting of characters that are highlighted (flashed) in random order (see Fig. 1A). Communication is established by focusing attention on the intended character and counting the number of flashes. The attended target stimuli elicit ERPs (for review, Polich, 2007) in the electroencephalogram (EEG) that can be classified and hence indicate the intended character for communication (Farwell and Donchin, 1988).

Although being fast and reliable in healthy participants, studies on ERP–BCI use by impaired participants reveal large inter-individual variations in achieved BCI performance (Nijboer et al., 2008, Kübler and Birbaumer, 2008; for review, Mak et al., 2011). Reliable communication does not require perfect spelling performance since human communication partners or advanced predictive text algorithms are able to correct for spelling errors and extract informational content. Yet, BCI performance of patients is often even below a minimum level of accuracy required for basic communication (e.g., 70% accuracy as suggested by Kübler et al., 2001). The term “BCI illiteracy” has often been used to describe non-successful BCI use (e.g. Kübler and Müller, 2007, Vidaurre and Blankertz, 2010, Blankertz et al., 2010), but was suggested to be replaced by BCI inefficiency to better stress that the inability may be inherent in the system, not in the user (Kübler et al., 2011). The major goal of ERP–BCI research can thus be described as a 2-step process, (1) establishing a sufficient accuracy level for communication and if successful (2) increasing spelling speed without decreasing accuracy, i.e. increasing communication bit rate (correctly spelled characters per time unit).

Recently we addressed this issue by developing a new paradigm which aims at increasing the signal-to-noise ratio (SNR) within the entire classification time window by eliciting additional target specific ERPs (Kaufmann et al., 2011, see Fig. 1B). When flashing characters with transparently superimposed well-known faces, ERPs involved in face processing are elicited (N170, N400f) and were found to significantly increase classification accuracy in an offline setting. For example when well-known faces were used as stimulus material (face flashing, FF) a stable level of 100% offline accuracy was achieved in healthy participants with significantly fewer sequences necessary for classification than with classic character flashing (character flashing, CF).

Consequently with FF, it was possible to achieve high accuracy levels in an online setting using single trial classification in seven healthy participants (Zhang et al., 2012). However, until now it has not been known how the FF paradigm affects BCI inefficiency in severely motor-impaired end-users. Thus, this study systematically validated online classification accuracy in both healthy and motor-impaired BCI users.

Furthermore, our study explored optimization of stimulus material. Jin and colleagues (2012) investigated if face emotion and/or motion of FF stimuli may increase spelling accuracy, yet no difference was found between these stimuli. Herein we investigated the role of face familiarity. Initially we proposed use of famous faces (Kaufmann et al., 2011). Touryan and colleagues (2011) found that faces of family members elicited larger N400 potentials than celebrity faces. Such personally known faces may also increase end-user acceptance of the stimulus material due to personal meanings of the pictures (Kaufmann et al., 2011). Jin and colleagues (2012) used one face that was personally known by all participants (fellow student) whereas Zhang and colleagues (2012) used a prior unfamiliar face. In the current study, we compared these conditions (unfamiliar FF, famous FF, and personally known FF) to investigate effects of face familiarity on spelling accuracy.

Section snippets

Stimulus material

In our previous study (Kaufmann et al., 2011) we used two face flashing (FF) stimuli, i.e. famous photographs of Albert Einstein and Ernesto ‘Che’ Guevara. As we found no difference in classification accuracy between these FF conditions, we herein used the famous face that was rated as marginally more familiar, i.e. the face of Albert Einstein. Besides this famous face we included a prior unfamiliar face and personally known faces for comparison across FF stimuli. All participants sent a

Spelling performance

All participants completed all scheduled sessions, except for patients 1 and 2, who skipped the last online session due to strain (OS5, i.e. session with NoS = 1 for all three stimulus conditions). Classification accuracy estimated offline from calibration data was in line with our previous report (Kaufmann et al., 2011) in that face stimuli effectively increased offline classification accuracy (Fig. 2, CF vs. FF).

The superior efficiency of the FF paradigm emerged particularly when exposing

Discussion

Online ERP–BCI performance significantly increased when superimposing the characters of the matrix with faces. As known from several studies, patients performed significantly lower than healthy subjects in the classic character flashing condition. Not so, however, in the flashing faces condition.

Conclusion

This study compared classic target flashing against paradigms in which targets were transparently overlaid with faces. All face stimuli resulted in significantly increased online accuracy compared to classic character flashing. Patients benefited from the flashing face conditions to such an extent that their performance in single-sequence online classification did not significantly differ from that achieved by healthy users. This allowed for substantially decreasing the number of character

Acknowledgments

The authors declare no competing financial interests. This work was supported by the Deutsche Forschungsgesellschaft (DFG) RTG 1253/1 and the European ICT Programme Project FP7-224631. This paper only reflects the authors’ views and funding agencies are not liable for any use that may be made of the information contained herein.

References (25)

  • S. Bentin et al.

    Electrophysiological studies of face perception in humans

    J Cogn Neurosci

    (1996)
  • J. Jin et al.

    The changing face of P300 BCIs: a comparison of stimulus changes in a P300 BCI involving faces, emotion, and movement

    PLoS One

    (2012)
  • Cited by (0)

    View full text