A novel P300-based brain–computer interface stimulus presentation paradigm: Moving beyond rows and columns

Abstract

Objective

An electroencephalographic brain–computer interface (BCI) can provide a non-muscular means of communication for people with amyotrophic lateral sclerosis (ALS) or other neuromuscular disorders. We present a novel P300-based BCI stimulus presentation – the checkerboard paradigm (CBP). CBP performance is compared to that of the standard row/column paradigm (RCP) introduced by Farwell and Donchin (1988).

Methods

Using an 8 × 9 matrix of alphanumeric characters and keyboard commands, 18 participants used the CBP and RCP in counter-balanced fashion. With approximately 9–12 min of calibration data, we used a stepwise linear discriminant analysis for online classification of subsequent data.

Results

Mean online accuracy was significantly higher for the CBP, 92%, than for the RCP, 77%. Correcting for extra selections due to errors, mean bit rate was also significantly higher for the CBP, 23 bits/min, than for the RCP, 17 bits/min. Moreover, the two paradigms produced significantly different waveforms. Initial tests with three advanced ALS participants produced similar results. Furthermore, these individuals preferred the CBP to the RCP.

Conclusions

These results suggest that the CBP is markedly superior to the RCP in performance and user acceptability.

Significance

The CBP has the potential to provide a substantially more effective BCI than the RCP. This is especially important for people with severe neuromuscular disabilities.

Introduction

Brain–computer interfaces (BCIs) facilitate reestablishing communication and environmental control for people whose motor and communicative abilities have been impaired by severe neuromuscular disease (Wolpaw and Birbaumer, 2006). For example, although cognitive function is usually spared, the motoneuron death associated with amyotrophic lateral sclerosis (ALS) ultimately renders people physically incapacitated as they lose all voluntary muscle control. These people may become “locked-in” to their bodies, unable to communicate, and completely dependent upon caregivers to attend to their basic needs. Importantly, however, the use of a BCI can mitigate the isolation and dependence they experience by providing a mode of communication not contingent on neuromuscular activity.

BCIs translate volitional modulation of brain signals into computer commands, which can be recorded from the scalp using electroencephalography (EEG; e.g., Farwell and Donchin, 1988, Wolpaw and McFarland, 2004), from the dura mater or cortical surface using electrocorticography (ECoG; e.g., Leuthardt et al., 2004), or from neurons within the cortex (e.g., Hochberg et al., 2006). A common signal for BCI is the P300 event-related potential (ERP). The P300 ERP is a positive deflection in the EEG over parietal cortex that occurs approximately 300 ms after an “oddball” stimulus: a rare but meaningful stimulus among a series of frequently occurring stimuli. Because the P300 occurs amid other ongoing EEG activity, several P300 responses must usually be averaged for the response to be recognized (Fabiani et al., 1987, Polich, 2007, Pritchard, 1981).

Farwell and Donchin (1988) introduced the first P300-based BCI paradigm. In this paradigm, a computer presents a 6 × 6 matrix of letters and commands on-screen and participants attend to the item they wish to select. Groups of matrix items are flashed randomly. Only flashes of groups containing the attended item should elicit a P300. In this original implementation of a P300 BCI, and in most subsequent implementations, items are grouped for flashing as rows and columns; hence, the row–column paradigm, or RCP. The computer identifies the attended item as the intersection of the row and column that elicited the largest P300.

The RCP has been tested in various configurations to achieve efficient communication that is practical for in-home use. For example, researchers have explored various electrode montages (Krusienski et al., 2006), stimulus properties such as inter-stimulus interval (ISI) and matrix size (Sellers et al., 2006), and various signal processing methods (Kaper et al., 2004, Krusienski et al., 2006, Lenhardt et al., 2008, Serby et al., 2005).

Others have modified the RCP paradigm. For example, Martens et al. (2009) compared the RCP speller to an apparent motion paradigm where motion occurs in rows and columns. Similarly, Hong et al. (2009) compared the RCP to an apparent motion and color onset paradigm that also presents the color and motion stimuli in a row/column arrangement. Takano et al. (2009) recently investigated RCP accuracy using three different luminance and chromatic flash patterns: a white/grey pattern (luminance condition); a green/blue isoluminance pattern (chromatic condition); and a green/blue luminance pattern (luminance chromatic condition). The luminance chromatic condition produced online accuracy higher than the luminance or chromatic condition alone. Salvaris and Sepulveda (2009) compared changes to the background/foreground colors, item size, and distances between items. Their results demonstrated that, although no single paradigm was best for everybody, a white background produced the highest mean offline classification accuracy, and small symbol sizes produced the lowest mean classification accuracy. Finally, Guger et al. (2009) compared the RCP to a paradigm in which single items flash at random. They found that the RCP yielded higher accuracy and bit rate than the single item flash paradigm, even though the P300 responses were larger for the latter. In sum, none of these alternative paradigms substantially improves P300-based BCI performance.

Two additional studies have used stimuli that are not presented in a RCP format. Allison (2003) presented random groups of items in an arrangement referred to as a “splotch” presentation, somewhat similar to the method presented in this article. The splotch presentation reduced the number of flanking items that flash with the target, and participants reported that they preferred the method; however, no data with regard to BCI performance were reported. Hill et al. (2009) also tested a variation of a random stimulus presentation using an offline leave-one-out cross-validation. Their results suggested that the random presentation did not perform as well as the standard RCP; however, no statistical analyses were performed to test the performance difference.

The RCP remains subject to errors that slow communication, cause frustration, and diminish attentional resources. Importantly, these errors appear to have two primary causes.

First, errors typically occur with the greatest frequency in locations adjacent to the attended item (i.e., the target item) and almost always in the same row or column (Fazel-Rezai, 2007). This inherent RCP error occurs because each time a target item flashes, a P300 is produced for every item in the row or column. However, only the intersection of the target row and column is unique to the target item. Errors arise when flashes of non-target rows or columns that are adjacent to the target, attract the participant’s attention, thereby producing P300 responses. We refer to these relatively systematic errors as “adjacency-distraction errors” (or the “adjacency-distraction problem”). This phenomenon is well documented in the spatial attention literature. For example, in a standard flanker task, response time significantly increases when nearby items belong to a response class different from the target class (e.g., Sanders and Lamers, 2002). In the RCP, when adjacency-distraction errors occur with sufficient frequency, the distractions cause one of the four adjacent items (or another item in the same row or column of the target) to be selected unintentionally.

Second, in order to conform to the oddball paradigm, sets of items must be intensified in random order. This allows target items to, at times, flash consecutively. That is, when a row flash is followed by a column flash (or vice versa), and the target item is at the intersection of that particular row and column, the target item flashes twice in immediate succession. Due to the rapid rate of intensification, double flashes can cause errors of two types. One, if the target item is involved in a double flash, the second flash may go unnoticed by the participant, so that it does not produce a P300 response. Two, even if the second flash is perceived, the P300 responses to the two flashes overlap temporally. This can reduce P300 amplitude or change its morphology (Martens et al., 2009, Woldorff, 1993). We refer to these errors as “double-flash errors” (or the “double-flash problem”).

Further RCP research could possibly help severely disabled BCI users, who desire speed, accuracy, and ease of use. However, the kinds of errors that are inevitably associated with the RCP can still make it frustrating for users and burdensome for their caregivers (Vaughan et al., 2006). Moreover, with the RCP, some people are not able to achieve accuracy high enough for practical BCI use (Sellers and Donchin, 2006). In recognition of these issues, we sought to create an alternative stimulation paradigm that is faster, more accurate, and more reliable than the RCP.

To achieve this goal, we designed an alternative to the RCP that is called the checkerboard paradigm, or CBP. We used an 8 × 9 matrix containing 72 items. In the RCP, the eight columns and nine rows flash at random (Fig. 1A). In contrast, in the CBP, the standard 8 × 9 matrix is virtually superimposed on a checkerboard (Fig. 1B, left), which the participants never actually see. The items in white cells of the 8 × 9 matrix are segregated into a white 6 × 6 matrix and the items in the black cells are segregated into a black 6 × 6 matrix. Before each sequence of flashes, the items in Fig. 1B (left) randomly populate the white or black matrix, respectively, as shown in Fig. 1B (middle). The virtual checkerboard layout controls for adjacency-distraction errors, because adjacent items cannot be included in the same flash group. The end result is that the participants see random groups of six items flashing (as opposed to rows and columns) because the virtual rows and columns depicted in Fig. 1 (middle) flash. For example, the top row of the white matrix includes the items: 2, Bs, Shift, H, Sp, EC. In this example, the participant is shown the standard 8 × 9 matrix Fig. 1B (right) and the six items from the top row of Fig. 1B (middle, top) flash. In other words, the standard matrix never changes; only the pattern of flashing items is changed. During one sequence, the six virtual rows in the white matrix (Fig. 1B, middle) flash in order from top to bottom followed by the six virtual rows in the black matrix. Then the six virtual columns in the white matrix flash in order from left to right followed by the six virtual columns in the black matrix.

Due to the fact that the randomized virtual rows of each matrix flash first (12 flashes) and then the virtual columns of each matrix flash (12 flashes), any given matrix item cannot flash again for a minimum of six intervening flashes and a maximum of 18 intervening flashes. This eliminates the double-flash problem. After all rows and columns in both matrices have flashed (i.e., 24 flashes, comprising one complete sequence), the program re-randomizes the positions of the items in each virtual matrix and the next sequence of flashes begins. In addition, the CBP almost completely avoids overlapping target epochs because six intervening flashes correspond to 750 ms and we used classification epochs of 800 ms. Simply eliminating the double-flash problem does not ensure that enough time will be presented between target items to keep the target epochs from overlapping, and this has been shown to cause deleterious effects to the P300 (Squires et al., 1976). By maximizing the time between successive flashes of the target item, the CBP should increase the amplitude of the P300 responses (Polich et al., 1991) and should also improve BCI speed and accuracy.

In this study, our hypothesis is that the CBP will produce superior performance as compared to the RCP because it avoids the adjacency-distraction and double-flash errors to which the RCP is prone. In addition to comparing the two paradigms, we also sought to optimize the stepwise linear discriminant analysis (SWLDA; Draper and Smith, 1981) classifier to achieve the highest online speed and accuracy (i.e., bit rate) possible. Moreover, the expansion to an 8 × 9 matrix allows the inclusion of both alphanumeric keys and function keys, giving the participant more control and communication options. The larger matrix should also produce larger P300 amplitudes for the target items because the probability of the target stimulus occurring is reduced. This relationship is found in standard oddball experiments (e.g., Duncan-Johnson and Donchin, 1977) and also in BCI applications (Allison and Pineda, 2003, Sellers et al., 2006). Finally, while the larger matrix increases the time needed for each selection, it should increase the information transferred per selection.