• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Clin Neurophysiol. Author manuscript; available in PMC Aug 1, 2011.
Published in final edited form as:
PMCID: PMC2895010

Towards a User-Friendly Brain-Computer Interface: Initial Tests in ALS and PLS Patients



Patients usually require long-term training for effective EEG-based brain-computer interface (BCI) control due to fatigue caused by the demands for focused attention during prolonged BCI operation. We intended to develop a user-friendly BCI requiring minimal training and less mental load.


Testing of BCI performance was investigated in three patients with amyotrophic lateral sclerosis (ALS) and three patients with primary lateral sclerosis (PLS), who had no previous BCI experience. All patients performed binary control of cursor movement. One ALS patient and one PLS patient performed four-directional cursor control in a two-dimensional domain under a BCI paradigm associated with human natural motor behavior using motor execution and motor imagery. Subjects practiced for 5-10 minutes and then participated in a multi-session study of either binary control or four-directional control including online BCI game over 1.5 – 2 hours in a single visit.


Event-related desynchronization and event-related synchronization in the beta band were observed in all patients during the production of voluntary movement either by motor execution or motor imagery. The online binary control of cursor movement was achieved with an average accuracy about 82.1±8.2% with motor execution and about 80% with motor imagery, whereas offline accuracy was achieved with 91.4±3.4% with motor execution and 83.3±8.9% with motor imagery after optimization. In addition, four-directional cursor control was achieved with an accuracy of 50-60% with motor execution and motor imagery.


Patients with ALS or PLS may achieve BCI control without extended training, and fatigue might be reduced during operation of a BCI associated with human natural motor behavior.


The development of a user-friendly BCI will promote practical BCI applications in paralyzed patients.

Keywords: EEG, brain-computer interface (BCI), event-related desynchronization (ERD), event-related synchronization (ERS), user-friendly, amyotrophic lateral sclerosis (ALS), primary lateral sclerosis (PLS), motor control

1. Introduction

Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease that affects nerve cells which are responsible for controlling voluntary movement. Primary lateral sclerosis (PLS) is a variant of ALS that affects the corticospinal upper motor neurons, limiting movement. ALS/PLS patients, as well as patients disabled from other degenerative diseases or brain injuries, have difficulty with everyday motor behaviors such as moving, swallowing, and speaking. In the later stages of disease, some patients may completely lose motor function and become totally ‘locked-in’ (Hayashi and Oppenheimer 2003). Loss of motor function significantly affects patients’ quality of life (QoL) (Mockford et al. 2006; Bromberg 2008; Williams et al. 2008; Lule et al. 2009) andincreases the financial burden for the cost of care (Mutsaarts et al. 2004). A brain-computer interface (BCI) or brain-machine interface (BMI), has been proposed as an alternative communication pathway, bypassing the normal cortical-muscular pathway (Joseph 1985; Kennedy et al. 2000). BCI is a system that provides a neural interface to substitute for the loss of normal neuromuscular outputs by enabling individuals to interact with their environment through brain signals rather than muscles (Wolpaw et al. 2002; Daly and Wolpaw 2008). Recent years have featured a rapid growth of BCI research and development owing to increased societal interest and appreciation of the serious needs and impressive potential of patients with severe motor disabilities (Birbaumer and Cohen 2007; Daly and Wolpaw 2008). The majority of BCI-related publications have studied performance in healthy volunteers and focused on the development of signal processing/computational algorithms to improve BCI performance (Bashashati et al. 2007). Practical BCI clinical applications for the potential patient users, however, are still limited (Birbaumer 2006a).

The BCIs using invasive signal methods to record intracortical neuronal activities have shown great promise in direct brain control of external devices in primates, for example, to restore self-feeding by controlling a 3-D robotic arm (Velliste et al. 2008). However, due to the technical concerns such as associated surgical risks as well as unclear long-term benefit and robustness, non-invasive signal methods, mainly EEG, have been extensively explored because of the ease of use. In contrast to invasive methods, noninvasive methods may only provide an extremely low signal-to-noise (s/n) ratio, which is a major technical difficulty in EEG-based BCI development. Conventionally, s/n ratio can be improved by repeated averaging, for example, as in event-related potentials (ERPs), which can be obtained by averaging across trials time-locked to the stimuli. The P300-based BCIs (Sellers et al. 2006) use this principle by averaging across multiple trials. However, due to the requirement for repeated measurements, the communication speed is greatly reduced. An alternative method to improve s/n ratio for reliable BCI control is to train users to regulate their brain activity, such as by modulation of the slow-cortical potentials (SCP) (Birbaumer et al. 2000) or the 8-12 Hz sensorimotor Mu rhythm (Wolpaw and McFarland 1994). Once people learn to effectively regulate their brain activity, reduction of the variance in the EEG signal can be expected and as a result, the s/n ratio is increased. However, due to the variance of spontaneous activity in EEG, long-term training is usually required for users to achieve effective and accurate regulation of either SCPs or sensorimotor Mu rhythms. The long-term training may require a couple of months to 1 or 2 years (Wolpaw and McFarland 2004; Iversen et al. 2008b). Moreover, users may be easily fatigued from the sustained attention that is required to regulate their brain activities and as a result, render the BCI control unreliable. Studies using short-term training of the EEG-based BCIs, however, are of unclear quality, because EMG or EOG artifacts may be used as the features for BCI classification (McFarland et al. 2005; Fatourechi et al. 2007).

Fatigue is a serious problem in severely paralyzed patients who demonstrate not only reduced physical but also mental endurance(Sykacek et al. 2003; Birbaumer 2006b). The fatigue factor not only affects the performance in rhythm-based BCIs, but also affects P300-based BCIs. Fatigue was identified as a serious problem for ALS patients to reliably select virtual keys on a computer monitor using the P300 BCI because of the requirement to keep constant and sustained attention (Sellers and Donchin 2006). Recent pilot studies of BCI feasibility for ALS patients shows that they may not be able to learn the skills for effective regulation of brain activities because they are too weak to tolerate long-term training and/or active regulation with focused attention (Kubler et al. 1999; Kubler et al. 2001; Hill et al. 2006). Though healthy persons or less severely paralyzed patients may operate current EEG-based BCIs efficiently (Birch et al. 2002; Blankertz et al. 2007), the performance of current BCIs in severely paralyzed patients with degenerative diseases such as ALS, however, was much lower because they were easily fatigued or could not tolerate long-term training. The accuracy was just over the random level for ALS patients, in contrast to the 90% accuracy level achieved in healthy subjects (Sellers and Donchin 2006; Iversen et al. 2008a). Therefore, the inconvenience in operation may prevent current BCIs from practical clinical applications for severely paralyzed patients who are the users most in need of direct brain control of external devices to restore function.

We have identified that the human volition to move or cease to move associated with natural motor behavior can be reliably decoded online from EEG signals, where users do not need to learn to regulate brain activities. Our BCI strategy was established on a new robust feature: beta rebound or beta-ERS (event-related synchronization) associated with natural motor behavior. We found that the discrimination of ERD (event-related synchronization) from beta-ERS was much more reliable than the discrimination of ERD from background activities in conventional BCI methods (Bai et al. 2008; Kayagil et al. 2009). A short-lasting burst of EEG oscillation, termed as beta rebound or beta-ERS, has been observed in beta band (16-30 Hz) over human sensorimotor area after subjects produce a self-paced movement (Salmelin et al. 1995; Pfurtscheller and Lopes da Silva 1999; Neuper and Pfurtscheller 2001). Though the beta rebound has been postulated as the result of afferent input (Cassim et al. 2001), other studies show that the beta rebound does not necessarily depend on motor cortex output and muscle activation, and it may reflect a short lasting state of deactivation or inhibition of the motor cortex (Pfurtscheller 1992; Pfurtscheller et al. 1996). The feasibility of the beta rebound for BCI application derives from the fact that beta rebound may not only occur with real physical movement but that it also presents with motor imagery (Pfurtscheller et al. 2005). This comes into consideration since the patients who lose their voluntary muscle contraction may only imagine movement instead of producing real movement (Bai et al. 2008). The beta rebound results in a strong synchronization, i.e., higher amplitude of rhythmic activities in beta band than background activities. As ERD features lower amplitude beta band activities, the discrimination of beta rebound or beta-ERS from beta ERD is presumably more accurate than the discrimination of ERD from background activity. Furthermore, the beta rebound also features strict somatotopic organization (Salmelin et al. 1995), allowing for potential discrimination of different limb movements spatially according to human somatotopy. The spatial feature of the beta rebound, therefore, may support a multi-dimensional BCI by reliable decoding of intentions to move individual limbs (Huang et al. 2009).

In this study, we intend to determine whether beta-ERD and beta-ERS features associated with human natural motor control (either motor execution or motor imagery) are available in patients with ALS or PLS. Further, we determined whether accurate BCI control could be achieved in patients when using ERD and beta-ERS features without extensive training for brain rhythmic regulation.

2. Methods

2.1 Subjects

Three patients with ALS and three patients with PLS participated in the study. The detail clinical conditions of sex, age, duration of disease, dementia rating scale (DRS), ALS functional rating scale revised (ALSFRS-R) (Cedarbaum et al. 1999), finger tapping rate and wrist extensor strength were provided in Table 1. All patients underwent a neuropsychological testing battery that included testing for memory, attention, frontal executive function, and frontal behavioral assessment (Mattis Dementia Rating Scale (Schmidt et al. 1994), Delis-Kaplan Executive Function Battery (Delis et al. 2004), Frontal Systems Behavioral Scale (Stout et al. 2003)), and had no evidence of cognitive impairment. All subjects had no prior exposure to BCI experiments before participating in this study. Both upper and lower limbs in all patients were affected. All ALS patients were wheelchair-bound. All patients were right-handed according to the Edinburgh inventory (Oldfield 1971). The protocol was approved by the Institutional Review Board; all subjects gave their written informed consent for the study.

Table 1
Subject information

2.2 Experimental paradigm

All subjects participated in the first visit study of the sequential binary control for 2-D cursor movement. ALS1 and PLS1 participated in a second visit: study of four-directional control for 2-D cursor movement on the next day subsequent to their first visit. There were two sessions to examine the BCI performance associated with motor execution and motor imagery during each visit. During motor imagery, the investigator monitored the EMG activity continuously. Subjects were reminded to relax the muscle when EMG activity was observed, and trials with EMG activity were excluded for both classification and analysis. Each motor execution or motor imagery session was composed of an initial calibration step to determine optimal frequency band and spatial channels, and to create the mathematical model for classification. Subsequently, the selected features and determined model parameters were used to test the subject while performing the 2-D cursor-control game. Subjects were seated in a chair with the forearm semi-flexed and supported by a pillow and instructed to suppress eye blink during the visual cue onset. A computer monitor was placed about 1.5 m in front of the subjects to display the visual paradigm. The experiment in each visit was completed within three hours including EEG setup time.

2.3 Data acquisition and online data processing system

EEG was recorded from 29 (tin) surface electrodes (Fp1, F3, F7, C3A, C1, C3, C5, T3, C3P, P3, T5, O1, Fp2, F4, F8, C4A, C2, C4, C6, T4, C4P, P4, T6, O2, FZ, FCZ, CZ, CZP, and PZ), attached on an elastic cap (Electro-Cap International, Inc., Eaton, OH, U.S.A.) according to the international 10-20 system (Jasper and Andrews 1938). Bipolar recordings of electrooculogram (EOG) above the left eye and below the right eye, and electromyogram (EMG) from the right and left forearm extensors were also obtained. Signals from all channels were amplified (Neuroscan Inc., El Paso, TX), filtered (0.1-100Hz), and digitized (sampling frequency, 250 Hz).

The amplified analog signal was sent to a HP PC workstation, and converted to digital signal from an analog-digital converter (National Instruments Corp., Austin, TX). The digital signal was online processed using a home-made MATLAB (MathWorks, Natick, MA) Toolbox: brain-computer interface to virtual reality (BCI2VR). The BCI2VR program provided both the visual stimulus for the calibration and the binary cursor-control game as well as online processing of the EEG signal. The signal for decoding was extracted following the visual cues. The online signal processing to decode movement intention consisted of three steps: (1) preprocessing with spatial and temporal/frequency filter, (2) feature extraction, and (3) feature classification.

2.4 Experimental paradigms

2.4.1 Binary control paradigm

A text box was provided in the center of the computer monitor. The text message was either a blue ‘Yes’ or ‘No’ as illustrated in Fig. 1. Subjects were instructed to start a motor task with motor execution or motor imagery of repetitive wrist extension when they perceived the blue text message of either ‘Yes’ or ‘No’. Subjects kept performing the motor task in the Condition window of 2.5 s until the color of the text message changed from blue to green. In the ‘Yes’ case, subjects were instructed to continue the motor task of either motor execution or motor imagery until the text message disappeared. In the ‘No’ case, subjects were asked to stop the motor task and relax as soon as possible. The duration of the Detection window from text color change to text removal was also 2.5 s. Considering the response delay, the signal from 1 s after color change to the end of the Detection window was extracted for classification. After an intertrial interval randomly from 4-6 s, the next text message was provided. The detailed paradigm was provided in a previous study (Bai et al. 2008; Kayagil et al. 2009).

Fig. 1
Binary control paradigm. Subject started motor tasks of motor execution or motor imagery when they perceived the blue color text message of either ‘Yes’ or ‘No’. When the color of text message changed from blue to green, ...

Patients participated in both motor execution and motor imagery sessions. The purpose of the motor execution session was two-fold: the patients are more comfortable with the paradigms, and the investigators could check whether patients performed the instructions properly by monitoring their motor output from EMG. One important factor was that patients need to be relaxed as soon as possible at the beginning of the ‘Detection’ window in order to induce a transient feature of ERS for BCI detection.

ERD was expected when subjects performed the active motor task during the Detection window, whereas ERS was expected when subjects stopped the motor task in the Detection window. This paradigm would yield a more accurate classification between ERD and ERS compared with that between ERD and baseline activity.

2.4.2 Four-directional control paradigm

Each of the four text messages ‘RYes’, ‘RNo’, ‘LYes’ and ‘LNo’ was assigned to one of the four directions of a computer cursor, provided in the center of the computer monitor (See Fig. 2). One of four text messages in the corresponding cursor direction was provided each time. The message text was a blue color at first; in the cases of ‘RYes’ or ‘RNo’, subjects started to perform motor execution or motor imagery of their right wrist in the form of repetitive extension; and in the cases of ‘LYes’ or ‘LNo’, subjects started to perform motor execution or motor imagery of their left wrist in the form of repetitive extension. Subjects kept performing the motor task until the color change of the text message. In the Detection window after the color change, subjects were instructed to continue the motor task of right wrist extension or left wrist extension with text messages of ‘RYes’ or ‘LYes’, respectively. Subjects were asked to cease the motor task as soon as possible and relax when they saw the messages of ‘RNo’ or ‘LNo’. The durations of the Condition and Detection window were both 2 s. The signal between 1 s after the text color change and the end of the Detection window was extracted for classification. The detailed paradigm can be found in the previous study (Huang et al. 2009).

Fig. 2
Four-directional control paradigm. Subjects started motor execution or motor imagery of right hand movement upon perceiving the blue text message of ‘RYes’ or ‘RNo’, or left hand movement when perceiving ‘LYes’ ...

In the Detection window, the four motor tasks of ‘RYes’, ‘RNo’, ‘LYes’ and ‘LNo’ were associated with four spatial patterns of ERD over the left hemisphere, ERS over the left hemisphere, ERD over the right hemisphere and ERS over the right hemisphere according to human somatotopy of hand control. The spatial distribution of the four patterns provided the basis for the classification of ‘RYes’, ‘RNo’, ‘LYes’ and ‘LNo’ to achieve control of the four-directions of the computer cursor.

2.4.3 Online two-dimensional cursor control game

A computer game of virtual computer cursor control using BCI was developed to facilitate subjects’ interest and active involvement for BCI development (Kayagil et al. 2009). Subjects were asked to control the cursor movement in a two-dimensional space on the computer monitor (see Fig. 3) by performing motor tasks with either motor execution or motor imagery.

Fig. 3
Consecutive binary control and four-directional control of two-dimensional computer cursor movement: an online computer game to test the performance of binary control and four-directional control paradigms (see detail in the text).

The binary control of two-dimensional cursor movement was achieved by consecutive binary classification to determine one of up, down, right and left directions. Subjects were instructed to move the cursor (the square box in blue) towards the target (the red circle) with minimal cursor movements in the grids, and at the same time, avoid the trap (the black ghost). The initial position of the cursor as well as the target and trap position were randomly generated by the computer. Fig. 3 shows screen shots of a binary control in the up row. As the target was in the upper left direction of the cursor, the subjects would select either up or left cursor, i.e. ‘No’ directions. Similar to the binary control paradigm, subjects started motor task with either motor execution or motor imagery when the four text boxes were provided. Because the ‘No’ direction was closer to the target, subjects would stop the motor task when the cursor color changed to green so that the ERS activity was voluntarily produced. The computer determined whether the subjects intended to move to ‘Yes’ or ‘No’ direction according to the extracted EEG signal, i.e. ERS, with respect to the computer model created from the data obtained from binary control paradigm. The two ‘Yes’ directions were removed when the computer detected ERS signal. The two ‘No’ directions were changed to one ‘Yes’ direction and one ‘No’ direction, and the subjects performed the motor task to voluntarily ‘tell’ the computer which direction they wanted to move to. In the illustrated sample, the subjects performed a sustained movement, and the computer determined the ‘Yes’ direction and move the cursor upward. Similarly, subjects would control the cursor movement until it reached the target. The detailed explanation of the binary cursor control game was described in the previous study (Kayagil et al. 2009).

The scheme of the four-directional control of two-dimensional cursor movement was similar to that of binary cursor control. Because one of the possible four directions was able to be determined from one of ‘RYes’, ‘RNo’, ‘LYes’ and ‘LNo’, provided in four-directional control paradigm, the consecutive two binary classification was reduced to one classification from four options as shown in the lower row in Fig. 3. The detailed explanation of the four-directional cursor control game was described in the previous study (Huang et al. 2009).

2.5 Computation methods

2.5.1 Pre-processing

EEG signal in the Detection window was extracted for modeling and classification. Signals from 29 channels were spatially filtered by surface Laplacian derivation (SLD), i.e. signal from each electrode was referenced to the averaged potentials from the nearby four orthogonal electrodes (Hjorth 1975). The temporal filtering was achieved by power spectral estimation with Welch method. A 4 Hz frequency resolution with segment length of 0.25 s and 50% overlapping was determined for spectral estimation (Bai et al. 2007).

2.5.2 Feature extraction

Empirical feature reduction

assuming that movement intention associated cortical activities occur over the motor cortex, we reduced the channel number from 29 to 14, which covered both left and right motor areas. Furthermore, as we did not expect relevant activities in the delta, theta or gamma band, only alpha and beta band (8-30Hz) activities were extracted for modeling and classification. Thus, the total number of extracted features were 8 (frequency bins) × 14 (channels) = 112 features.

Bhattacharyya distance

Bhattacharyya distance provides an index of feature separability for binary classification, which is proportional to the inter-class mean difference divided by intra-class variance (Chatterjee et al. 2007). The empirically extracted features were ranked by the Bhattacharyya distance for further classification.

Genetic algorithm

Genetic algorithm (GA)-based feature selection is a stochastic search in the feature space guided by the concept of inheriting, where at each search step, good properties of the parent subsets found in previous steps are inherited. 10-fold cross-validation was used with a Mahalanobis linear distance (MLD) classifier for feature evaluation (Li and Doi 2006). In this approach, the population size we used was 20, the number of generations was 100, the crossover probability was 0.8, the mutation probability was 0.01, and the stall generation was 20.

2.5.3 Feature classification


A receiver operating characteristics (ROC) was generated from the feature with the largest Bhattacharyya distance, i.e. the one providing the largest inter-class separability. The working point was determined from the ROC curve that was the closet point to 100% true positive with 0% false positive.


The sub-optimal feature subset was selected by genetic algorithm (GA) with Mahalanobis Linear Distance (MLD) as the evaluation function. Then, the selected features providing the best cross-validation accuracy were applied to a Mahalanobis Linear Distance Classifier. The number of features for the subset was 4, which was determined from the cross-validation accuracy with feature numbers of 2, 4, 6, 8 and 10.


Similar with GA-MLD, the support vector machine (SVM) was employed as the evaluation function in GA-SVM. We employed a SVM approach provided in LIBSVM (Fan et al., 2005). The radial basis function was used as the SVM kernel function as it can provide similar classification outcome compared with other kernels (Keerthi and Lin 2003). As the performance of SVM depends on the regulation parameters or hyper-parameters C and the width of the kernel σ (Chang and Lin 2001; Muller et al. 2001), 10-fold cross-validation was performed; 2K , K from -5 to 15 with step of 2 for the penalty parameter and 2K, K from −15 to 5 with step of 2 for the spread parameter. These parameters were determined by the training dataset only.

2.5.4 Offline cross-validation

The offline performance of binary classification and four-directional classification were evaluated from 10-fold cross-validation; 90% of the data pool was used for training, and the other 10% was used for validation so that the validation dataset was independent from the training dataset. For classification methods using hyper-parameters (in SVM) or feature evaluation for feature selection, those parameters or features were also determined by training data set only. In the online game, the features for decoding the movement intention was extracted and classified using the parameters determined from the dataset recorded with binary control paradigm and four-directional control paradigm.

2.6 Data processing for neurophysiological analysis

Offline data analysis was performed to investigate the neurophysiology following the tasks of ‘Yes’ and ‘No’ for binary classification, and ‘RYes’, ‘RNo’, ‘LYes’ and ‘LNo’ for four-directional classification. The calibration datasets were used for analysis. Data processing was performed using MATLAB Toolbox: BCI2VR. Epoching was done with windows of −2 s to 7 s with respect to the first cue onset. Any epochs contaminated with artifacts were rejected. ERD and ERS were calculated for each case. Epochs were linearly de-trended and divided into 0.25 s segments. The power spectrum of each segment was calculated using FFT with Hamming window resulting in a bandwidth of 4 Hz. ERD and ERS were obtained by averaging the log power spectrum across epochs and having baseline corrected with respect to −2 s to 0 s.

3. Results

3.1 ERD and ERS in ALS/PLS

The time-frequency analysis of EEG activity associated with the binary control paradigm is illustrated in Fig. 4. ERD in blue color was observed in the conditional window from 0 to 2.5 s for both ‘Yes’ and ‘No’ cases. ERD was sustained when subjects (see ALS1 and PLS1) performed the repetitive motor tasks in the Detection window (2.5 s to 5 s) in the ‘Yes’ case, whereas ERS in red color presented in ‘No’ case. Both ERD and ERS were found over the left motor area contralateral to the right hand moved or imagined. In ALS1, ipsilateral ERD over right motor area was also detected showing the involvement of ipsilateral motor cortex in the motor control of right hand. However, ERS was predominantly over the contralateral left hemisphere. The ERD activity occurred in both alpha and beta bands from 8 Hz to 30 Hz, and ERS activity was mainly found in the central beta band around 20-24 Hz. The scalp topography demonstrates the spatial distribution of ERD and ERS in the central beta band. The ERD and ERS during motor imagery in ALS and PLS presented similar spatiotemporal patterns with motor execution, though the amplitudes of ERD and ERS during motor imagery were smaller than those during motor execution.

Fig. 4
ERD and post-beta ERS activities associated with motor execution and motor imagery in binary control paradigm. Both ERD and post-beta ERS were observed in all ALS and PLS patients. For each subject, boxplots illustrate ERD (left), idle (middle) and ERS ...

The sample distributions of single trial EEG activity in central beta band from a contralateral motor area during either motor execution or motor imagery were provided for all six subjects using box plot; the median, the edges of the box are the 25th and 75th percentiles, the whiskers extend to the most extreme data points not considered outliers, and outliers. The median values of ERD were higher than the power values in idle state/baseline, confirming that the frequency power decreased over contralateral motor area during active motor tasks of motor execution and motor imagery. Except for PLS3 with motor imagery task, the median values of ERS in the Detection window after active motor tasks were larger than the values of idle state confirming the power increase after active motor tasks. For PLS3, the ERS during motor imagery was maximized over central-medial area instead of the contralateral left motor area. The different spatial distribution of ERS as well as ERD showed that feature selection was required in order to achieve an optimal classification.

For motor execution in six subjects, the mean difference between ERD and idle state activities was 4.2±1.1 dB, whereas the mean difference between ERS and ERD was 5.6±1.2 dB. For motor imagery in six subjects, the mean difference between ERD and idle state activities was 1.3±1.4 dB, whereas the mean difference between ERS and ERD was 2.7±1.5 dB. The higher difference between ERS and ERD indicated that a better classification could be obtained in the discrimination of ERD from ERS than the discrimination of ERD from idle state activities.

ALS1 and PLS1 participated in the additional study of four-directional control. ERD and ERS associated with motor execution were presented over left and right hemispheres corresponding to right hand and left hand movements as illustrated in Fig. 5. ERD was observed over contralateral hemispheres to the right and left hand for both subjects. Similar to the ERD pattern in binary control paradigm, ipsilateral ERD was also seen in ALS1 during the active motor task. The contralateral ERS after active motor task was clearly seen in PLS1, whereas ERS pattern was not distinguishable in ALS1. In the experiment, ALS1 was not able to cease the motor task as soon as the color was changed. The subject reported that the muscle stiffness delayed her relaxation response. The time-frequency analysis was not presented because the ERD and ERS patterns were not distinguishable.

Fig. 5
ERD and post-beta ERS activity over left and right motor areas associated with motor execution of right and left hand movement in the four-directional control paradigm. ERD was detected over the motor area contralateral to the hand moved in both ALS1 ...

3.2 Feature selection and classification

The spatiotemporal features associated with motor execution and motor imagery were evaluated by Bhattacharyya distance to test the separability for binary classification (Fig. 6) and four-direction classification (Fig. 7). The features with better separability (red color) for binary classification were in central beta band over contralateral motor cortex and central-medial cortex. In PLS1, features over central-medial area provided better binary classification, which was consistent with the distribution of ERS activity in Fig. 4. The Bhattacharyya distance was employed to determine the best feature to create ROC and was used for online classification in the binary control of cursor movement game. For example, the power feature with centre frequency of 21 Hz from electrode C1 had the highest Bhattacharyya distance value and was used for classification in ALS1 with motor execution. From the sample distribution obtained from single-trial activity of ERD and ERS, a perfect discrimination could be achieved when an appropriate threshold was set. The spatiotemporal distribution from Bhattacharyya distance further confirmed that beta activity over contralateral motor cortex might provide better feature for a reliable binary control.

Fig. 6
Bhattacharyya distance for selecting better spatiotemporal features for binary classification. The features with higher Bhattacharyya distance values (red color) were found in beta band over contralateral left motor areas. In motor imagery, frequency ...
Fig. 7
Bhattacharyya distance for selecting better spatiotemporal features for four-directional classification. The frequency power features over left motor areas in beta band provided better detection of ‘RYes’ and ‘RNo’ associated ...

The Bhattacharyya distance was also analyzed from the activity associated with four-directional control paradigm. Fig. 7 shows the Bhattacharyya distance values in ALS 1 and PLS1 who performed motor execution. In consistence with the ERD and ERS patterns presented in Fig. 5, the better features to classify one of four directions from the other three directions were the activities in the beta band over the motor cortex contralateral to the hand moved. The EEG activities over central-medial area in PLS1 also provided good discrimination of ‘LYes’ and ‘LNo’.

The classification performance was evaluated offline using 10-fold cross-validation and online using the cursor control game. Table 2 shows the offline performance of binary control paradigm and online performance in cursor control game. In motor execution, the average performance using simple ROC methods achieved accuracy of 82.1%, whereas higher average accuracy of 91.4% was achieved by a better feature selection and classification method. The performance of ROC, GA-MLD, and GA-SVM varied among subjects. A better classification method was unable to be determined when considering the variance in 10-fold cross-validation. ALS3 provided the lowest performance among six subjects. This subject reported that she was unable to stop movement in time because of excessive muscle stiffness, where EMG activities also verified the delayed response in relaxation after active motor execution. In the motor execution session, PLS2 played the game with motor imagery at the beginning, and after a few moves, she realized that she did not need to make actual movements but only imagined movement to achieve the cursor control (no recognizable EMG activity).

Table 2
Decoding binary movement intention from ERD and beta-ERS associated with human natural motor behavior

The performance associated with motor imagery achieved average accuracy of 83.3%, which was lower than that with motor execution. Although the subjects were instructed to attempt vivid motor imagery of repetitive wrist extension, they all reported difficulty in imagination. This might be one of the reasons that performance in motor imagery was lower. ALS1 and ALS2 played the cursor control game with motor imagery. The online control accuracy was estimated from the cursor movement traces because subjects determined the movement directions by themselves, and no objective information such as EMG activity during motor execution was available to determine the actual direction that the subjects wanted. The online motor imagery accuracy was determined whether the cursor movement was moved towards the target or away from the target. None of the patients complained of mental fatigue after the motor imaged-based calibration and testing with computer game.

From the comparison of simple and more complicated computational algorithms for binary classification of ERD and ERS, the GA-MLD provided overall better classification performance as shown in Table 2. We employed GA-MLD to classification between ERD in the ‘Detection’ window and baseline activity from 1 s before to the start of the ‘Condition’ window. The classification accuracy between ERD and baseline in six patients were: 91.7%, 94.7%, 75.3%, 81.7%, 85.0% and 70.0% for motor execution and 70.0%, 80.1%, 74.7%, 75.1%, 80.0%, and 65.1% for motor imagery. For motor execution, the average performance in the six patients was improved 8.3% when employing the proposed ERD/ERS approach (91.4±3.4%) than the conventional ERD/baseline approach (83.1±9.5%). For motor imagery, the average performance in the six patients was improved 9.2% when employing the proposed ERD/ERS approach (83.3±8.9%) than the conventional ERD/baseline approach (74.1±5.8%).

The four-directional classification performance was provided in Table 3. The four-direction classification accuracy was about 60%, which was much higher than the random level of 25% in the case of 4-class discrimination. The subjects also reported that it was more difficult to imagine wrist movement of the non-dominant left hand than the dominant right hand. An appropriate training to teach effective motor imagery maybe necessary for this motor imagery task. The online game provided a better accuracy than that of offline analysis of data recorded using the four-direction control paradigm. A possible reason might be that subjects were more actively involved with the interactive game than the paradigm without performance feedback. Further, subjects might be able to adapt to the computer model for the classification from the cursor movement feedback.

Table 3
Decoding four-directional movement intention from lateral ERD and beta-ERS associated with human natural motor behavior

4. Discussions

We analyzed ERD and ERS activity from EEG associated with human natural motor control in ALS and PLS patients. ERD associated with active motor control and post-beta ERS associated with cessation of active motor control were preserved in all six ALS and PLS patients participating in this study. ERD and ERS occurred not only with motor execution with physical movement, but also with motor imagery without overt movement. Similar to our previous findings with healthy volunteers (Bai et al. 2008; Huang et al. 2009), the amplitudes of ERD and ERS were less with motor imagery than during motor execution. There are at least two possible factors related to the physiology of motor control or cognitive performance.

The ERD and ERS over sensori-motor cortex during the production of voluntary physical movement are comprised of a top-down motor efferent component and bottom-up sensory re-afferent component (Alegre et al. 2002). Because the sensory re-afferent component is not available during motor imagery of covert movement, the amplitudes of ERD and ERS will presumably be lower than those with overt motor execution. Therefore, the discrimination between ERD and ERS will be more difficult with motor imagery than motor execution. This was one of the factors that affected classification accuracy in healthy subjects (Kayagil et al. 2009) and is redemonstrated in ALS/PLS patients in this study.

Another possible factor is related to the cognitive performance of motor imagery. Motor imagery is the ability to create a vivid imagination of movement (Malouin et al. 2008). Marks and Isaac measured ERD activity from vivid and non-vivid imagers (Marks and Isaac 1995) and found enhanced ERD in subjects with higher motor imagery vividness indicated by the Vividness of Movement Imagery Questionnaire (VMIQ) (Isaac and Marks 1994). Other studies suggested that the vividness of motor imagery could be improved by efficient motor imagery teaching (Braun et al. 2008; Verbunt et al. 2008). In this study, we instructed the patients to perform mtor imagery of their own wrist extension in the first person (kinesthetic motor imagery), and at the same time, to imagine a virtual sensory feedback although keeping their wrist still. Most of the patients reported that they could imagine their dominant right hand movement well after the simple motor imagery instruction, although a few subjects still reported that they could not imagine the movement vividly. It was more difficult to imagine both dominant right hand and non-dominant left hand movement in ALS 1 and PLS1. Both reported that they had difficulty switching from one hand to another hand. Exploration of effective teaching of motor imagery is needed in future studies.

We further confirmed that the difference between ERS and ERD provided better contrast than the difference between idle state or baseline activities and ERD in ALS and PLS patients. The better contrast provided better classification rate by reducing the inter-class pattern overlapping. In this study, none of the patients received BCI-related training for rhythm regulation. Under the proposed ERD and ERS-based paradigm, subjects achieved a high accuracy of binary control (80-90% for motor execution/motor imagery) despite not receiving extensive training. The accuracy for four-directional control was also much higher than the random level, though further training of effective motor imagery of right and left hand might be required. The successful test on the ERD and ERS-based method associated with human natural motor control will promote the development of a practical user-friendly BCI because long-term training becomes unnecessary. This is important for severely affected patients who are unable to tolerate prolonged training. Further, users may not need to keep sustained attention to regulate EEG rhythm in the proposed BCI associated with human natural motor control. We expect that the fatigue from sustained attention might also be reduced in the proposed BCI.

The peri-movement ERD is usually near-continuous when movement is maintained, while the ERS is always transient and occurs after the termination of movement only. Therefore, the use of an ERD/ERS approach may limit the use when compared to an ERD/baseline approach. The ERD/ERS approach requires that the BCI system can reliably detect the transient ERS feature for BCI control. A possible solution was provided in this manuscript; we proposed a paradigm with ‘Condition’ and ‘Detection’ window that may effectively detect the potential ERS.

A previous study showed that the beta ERS was smaller when subjects were forced to stop a movement than when subjects stopped the movement spontaneously (Alegre et al. 2008). This result suggested the potential lack of ERS in some cases when patients were instructed to stop movement when perceiving a certain visual/auditory cue that might limit the advantages of the ERD/ERS approach for BCI control. In order to foster the efficiency of the proposed approach, the length of active ‘Condition’ window should be tuned as if the patients stop the movement spontaneously without the notice of the visual cue. This provides another factor that should be optimized in future development.

In this study, we tested potential BCI control in a small group of less clinically affected ALS and PLS patients. Further study on a larger population of patients, in particular, a study on significantly affected patients including ‘locked-in’ patients will be needed. PLS disease is rarer than ALS disease. Though the two patient groups have the commonality of the involvement of descending tracts, PLS provides a better test of whether the EEG signal for potential BCI control can be detected since PLS patients have been shown to have a loss of movement-related cortical potentials compared to patients who have predominantly lower motor neuron degeneration.

The major aim in this study was to investigate whether ALS and PLS patients might achieve binary BCI control without extensive training. The secondary aim was to test potential multi-dimensional BCI control in ALS and PLS patients. We provided initial test results for only two patients in this pilot study. More patients will be studied in our future investigator in order to provide more solid evidence of potential BCI application in ALS and PLS patients.


This research was partly supported by the Intramural Research Program of the NIH, National Institute of Neurological Disorders and Stroke. O. Bai received partial support from State Key Laboratory of Digital Manufacturing, Equipment & Technology, Huazhong University of Science and Technology. The authors would like to thank Dr. Mark Hallett for his valuable suggestions and Dr. Nobue Iwata for her coordination with patients.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.


  • Alegre M, Alvarez-Gerriko I, Valencia M, Iriarte J, Artieda J. Oscillatory changes related to the forced termination of a movement. Clin Neurophysiol. 2008;119:290–300. [PubMed]
  • Alegre M, Labarga A, Gurtubay IG, Iriarte J, Malanda A, Artieda J. Beta electroencephalograph changes during passive movements: sensory afferences contribute to beta event-related desynchronization in humans. Neurosci Lett. 2002;331:29–32. [PubMed]
  • Bai O, Lin P, Vorbach S, Floeter MK, Hattori N, Hallett M. A high performance sensorimotor beta rhythm-based brain-computer interface associated with human natural motor behavior. J Neural Eng. 2008;5:24–35. [PubMed]
  • Bai O, Lin P, Vorbach S, Li J, Furlani S, Hallett M. Exploration of computational methods for classification of movement intention during human voluntary movement from single trial EEG. Clin Neurophysiol. 2007;118:2637–2655. [PubMed]
  • Bashashati A, Fatourechi M, Ward RK, Birch GE. A survey of signal processing algorithms in brain-computer interfaces based on electrical brain signals. J Neural Eng. 2007;4:R32–57. [PubMed]
  • Birbaumer N. Brain-computer-interface research: coming of age. Clin Neurophysiol. 2006a;117:479–483. [PubMed]
  • Birbaumer N. Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. Psychophysiology. 2006b;43:517–532. [PubMed]
  • Birbaumer N, Cohen LG. Brain-computer interfaces: communication and restoration of movement in paralysis. J Physiol. 2007;579:621–636. [PMC free article] [PubMed]
  • Birbaumer N, Kubler A, Ghanayim N, Hinterberger T, Perelmouter J, Kaiser J, Iversen I, Kotchoubey B, Neumann N, Flor H. The thought translation device (TTD) for completely paralyzed patients. IEEE Trans Rehabil Eng. 2000;8:190–193. [PubMed]
  • Birch GE, Bozorgzadeh Z, Mason SG. Initial on-line evaluations of the LF-ASD brain-computer interface with able-bodied and spinal-cord subjects using imagined voluntary motor potentials. IEEE Trans Neural Syst Rehabil Eng. 2002;10:219–224. [PubMed]
  • Blankertz B, Dornhege G, Krauledat M, Muller KR, Curio G. The non-invasive Berlin Brain-Computer Interface: fast acquisition of effective performance in untrained subjects. Neuroimage. 2007;37:539–550. [PubMed]
  • Braun S, Kleynen M, Schols J, Schack T, Beurskens A, Wade D. Using mental practice in stroke rehabilitation: a framework. Clin Rehabil. 2008;22:579–591. [PubMed]
  • Bromberg MB. Quality of life in amyotrophic lateral sclerosis. Phys Med Rehabil Clin N Am. 2008;19:591–605. x–xi. [PubMed]
  • Cassim F, Monaca C, Szurhaj W, Bourriez JL, Defebvre L, Derambure P, Guieu JD. Does post-movement beta synchronization reflect an idling motor cortex? Neuroreport. 2001;12:3859–3863. [PubMed]
  • Cedarbaum JM, Stambler N, Malta E, Fuller C, Hilt D, Thurmond B, Nakanishi A. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. BDNF ALS Study Group (Phase III) J Neurol Sci. 1999;169:13–21. [PubMed]
  • Chang CC, Lin CJ. Training nu-support vector classifiers: theory and algorithms. Neural Comput. 2001;13:2119–2147. [PubMed]
  • Chatterjee A, Aggarwal V, Ramos A, Acharya S, Thakor NV. A brain-computer interface with vibrotactile biofeedback for haptic information. J Neuroeng Rehabil. 2007;4:40. [PMC free article] [PubMed]
  • Daly JJ, Wolpaw JR. Brain-computer interfaces in neurological rehabilitation. Lancet Neurol. 2008;7:1032–1043. [PubMed]
  • Delis DC, Kramer JH, Kaplan E, Holdnack J. Reliability and validity of the Delis-Kaplan Executive Function System: an update. J Int Neuropsychol Soc. 2004;10:301–303. [PubMed]
  • Fatourechi M, Bashashati A, Ward RK, Birch GE. EMG and EOG artifacts in brain computer interface systems: A survey. Clin Neurophysiol. 2007;118:480–494. [PubMed]
  • Hayashi H, Oppenheimer EA. ALS patients on TPPV: totally locked-in state, neurologic findings and ethical implications. Neurology. 2003;61:135–137. [PubMed]
  • Hill NJ, Lal TN, Schroder M, Hinterberger T, Wilhelm B, Nijboer F, Mochty U, Widman G, Elger C, Scholkopf B, Kubler A, Birbaumer N. Classifying EEG and ECoG signals without subject training for fast BCI implementation: comparison of nonparalyzed and completely paralyzed subjects. IEEE Trans Neural Syst Rehabil Eng. 2006;14:183–186. [PubMed]
  • Hjorth B. An on-line transformation of EEG scalp potentials into orthogonal source derivations. Electroencephalogr Clin Neurophysiol. 1975;39:526–530. [PubMed]
  • Huang D, Lin P, Fei DY, Chen X, Bai O. Decoding human motor activity from EEG single trials for a discrete two-dimensional cursor control. J Neural Eng. 2009;6:46005. [PubMed]
  • Isaac AR, Marks DF. Individual differences in mental imagery experience: developmental changes and specialization. Br J Psychol. 1994;85(Pt 4):479–500. [PubMed]
  • Iversen I, Ghanayim N, Kubler A, Neumann N, Birbaumer N, Kaiser J. Conditional associative learning examined in a paralyzed patient with amyotrophic lateral sclerosis using brain-computer interface technology. Behav Brain Funct. 2008a;4:53. [PMC free article] [PubMed]
  • Iversen IH, Ghanayim N, Kubler A, Neumann N, Birbaumer N, Kaiser J. A brain-computer interface tool to assess cognitive functions in completely paralyzed patients with amyotrophic lateral sclerosis. Clin Neurophysiol. 2008b;119:2214–2223. [PubMed]
  • Jasper HH, Andrews HL. Electro-encephalography. III. Normal differentiation of occipital and precentral regions in man. Arch Neurol Psychiat. 1938;39:95–115.
  • Joseph AB. Design considerations for the brain-machine interface. Med Hypotheses. 1985;17:191–195. [PubMed]
  • Kayagil TA, Bai O, Henriquez CS, Lin P, Furlani SJ, Vorbach S, Hallett M. A binary method for simple and accurate two-dimensional cursor control from EEG with minimal subject training. J Neuroeng Rehabil. 2009;6:14. [PMC free article] [PubMed]
  • Keerthi SS, Lin C. Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation. 2003;15:1667–1689. [PubMed]
  • Kennedy PR, Bakay RA, Moore MM, Adams K, Goldwaithe J. Direct control of a computer from the human central nervous system. IEEE Trans Rehabil Eng. 2000;8:198–202. [PubMed]
  • Kubler A, Kotchoubey B, Hinterberger T, Ghanayim N, Perelmouter J, Schauer M, Fritsch C, Taub E, Birbaumer N. The thought translation device: a neurophysiological approach to communication in total motor paralysis. Exp Brain Res. 1999;124:223–232. [PubMed]
  • Kubler A, Neumann N, Kaiser J, Kotchoubey B, Hinterberger T, Birbaumer NP. Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. Arch Phys Med Rehabil. 2001;82:1533–1539. [PubMed]
  • Li Q, Doi K. Analysis and minimization of overtraining effect in rule-based classifiers for computer-aided diagnosis. Med Phys. 2006;33:320–328. [PubMed]
  • Lule D, Zickler C, Hacker S, Bruno MA, Demertzi A, Pellas F, Laureys S, Kubler A. Life can be worth living in locked-in syndrome. Prog Brain Res. 2009;177:339–351. [PubMed]
  • Malouin F, Richards CL, Durand A, Doyon J. Reliability of mental chronometry for assessing motor imagery ability after stroke. Arch Phys Med Rehabil. 2008;89:311–319. [PubMed]
  • Marks DF, Isaac AR. Topographical distribution of EEG activity accompanying visual and motor imagery in vivid and non-vivid imagers. Br J Psychol. 1995;86(Pt 2):271–282. [PubMed]
  • McFarland DJ, Sarnacki WA, Vaughan TM, Wolpaw JR. Brain-computer interface (BCI) operation: signal and noise during early training sessions. Clin Neurophysiol. 2005;116:56–62. [PubMed]
  • Mockford C, Jenkinson C, Fitzpatrick R. A Review: carers, MND and service provision. Amyotroph Lateral Scler. 2006;7:132–141. [PubMed]
  • Muller KR, Mika S, Ratsch G, Tsuda K, Scholkopf B. An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw. 2001;12:181–201. [PubMed]
  • Mutsaarts M, Steenbergen B, Meulenbroek R. A detailed analysis of the planning and execution of prehension movements by three adolescents with spastic hemiparesis due to cerebral palsy. Exp Brain Res. 2004;156:293–304. [PubMed]
  • Neuper C, Pfurtscheller G. Evidence for distinct beta resonance frequencies in human EEG related to specific sensorimotor cortical areas. Clin Neurophysiol. 2001;112:2084–2097. [PubMed]
  • Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9:97–113. [PubMed]
  • Pfurtscheller G. Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest. Electroencephalogr Clin Neurophysiol. 1992;83:62–69. [PubMed]
  • Pfurtscheller G, Lopes da Silva FH. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clin Neurophysiol. 1999;110:1842–1857. [PubMed]
  • Pfurtscheller G, Neuper C, Brunner C, da Silva FL. Beta rebound after different types of motor imagery in man. Neurosci Lett. 2005;378:156–159. [PubMed]
  • Pfurtscheller G, Stancak A, Jr., Neuper C. Post-movement beta synchronization. A correlate of an idling motor area? Electroencephalogr Clin Neurophysiol. 1996;98:281–293. [PubMed]
  • Salmelin R, Hamalainen M, Kajola M, Hari R. Functional segregation of movement-related rhythmic activity in the human brain. Neuroimage. 1995;2:237–243. [PubMed]
  • Schmidt R, Freidl W, Fazekas F, Reinhart B, Grieshofer P, Koch M, Eber B, Schumacher M, Polmin K, Lechner H. The Mattis Dementia Rating Scale: normative data from 1,001 healthy volunteers. Neurology. 1994;44:964–966. [PubMed]
  • Sellers EW, Donchin E. A P300-based brain-computer interface: initial tests by ALS patients. Clin Neurophysiol. 2006;117:538–548. [PubMed]
  • Sellers EW, Krusienski DJ, McFarland DJ, Vaughan TM, Wolpaw JR. A P300 event-related potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance. Biol Psychol. 2006;73:242–252. [PubMed]
  • Stout JC, Ready RE, Grace J, Malloy PF, Paulsen JS. Factor analysis of the frontal systems behavior scale (FrSBe) Assessment. 2003;10:79–85. [PubMed]
  • Sykacek P, Roberts S, Stokes M, Curran E, Gibbs M, Pickup L. Probabilistic methods in BCI research. IEEE Trans Neural Syst Rehabil Eng. 2003;11:192–195. [PubMed]
  • Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB. Cortical control of a prosthetic arm for self-feeding. Nature. 2008;453:1098–1101. [PubMed]
  • Verbunt JA, Seelen HA, Ramos FP, Michielsen BH, Wetzelaer WL, Moennekens M. Mental practice-based rehabilitation training to improve arm function and daily activity performance in stroke patients: a randomized clinical trial. BMC Neurol. 2008;8:7. [PMC free article] [PubMed]
  • Williams MT, Donnelly JP, Holmlund T, Battaglia M. ALS: Family caregiver needs and quality of life. Amyotroph Lateral Scler. 2008;9:279–286. [PubMed]
  • Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002;113:767–791. [PubMed]
  • Wolpaw JR, McFarland DJ. Multichannel EEG-based brain-computer communication. Electroencephalogr Clin Neurophysiol. 1994;90:444–449. [PubMed]
  • Wolpaw JR, McFarland DJ. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc Natl Acad Sci U S A. 2004;101:17849–17854. [PMC free article] [PubMed]


Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


  • MedGen
    Related information in MedGen
  • PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...