pmc logo image
Logo of frontintegneuroJournal URL: http://frontiersin.org/integrativeneuroscience/
Journal List > Front Integr Neurosci > v.3; 2009

Formats:
Front Integr Neurosci. 2009; 3: 3.
Published online 2009 March 9. Prepublished online 2009 February 19. doi: 10.3389/neuro.07.003.2009.
PMCID: PMC2659168
Copyright © 2009 Fitzsimmons, Lebedev, Peikon and Nicolelis.
Extracting Kinematic Parameters for Monkey Bipedal Walking from Cortical Neuronal Ensemble Activity
Nathan A. Fitzsimmons,1,2 Mikhail A. Lebedev,1,2 Ian D. Peikon,2,3 and Miguel A. L. Nicolelis1,2,3,4,5,6*
1Department of Neurobiology, Duke University, Durham, NC, USA
2Center for Neuroengineering, Duke University, Durham, NC, USA
3Biomedical Engineering, Duke University, Durham, NC, USA
4Psychology and Neuroscience, Duke University, Durham NC, USA
5Edmond and Lily Safra International Institute of Neuroscience of Natal, Natal, Rio Grande do Norte, Brazil
6Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
Edited by: Mark Laubach, The John B. Pierce Laboratory, USA; Yale University School of Medicine, USA
Reviewed by: Bruno Averbeck, Institute of Neurology, UCL, UK; Marshall Shuler, Johns Hopkins University, USA
*Correspondence: Miguel Nicolelis, Department of Neurobiology, Duke University Medical Center, Room 327E, Bryan Research Building, Box 3209, Durham, NC 27710, USA. e-mail: nicoleli/at/neuro.duke.edu
Received February 3, 2009; Accepted February 23, 2009.
This is an open-access article subject to an exclusive license agreement between the authors and the Frontiers Research Foundation, which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are credited.
Abstract
The ability to walk may be critically impacted as the result of neurological injury or disease. While recent advances in brain–machine interfaces (BMIs) have demonstrated the feasibility of upper-limb neuroprostheses, BMIs have not been evaluated as a means to restore walking. Here, we demonstrate that chronic recordings from ensembles of cortical neurons can be used to predict the kinematics of bipedal walking in rhesus macaques – both offline and in real time. Linear decoders extracted 3D coordinates of leg joints and leg muscle electromyograms from the activity of hundreds of cortical neurons. As more complex patterns of walking were produced by varying the gait speed and direction, larger neuronal populations were needed to accurately extract walking patterns. Extraction was further improved using a switching decoder which designated a submodel for each walking paradigm. We propose that BMIs may one day allow severely paralyzed patients to walk again.
Keywords: neuronal ensemble recordings, brain–machine interface, primate, sensorimotor, locomotion, neuroprosthetics
Introduction
Bipedal locomotion control is of great interest to the field of brain–machine interfaces (BMIs), i.e. devices that utilize neural activity to control limb prostheses (Chapin, 2004; Fetz, 2007; Lebedev and Nicolelis, 2006; Nicolelis, 2001; Schwartz et al., 2006; Taylor et al., 2002). Since locomotion deficits are commonly associated with spinal cord injury (Dietz, 2001; Dietz and Colombo, 2004; Rossignol et al., 2007; Scivoletto and Di Donna, 2008; Wood-Dauphinee et al., 2002) and neurodegenerative diseases (Boonstra et al., 2008; Green and Hurvitz, 2007; Morris, 2006; Pearson et al., 2004; Sparrow and Tirosh, 2005; Yogev-Seligmann et al., 2008), there is a need to seek new potential therapies to restore gait control in such patients. While the feasibility of a BMI for upper limbs has been demonstrated in studies in monkeys (Carmena et al., 2003, 2005; Serruya et al., 2002; Taylor et al., 2002; Velliste et al., 2008; Wessberg et al., 2000) and humans (Hochberg et al., 2006; Patil et al., 2004), it remains unknown whether BMIs could aid patients suffering from lower limb paralysis, e.g. by driving a leg prosthesis or artificial exoskeleton (Fleischer et al., 2006; Hesse et al., 2003; Veneman et al., 2007).
Pioneered by Borelli (Borelli, 1680), investigations in biological systems have generated a wealth of knowledge about the biomechanics (Alexander, 2004; Andriacchi and Alexander, 2000; Dickinson et al., 2000; Koditschek et al., 2004; Ounpuu, 1994; Saibene and Minetti, 2003; Stevens, 2006; Vaughan, 2003; Zajac et al., 2002; Zatsiorky et al., 1994) and neurophysiological mechanisms underlying locomotion (Beloozerova et al., 2003; Deliagina et al., 2008; Drew et al., 2004; Georgopoulos and Grillner, 1989; Grillner, 2006; Grillner and Wallen, 2002; Grillner et al., 2008; Hultborn and Nielsen, 2007; Kagan and Shik, 2004; Orlovsky et al., 1999; Takakusaki, 2008). Many such biological principles are being applied to robotic locomotion (Azevedo et al., 2007; Kimura et al., 2007; Morimoto et al., 2008; Nakanishi et al., 2004).
A neuroprosthetic for the restoration of locomotion could be designed in several different ways. For example, a very simplified interface could be built, extracting only speed and directional information from neural activity, and offloading all precise movement control to onboard computerized systems. However, such a neuroprosthetic would not offer much more functionality to the user than a motorized wheelchair. Alternatively, a far more ambitious neuroprosthetic could attempt to extract control signals for every articulated joint in the robotic prosthetic, trusting the user to learn to control every aspect of the limb's usage. However, much of balance and postural adjustments involve involuntary mechanisms (Deliagina and Orlovsky, 2002; Grillner et al., 2007; Maki and McIlroy, 2007), and the lack of perfect decoding and sensory feedback could result in falls and injuries. Our approach takes the middle ground, if we can decode key walking parameters: step time, step length, foot location, and leg orientation, while offloading other automatic-level controls: foot orientation, load placement, balance, and safety concerns to onboard computerized systems, then we can achieve a BMI that follows the general commands of the user while enforcing stability, and overriding motions and configurations likely to result in falls.
We have therefore extended our laboratory's BMI approach (Carmena et al., 2003; Lebedev and Nicolelis, 2006; Wessberg et al., 2000) to investigate whether cortical activity can be utilized to extract the kinematics of bipedally walking rhesus macaques. Although macaques are quadrupeds (Chatani, 2003; Courtine et al., 2005a), they can be trained to walk bipedally (Hirasaki et al., 2004; Matsuyama et al., 2004; Mori et al., 2001, 2004; Tachibana et al., 2003). In the present study, rhesus macaques walked bipedally on a treadmill while neuronal ensemble activity was recorded from the representation of the lower limbs in the primary motor (M1) and somatosensory (S1) cortices. We confirmed that a BMI using a series of independent linear decoders can accurately extract walking patterns from the activity of multiple cortical areas. As the locomotion task demands increased, significantly more neurons were needed to achieve accurate extraction. Finally, we demonstrated that locomotor parameters can be extracted in real time to control artificial actuators that reproduce walking patterns.
Materials and Methods
Experimental setup
All studies were conducted with approved protocols from the Duke University Institutional Animal Care and Use Committee and were in accordance with the NIH guidelines for the Care and Use of Laboratory Animals.
Two adult female rhesus macaques were trained to walk bipedally on a custom modified treadmill. The treadmill was a human fitness treadmill, modified to be hydraulically driven so that the pump motor could be located remotely, in a different room than the experimental setup and thus reducing the electronic noise that could be picked up by the neuronal recording equipment. Treadmill speed and direction were separately controlled via independent throttle and flow routing controls.
Around the treadmill was a metal frame that supported both the recording equipment and the monkey restraints. The monkeys were loosely restrained by an adjustable 5-degree of freedom neckplate. By adjusting the three dimensional position of the neckplate as well as two dimensional tilt, the monkeys could be comfortable in their normal walking posture. Additionally, a bar was placed within reach of the monkeys' arms to allow for a comfortable stability aid. Each of the monkey's legs and arms were unrestrained during experimental walking sessions. After 1 month of training, the monkeys learned to walk bipedally on the treadmill (Figure (Figure1A).1Figure 1A). Monkey 1 was trained to walk in both the forward and backward direction at varying speeds (Figure (Figure1C),1Figure 1C), and Monkey 2 was trained to walk only in the forward direction. The treadmill speed varied in the range −0.5 to 0.5 m/s. The monkeys received food treats during walking sessions lasting 40–60 min, which encouraged them to face to the front of the treadmill.
Figure 1
Figure 1
Figure 1
Experimental setup and Kinematic Analysis. (A) Diagram of the bipedal walking setup, consisting of a custom modified, hydraulically driven treadmill; wireless, 2-camera tracking of kinematics; and a MAP neural acquisition system (Plexon, Inc., TX, USA). (more ...)
Limb movement tracking
During experimental sessions, movements of the right legs of the monkeys were tracked using a wireless, video-based tracking system. The three dimensional coordinates of fluorescent markers applied to the hip, knee, and ankle were tracked using two cameras at a 30 Hz frame rate. Initially a commercial offline system (SIMI Motion 3D) was used for tracking purposes; in later sessions a custom real-time video-based tracking system developed in the lab was used (Peikon et al., 2007). Data were cross-validated between the two systems to ensure that both produced equally accurate results and to ensure no biases were present in our custom system. To ensure consistent placement of the markers which were tracked on video, each of the monkeys was tattooed over the hip, knee, and ankle joints of their right legs. The markers themselves were applied before each session, and were made of fluorescent, non-toxic stage makeup. This approach offered several advantages. First, the markers were virtually weightless on the skin of the monkeys and thus did not impact the walking mechanics or cause distraction. Second, using fluorescent markers allowed us to achieve very high contrast ratios between markers and background on the recorded video stream. By turning visible light down to a low level, lowering the camera aperture, and bathing the experimental setup with a safe, filtered UV (or black) spotlight, the only significant source of visible light picked up by the cameras came from the markers which were being tracked. While the current tracking system is only capable of tracking the right leg, future versions will track both legs simultaneously, allowing us to investigate bipedal predictions. Figure Figure1B1Figure 1B illustrates frames of captured video that show an example of the bipedal walk cycle for each monkey. These figures have increased levels of ambient lighting to aid in viewing the step cycle. Both monkeys walked in a stereotyped bipedal manner and slightly leaned forward for comfort with arms holding a bar for support.
Kinematic analysis
The monkeys' limb tracking information was used to extract a number of experimentally relevant parameters in addition to the X, Y and Z coordinates of the joint markers. We extracted joint angles (hip and knee), foot contact with the treadmill, walking speed, step frequency and step length. All these parameters were calculated from the joint position data provided that the markers were not occluded. The episodes during which any of the markers was occluded (typically, when the monkeys turned their bodies) were excluded from the analyses. Our video analysis algorithm used a combination of mathematical techniques to calculate additional parameters. Treadmill speed was extracted from the video tracking data. The frequency of the step was extracted from the kinematic movement pattern of the ankle in the axis of treadmill motion. Furthermore, a combination of ankle displacement during the step phase and treadmill speed was used to extract stride length with respect to the moving surface of the treadmill. Foot contact was extracted first from a filtered height (Y-axis) threshold, which determined when the ankle joint was flat on the surface of the treadmill. However, a simple height threshold was not adequate to reject shuffling walking types, and a further check was instituted, which rejected periods without constant X-velocity of the ankle equal to treadmill speed.
Surgical and electrophysiological procedures
During the implantation surgeries, each of the two adult female rhesus macaques was anesthetized and placed in a stereotaxic apparatus (for a full description see Nicolelis et al. 2003). All surgical procedures conformed to the National Research Council's Guide for the Care and Use of Laboratory Animals (1996) and were approved by the Institutional Animal Care and Use Committee.
A series of small craniotomies were made, both to grant access to the brain for the microwire arrays and for anchoring the dental acrylic to the skull. In each animal, multiple microwire arrays were chronically implanted in several cortical areas (Figure (Figure2).2Figure 2). The implanted areas were chosen based on previous cortical mapping studies (Nicolelis et al., 2003; Tanji and Wise, 1981; Wise and Tanji, 1981a,b).
Figure 2
Figure 2
Figure 2
Diagram of cortical implants. A generic outline of cortical sulci (view from the top) is shown to illustrate the locations of the implants. Positive X coordinates correspond to rostral brain areas, while negative X coordinates correspond to caudal brain (more ...)
Accordingly, multi-electrode arrays were inserted medially in the cortex approximately 2 mm in front of the central sulcus to target M1 and 2 mm behind it to target S1 (Figure (Figure2).2Figure 2). The arrays were matched in size to the cortical representations of the hind limb. Monkey 1 was implanted with 50 μm tungsten chronic microwire arrays in the leg representation of both the primary somatosensory (S1) and primary motor (M1) cortices. One 8 by 8 square array with electrodes spaced 1 mm apart (inserted 2.5 mm deep in the cortex) was implanted rostral to the central sulcus. A 6 by 6 double-layered square array, using the same electrode spacing, was placed caudal to the central sulcus, 1.5 mm deep. In the double layer implant, the second layer of electrodes was 300–400 μm shallower than the first layer. Monkey 2 was also implanted with chronic microwire arrays in the leg representation of both the primary somatosensory (S1) and primary motor (M1) cortices. In M1, two 3 by 6 double layer arrays were implanted, while in S1 a single double layer 3 by 6 array was implanted. In all implants one layer was 300 μm deeper than the other layer. All electrodes in Monkey 2 were stainless steel microwires ranging in diameter from 40 to 60 μm. In both monkeys the placement of the electrodes was accomplished using stereotaxic coordinates and connectors for the arrays were embedded in a head cap made of dental acrylic.
Upon recovery from the surgical procedure, the receptive fields of individual S1 neurons and multi-unit activity were briefly examined in the awake monkeys by lightly touching and palpating their hind limbs. This examination confirmed that in both monkeys the implanted S1 sites represented the thigh, calf and the foot. No clear somatosensory responses were identified for the M1 implants. In Monkey 2, we briefly tested motor responses to cortical microstimulation under ketamine anesthesia. The microstimulation of M1 evoked hind limb movements due to proximal muscle contraction in agreement with Hatanaka et al. (2001) and Tanji and Wise (1981).
A total of 200–300 well sorted single units were recorded from implants in both monkeys per experimental session. After the monkeys were placed in the treadmill-mounted restraint, head stage amplifiers were attached to the head-cap connectors. A flexible wire harness, in turn, connected the headstages to a 128 channel Multichannel Acquisition Processor, or MAP, (Plexon, Inc., TX, USA) recording system. Neuronal units were sorted in real time using the templates defined in MAP software. The ratio of the amplitude of each sorted unit to the amplitude of electrical noise was, on average 3.34 ± 1.66 (mean ± standard error). The quality of online sorted single units was further examined by analyzing the refractory period, estimated from the interspike intervals. For each unit to be qualified as single unit, in addition to having a distinct shape and amplitude (Nicolelis et al., 2003), it had to exhibit a refractory period greater than 1.6 ms (Hatsopoulos et al., 2004). Using these criteria, 66.0% of the recorded units were single units, and 34.0% were classified as multi-unit neuronal activity. Overall, extraction of locomotion patterns from single units versus multi-units yielded similar results.
Electrophysiological recordings spanned 399 days in Monkey 1 and 56 days in Monkey 2. The implants were connected to a multichannel recording system (Plexon, Inc., TX, USA) using light flexible cables. The total number of simultaneously recorded units ranged from 180 to 238 in Monkey 1, depending on the recording day, and from 173 to 334 units in Monkey 2. In Monkey 1, we recorded from 111.2 ± 19.3 units (mean ± standard deviation; standard deviation reflects day to day variability) in left M1, from 38.1 ± 6.1 units in left S1 and from 55.6 ± 16.0 units in right M1. In Monkey 2, we recorded from 106.0 ± 19.9 units in left M1 and from 166.0 ± 31.0 units in left S1. Statistical analysis (Wilcoxon signed rank test) confirmed that average neuronal firing rates increased during walking in both M1 (P < 0.001) and S1 (P < 0.001) in each monkey. The average firing rate of M1 units was 7.5 ± 8.9 spikes/s during standing versus 15.0 ± 13.4 spikes/s during walking in Monkey 1, and 7.7 ± 8.7 spikes/s versus 11.4 ± 11.9 spikes/s in Monkey 2. In S1, the average rates were 8.7 ± 7.7 spikes/s during standing versus 16.6 ± 10.3 spikes/s during walking in Monkey 1, and 14.7 ± 12.9 spikes/s versus 24.3 ± 20.3 spikes/s in Monkey 2.
The MAP system was used for receptive field testing and for obtaining neuronal recordings during walking sessions. Electromyogram (EMG) signals were recorded from Monkey 1's shaved skin surface, centered over the soleus, rectus femoris, and tibialis anterior muscles of both legs. Gold disc electrodes (Grass Instrument Co., RI, USA) were placed over conductive gel to obtain the EMG recordings. These EMGs were amplified up 10,000 times, band-pass filtered between 100 Hz and 1 kHz, rectified, and recorded using the MAP recording system to ensure consistent timing.
Models utilized for predicting leg kinematics
All the leg kinematic parameters extracted were reconstructed from neuronal ensemble activity using the linear decoding algorithm called the Wiener filter (Carmena et al., 2003; Haykin, 2002; Lebedev et al., 2005; Wessberg et al., 2000). The Wiener filter represented each decoded parameter as a weighted sum of neuronal rates measured before the time of decoding.
equation M1
(1)
where X(t) is the value of the decoded parameter (for example, X-coordinate of the ankle marker) at time t, ni is the firing rate of neuron i, N is the total number of neurons, (j − 1)τ is tap delay for tap j, wij is the weight for neuron i and tap j, b is the y-intercept, and ε(t) is the residual error. For extracting marker coordinates, joint angles, and foot contact with the treadmill the tap length parameter τ was set to 50 ms, and the number of taps (time bins of neuronal data) was set to 10, that is neuronal rates were sampled in a 500 ms window preceding the time of decoding. For extracting slower modulated characteristics such as walking speed and step length, a 5 s sample window was used composed of ten 500 ms taps.
Prediction of leg kinematics was performed using multiple Wiener filters applied to the activity of the entire population of the recorded neurons or subpopulations recorded in separate cortical areas. Thus, the activity of simultaneously recorded cortical neural ensembles allowed us to simultaneously extract a variety of motor parameters: position of hip, knee, and ankle; hip and knee angles; as well as foot contact, direction of walking, and periods of standing still. We used multiple linear models (Wiener filters; Carmena et al., 2003; Haykin, 2002; Lebedev et al., 2005; Wessberg et al., 2000) to describe the relationship between these parameters and neuronal ensemble activity. To calculate model weights, first Eq. 1 was converted to matrix form as:
equation M2
(2)
where X is the matrix of actual parameters, N is the matrix of neuronal rates, W is the weights of the model, ε is the error. Each row of N corresponds to a specific time and each column is a vector of data for a particular neuron and time lag. Since our models took into account ten lags, matrix N had ten columns for every neuron. The y-intercept was calculated from a column of ones prepended to matrix N. We then solved for matrix W by the following:
equation M3
(3)
Each Wiener filter was first trained (i.e. the values of weights W were calculated) and then used as the decoder for new data. Accordingly, each experimental record (10–15 min) was split in two halves: the training data and the predicting data. The model was trained on the first half of the experimental data and predictions were obtained using the second half. Decoding was also conducted for the reverse arrangement: training the models on the second half and using it to predict the first half. In addition to these offline analyses conducted for 80 experimental records (66 with Monkey 1, 14 with Monkey 2), real-time extraction was performed in 22 experimental sessions with Monkey 1. For real-time extraction, the neuronal and kinematic data were first recorded for 5 min while multiple Wiener filters were trained, and then online extraction of walking parameters was performed for 5–10 min.
General and switching model
To test whether models trained to accurately predict motor parameters for a specific behavioral paradigm would retain general kinematic prediction accuracy during alternative behaviors (i.e. be able to generalize to new paradigms), we trained models for one direction of movement (e.g. forward walking) and tested them in the other (e.g. backward walking; for an example of this, see Figure Figure7).7Figure 7). This allowed us to ask whether a model that is trained to predict leg kinematics while the monkey walks forward would retain prediction accuracy when applied to sessions of backward walking and vice versa. Next, we investigated whether our models would benefit from training on a variety of behavioral data, in this case both forward and backward walking. To address these questions, a series of linear models were trained on selected behavioral epochs. The models were trained on: (1) only forward walking, (2) only backward walking, and (3) an equal mixture of the two walking paradigms. Each model was trained on an equal length of time to avoid any performance biases and then tested on its ability to accurately predict walking forward or walking backward separately.
Figure 7
Figure 7
Figure 7
Generalized Training and the switching model. Panels A–C each represent a separate predictive model. In each, the left panel shows each model's ability to predict the kinematics of walking during sessions where the monkeys walk forward while the (more ...)
The switching mode was used to handle the conditions in which the monkey's locomotion consisted of two different paradigms: alternating periods of walking forward or walking backwards. Separate submodels were trained to decode each of these walking paradigms, and the paradigm predictor model served to detect the walking paradigm and select the appropriate submodel. The brief periods during treadmill mode switching when the monkey was standing still were classified as forward walking rather than introducing a third behavioral category. In our implementation, the switching model was a combination of three linear decoding models: a model for predicting forward walking, a model for predicting backwards walking and the paradigm predictor model (the switch). These models were arranged in a two layer structure (see Figure Figure7F)7Figure 7F) with the paradigm predictor model controlling a toggle between the two kinematic submodels which were then shunted to the final output of the switching model. Thus, when it was determined that the monkey was walking forward, one submodel was used to produce the output, and when backwards walking was detected, the other submodel was used.
To avoid any bias in our comparisons between the generalized and switching model, they were both trained on the same amount of data. This means that while the single generalized model was trained over the full training window, each of the submodels of the switching model were only trained on the portions of the data when the monkey's behavior fell in the relevant behavioral paradigm. Only the classifier portion of the switching model was trained on the full training window. This way, a true comparison of performance was achieved, and we could test whether the disadvantages of training the kinematic model on less total data were overcome by the advantages in prediction accuracy that come from behavioral segregation.
Model performance metrics
The signal to noise ratio (SNR) used in signal processing is a ratio between the power of the signal and the power of the noise, that is, the ratio of the squared amplitude of the signal and the squared amplitude of the noise. For our purposes, the signal was defined as the actual variable that we predicted. We calculated the variance, or power, of the signal by subtracting out the mean of the signal, then squaring and averaging the amplitude above or below that mean. The noise was the difference, or error, between the extracted and the actual signal. The error was calculated by subtracting the actual parameter from the extracted parameter, squaring the differences, then averaging to get the mean squared error, or the power of the noise. The ratio between these two was the SNR [or the signal to error ratio (SER)]. We then converted the ratio into a decibel (dB) scale:
equation M4
(4)
Where X is the actual parameter, equation M5 is the prediction of that parameter, Var is variance, and MSE is mean squared error. Theoretically, SNR can change from minus infinity (extremely poor predictions) to infinity (extremely good predictions). A SNR of 0 meant the signal and noise were in equal proportion in our predicted output. A SNR >0 meant we were extracting more signal than noise, and an SNR <0 meant there was more noise than signal, but the signal was still present for negative SNRs close to 0. SNRs equal to 10, 20, and 30 correspond to the ratio between the variances of 10, 100 and 1000, respectively
Additionally, Pearson's correlation coefficient, R, between the known signal and the predicted output was calculated:
equation M6
(5)
Where X is the actual parameter, equation M7 is the prediction of that parameter, cov is covariance, and σX and equation M8 are the standard deviation of X and equation M9 respectively. R can change in the range −1 to 1, with 1 corresponding to the highest possible correspondence between the actual and extracted values, and 0 corresponding to the absence of correspondence. Negative values of R, corresponding to an inverse relationship between the actual and extracted characteristics, occasionally occur when the extraction is poor, and R fluctuates around 0.
Signal to noise ratio proved to be a more sensitive measure compared to R in these analyses. This was because R describes the correspondence of signal waveforms, but is insensitive to amplitude scaling and offsets. SNR is sensitive to errors introduced by these factors, which is important for the purposes of efficient BMIs that require that the output of the model matches the true signal in all aspects. Additionally, R often saturates quickly for large ensemble sizes, whereas SNR better tracks the dependency of model performance on the ensemble size. A metric similar to SNR, called SER was used in previous studies in which behavioral parameters were extracted from neuronal data (Kim et al., 2006; Sanchez et al., 2002).
Neuron dropping analysis
Neuron dropping analysis (NDA) is a conventional way to characterize the prediction performance of neuronal ensembles of different size (Carmena et al., 2003; Lebedev et al., 2005, 2008; Wessberg et al., 2000). In this analysis, a number of decoding models were trained on random subsets of sorted neurons, ranging in size from a single neuron to the entire bank of sorted neurons. To characterize the performance of these decoding models, we calculated neuron dropping curves which described decoding accuracy as a function of neuronal sample size (Wessberg et al., 2000). The dropping curves were calculated by pooling randomly selected subsets of neurons and running the decoding model only for them. At each population subset size, five random subsets of neurons were generated and used to train separate predictive models. The predictive strength (SNR) of these models was calculated and plotted as scatter plots (for examples of this, see Figures Figures6A–F6Figure 6A–F and and7D,F).7Figure 7D,F). These scatter plots were then fitted using a power curve with simple linear decay:
equation M10
(6)
Figure 6
Figure 6
Figure 6
Neuronal dropping curves for simultaneously extracted variables. Dots represent SNR for the extraction of walking parameters using models trained on randomly selected neuronal populations of different sizes. In (A–C) red dots correspond to X coordinate, (more ...)
where x is neuronal subset size and y is SNR. Five parameters were fitted to generate the curves: yoffset translated the entire curve up or down, xoffset translated left or right, yamplitude defined the height of the fitted curve, p was the power of the curve and described the rate of the climb in SNR with increases in neuronal subset size, and mdecay described the slope of the linear decay.
For our analysis examining predictions of several kinematic variables simultaneously from the same neuronal population, we used a modified version of the NDA. Random subpopulations of the neuronal ensemble were selected and used to train models for a random combination of kinematic parameters simultaneously. Simultaneous prediction accuracies were calculated to be the minimum SNR level reached by all the predicted variables at each neuronal ensemble size. The overall simultaneous prediction dropping curves were then fitted by power curves (Eq. 6). The curves were normalized and averaged across all combinations of kinematic variables. The final curves were thresholded at several levels of prediction accuracy (0.75, 0.85 and 0.95) to determine the minimum number of neurons needed to predict each number of kinematic variables simultaneously with the given level of accuracy (for an example of this, see Figures Figures8A,B).8Figure 8A,B). Essentially, we calculated the expected number of neurons that would be required to predict, to a high accuracy threshold, any number of kinematic variables simultaneously in our experimental setup.
Figure 8
Figure 8
Figure 8
Simultaneous extraction of multiple parameters and real-time parameter extraction. (A) Neuronal dropping curves for extracting multiple parameters of walking. Each point on the curve represents the number of neurons needed to achieve a certain minimum (more ...)
Real-time system
We also developed a real-time BMI software suite capable of running all aspects of our experimental setups, including visual display, behavioral and multi-electrode neural recordings, model calculation and real-time predictions of kinematic and dynamic parameters. Written in Microsoft Visual C++, this BMI software suite can input behavioral data, multichannel extracellular single and multi-unit recordings, EMGs, and local field potentials.
Using a graphical user interface, models were trained to generate real-time predicted kinematics or EMGs from neural data. While our software suite currently implements the Wiener Filter, the Kalman Filter with optional principle components analysis dimensionality reduction, the n-th order Kalman Filter, and the n-th order Unscented Kalman Filter, in the real-time experiments described in this study we have used only the Wiener Filter option for extracting kinematic, dynamic, and EMG data.
The outputs of multiple real-time predictive models were displayed on a computer screen and simultaneously streamed over the network using a User Datagram Protocol.
Results
Kinematics of bipedal walking
During walking, both monkeys adopted a posture in which they leaned slightly forward while holding a bar and episodically assumed a more upright posture (Figures (Figures1A,B).1Figure 1A,B). The right leg was video-tracked in real time using fluorescent markers applied to the hip, knee and ankle joints. The monkeys' walking kinematics exhibited many basic features previously reported in both humans (Capaday, 2002; Grieve and Gear, 1966; Hirokawa, 1989; Murray et al., 1964, 1970; Nielsen, 2003; Perry, 1992; Vaughan et al., 1992) and monkeys (Hirasaki et al., 2004; Matsuyama et al., 2004; Mori et al., 2001; Tachibana et al., 2003). Step frequency rapidly increased as the treadmill speed increased from 0.05 to 0.2 m/s and began to plateau above 0.2 m/s (blue traces in Figure Figure1D).1Figure 1D). Step frequency during backward walking plateaued at a slightly higher value (1.1 steps/s) versus forward walking (1.0 steps/s). Step length also increased as the treadmill speed increased with a slight tapering off at higher speed (blue traces in Figure Figure1E).1Figure 1E). The stance phase constituted approximately 65% of leg stepping cycle (Monkey 1: 69.6 ± 13.1% during forward walking, 66.7 ± 16.2% during backward; Monkey 2: 57.6 ± 9.7% during forward).
During forward walking, the ankle and knee moved backward at a nearly constant speed after the foot touched the treadmill (Figure (Figure3A).3Figure 3A). At approximately mid-stance, the ankle started to lift, followed by the knee (Figure (Figure3B).3Figure 3B). These movements were accompanied by a lateral displacement (Z-dimension) of the ankle because of foot rotation (Figure (Figure3C).3Figure 3C). The knee angle monotonically increased during the stance phase as the knee joint extended during backward movement of the leg, and knee flexion started shortly before foot liftoff and continued during the swing (Figure (Figure3D).3Figure 3D). The soleus and rectus femoris were active during the stance phase and relaxed during the swing (Figure (Figure3E).3Figure 3E). The tibialis anterior was activated in a reciprocal manner: active during swing, relaxed during stance. This pattern of muscle activity is similar to EMG patterns during human walking (Capaday and Stein, 1986; Ivanenko et al., 2008; Kadaba et al., 1985; Shiavi et al., 1987; Winter and Yack, 1987). Examples of video frame during stance and swing are shown in Figure Figure33Figure 3F
Figure 3
Figure 3
Figure 3
Average changes in walking parameters during the step cycle. The data from all experimental sessions during which the monkey walked in forward direction with EMGs recorded were averaged. The average portions of the stance and swing phases of the step (more ...)
The discharge rate for each of the cortical neurons peaked at a preferred phase of the step cycle (Figure (Figure4).4Figure 4). The pattern of cortical activity during forward walking (Figure (Figure4A)4Figure 4A) was different for each recorded cortical area. In contralateral (left) M1, the number of neurons whose rates peaked during the stance was approximately equal to the number of neurons with the peak rate during the swing (61 ± 11 versus 63 ± 11, respectively, P > 0.05 Student's t-test). However, since the swing was shorter than the stance, the density of peak rates per unit of the step cycle (the slope of the red line in Figure Figure4A,4Figure 4A, 1st row) was lower for the stance phase (87.2 ± 18.6 peaks/s during stance versus 261.0 ± 29.0 peaks/s during swing, P < 0.001 Student's t-test), especially for its initial phase preceding the stance to swing transition. Ipsilateral (right) M1, showed a different pattern of neural activity in this analysis which used the movements of the right leg to designate the step cycle, with more neurons peaking during the stance phase than the swing phase (38 ± 6 versus 20 ± 6, respectively, P < 0.001 Student's t-test; Figure Figure4A,4Figure 4A, second row), evidently because neural modulations there represented the out of phase movements of the left leg. Activity in S1 neurons was heavily concentrated at the swing phase, especially during the stance to swing transition (55.0% of S1 neurons had their peak activity in the 20% of the step cycle centered at the stance to swing transition; Figure Figure4A,4Figure 4A, third row), when the foot was lifted from the treadmill surface.
Figure 4
Figure 4
Figure 4
Neural activity in each cortical area during the step cycle. The neural data from experimental sessions during which the monkey walked in the forward and backward direction were separately averaged for many step cycles. Each horizontal line corresponds (more ...)
A different pattern of ensemble modulations was revealed during backward walking (Figures (Figures4B,C).4Figure 4B,C). Notably, firing of the neurons tended to be more evenly spaced over the step cycle, especially in M1 (176.4 ± 29.4 peaks/s during stance versus 196.0 ± 19.6 peaks/s during swing, P > 0.05 Student's t-test). When the cortical activity during backwards walking was sorted according to where the cells were most active in forward walking (compare Figures Figures4A,C),4Figure 4A,C), it was apparent that there was little correspondence between individual cells' activity peaks between forward and backward walking. However, there was still a slight tendency for the neurons that were active in the swing phase during forward walking to remain active during the swing during backward walking (44.4% of the neurons with peak rate during the swing) and the same tendency for the stance-phase neurons (80.1%). This tendency was statistically significantly greater (P < 0.005 Student's t-test) than the percentage of neurons expected to stay in the same phase (29.9% and 70.1%) if there was no correlation between the neurons' peak firing phases in forward versus backward walking. Thus, although leg kinematics were similar during forward and backward walking, the underlying neuronal patterns were substantially different.
Extracting characteristics of walking from cortical ensemble activity
Multiple locomotion parameters were extracted using linear decoders which expressed the parameters as weighted sums of the neuronal firing rates (Carmena et al., 2003; Haykin, 2002; Lebedev et al., 2005; Wessberg et al., 2000). We decoded X (horizontal), Y (vertical), and Z (lateral) Cartesian coordinates of the leg joints, state of foot contact with the treadmill, step length and frequency, walking speed, and the EMGs of multiple leg muscles. Figures Figures44Figure 4 and and5E5Figure 5E represent modulations of M1 and S1 discharges as “waves” of neuronal activity that occurred during each step. While the individual neuronal firing rates were highly variable from step to step at the millisecond scale (Figure (Figure5E),5Figure 5E), combining the activity of many neurons in 100 ms bins using the linear decoder produced accurate predictions of leg movements, as evident from average traces for joint coordinates (Figures (Figures3A–C),3Figure 3A–C), joint angle (Figure (Figure3D)3Figure 3D) and EMGs of leg muscles (Figure (Figure3E),3Figure 3E), as well as step frequency (Figure (Figure1D)1Figure 1D) and step length (Figures (Figures1E,F)1Figure 1E,F) derived from predictions of joint positions. Figures Figures5A–E5Figure 5A–E illustrates a 5 s epoch of forward walking during which 18 motor parameters were simultaneously extracted: joint coordinates (Figure (Figure5A),5Figure 5A), joint angles (Figure (Figure5B),5Figure 5B), state of foot contact (Figure (Figure5C)5Figure 5C) and EMGs (Figure (Figure5D).5Figure 5D). Figure Figure5F5Figure 5F depicts direct extraction of step length and walking speed (characteristics modulated at a slower pace) from a different 50 s epoch.
Figure 5
Figure 5
Figure 5
Extraction of multiple characteristics of walking. In all traces, blue shows actual and red shows extracted value. All variables are shown over the same 5 s window, except in (F), where slow changing variables are shown over a longer 50 s (more ...)
The average extraction performance is summarized in Table 1. Overall, accuracy in predicting motor parameters was in line with that previously obtained for extracting arm movements from M1 activity (Carmena et al., 2003; Lebedev et al., 2005; Wessberg et al., 2000). We observed SNR in the range −2 to 7 dB and correlation coefficients, R, in the range 0.2–0.9. In both monkeys, the best extracted parameters were those related to the X and Y coordinates of the ankle, and knee angle (SNR in the range 3.8–6.0 dB and R in the range 0.79–0.87). The Z coordinate (lateral movements) was not predicted as well, because of minimal lateral movements of the joints during walking sessions. Likewise, the small amount of hip movement produced during walking may have contributed to low prediction accuracy for the hip's Cartesian coordinates compared to those of the knee and ankle. The X and Y coordinates of the knee and the hip were extracted with an average SNR in the range −0.7 to 4.3 dB and average R around 0.42–0.79. The average SNR and R for hip angle ranged from 0.9–3.0 and 0.58–0.73, respectively. Meanwhile, extraction accuracy for foot contact state (swing or stance) ranged from 1.2–3.1 in SNR and 0.58–0.61 in R, whereas the average SNR and R for the slowly modulated parameters were in the range −2.0 to 1.4 and 0.24–0.42 for walking speed, −1.8 to 0.9 and 0.48–0.57 for step frequency and −1.5 to 1.9 and 0.30–0.40 for step length. The low average accuracy of predicting the walking speed, especially in Monkey 2, reflected the fact that in many experiments the treadmill speed was constant for long periods of time and thus had very low variance. Predictions of EMGs (Figures (Figures3E3Figure 3E and and5D)5Figure 5D) were better (P < 0.001, Student's t-test) for the musculature of the right leg (SNR of 1.55 ± 0.39) than for the left leg (0.76 ± 0.17), likely because more neurons were recorded in the left hemisphere. Indeed, when prediction performance of equal size samples of neurons drawn from left versus right M1 was compared, contralateral M1 outperformed (P < 0.001, Student's t-test) ipsilateral M1 in predicting either leg EMG.
Table 1
Table 1
Average prediction accuracy. Correlation coefficient and signal to noise ratio (dB) for the predictions of different walking parameters. The numbers represent mean ± standard deviation
Neuron dropping analysis
Figures Figures6A–F6Figure 6A–F depicts NDA for the extraction of walking parameters. Each dot in the scatter plots represents prediction accuracy (quantified as SNR) calculated for a randomly selected neuronal subset. The plots are fitted with exponential curves. NDA revealed that the prediction accuracy of joint coordinates monotonically increased with neuronal sample size (Figures (Figures6A–C).6Figure 6A–C). In the particular case of predicting foot contact, NDA revealed a pattern consistent with the known phenomenon of overfitting (Babyak, 2004; Santucci et al., 2005), i.e. the SNR increased as the sample grew to 50 neurons, and it decreased thereafter (Figure (Figure6D).6Figure 6D). In agreement with Table 1, the Y-coordinates (vertical movements) for the ankle (Figure (Figure6A)6Figure 6A) and knee joints (Figure (Figure6B)6Figure 6B) were the most accurately predicted of all Cartesian coordinates. X was the second best extracted coordinate. Prediction accuracy for the knee and hip angles was comparable to the prediction of X and Y coordinates for these joints (Figures (Figures66Figure 6B,C).
Neuron dropping analysis for predictions of the ankle X coordinate was also performed separately for different cortical areas (Figures (Figures6E,F).6Figure 6E,F). In both monkeys, walking parameters could be predicted using neuronal activity recorded in either M1 or S1 contralateral to the right leg, as well as ipsilateral M1 in Monkey 1, but none of these areas taken alone reached the prediction performance level of the entire population. Predictions using random populations drawn from each individual cortical area were statistically significantly worse than predictions from equivalently sized random populations drawn from all cortical areas (Student's t-test, P < 0.001). For each area, the prediction performance improved with increases in population size. Slight overfitting was observed for S1 in Monkey 2 for ensemble sizes greater than 80 neurons (Figure (Figure6F,6Figure 6F, blue dots and curve). In Monkey 1, predictions from contralateral M1 neurons (Figure (Figure6E,6Figure 6E, red) were better than the predictions from ipsilateral M1 neurons (Figure (Figure6E,6Figure 6E, green) or contralateral S1 neurons (Figure (Figure6E,6Figure 6E, blue). In Monkey 2, predictions from S1 neurons were better than the predictions from M1 neurons (Figure (Figure66Figure 6F).
The performance of the extraction algorithm, using M1 versus S1 neurons, was further examined using a timing analysis in which a single 50 ms time window was used to measure extraction accuracy at various lags with respect to the present time (time 0; Figures Figures6G,H).6Figure 6G,H). Peaks in prediction accuracy for the lags preceding the present time (negative lags) indicated that neuronal modulations were predictive of future movements, suggesting a causal link between neuronal modulations and movements. Such a relationship was expected for M1 neurons and confirmed for both Monkey 1 (Figure (Figure6G)6Figure 6G) and Monkey 2 (Figure (Figure6H).6Figure 6H). Peaks for lags succeeding the present (positive lags) indicated a reverse relationship: present movements were the cause of future neuronal modulations. This was expected for S1 neurons, and confirmed in Monkey 2; the peak S1 extraction occurred at a positive lag (Figure (Figure6H).6Figure 6H). S1 activity was generally not strongly predictive at any lag in Monkey 1. There were minor peaks in S1 predictive power for both positive and negative lags (Figure (Figure6G).6Figure 6G). Still, the distinction between M1 and S1 was not clear cut because S1 modulations were capable of predicting motor parameters in the future (Figures (Figures66Figure 6E–H).
Generalization properties and a switching model
A linear decoder trained on a single walking paradigm (e.g. forward walking only or backward walking only) was able to accurately extract walking parameters during the same paradigm (SNR = 5.29 dB for the forward model applied to forward walking and SNR = 3.53 dB for the backward model applied to backward walking; Figures Figures7A,7Figure 7A, left and 7B, right). However, predictive power was not retained when applied to predict a different paradigm (SNR = −2.91 dB for the forward model applied to backward walking and SNR = −2.77 dB for the backward model applied to forward walking; Figures Figures7A,left7Figure 7A,left panel and 7B,right panel). These results were confirmed by NDA (Figure (Figure77Figure 7D).
At the same time, general models trained on a mixture of both forward and backward walking (walking forward 45.1 ± 1.4% and backward 54.9 ± 1.4% of the time) accurately predicted both forward (Figure (Figure7C,7Figure 7C, left panel; SNR = 4.74 dB) and backward (Figure (Figure7C,7Figure 7C, right panel; SNR = 2.24 dB) walking separately, albeit slightly worse compared to the single paradigm models. This was also evident from NDA (Figure (Figure77Figure 7D).
To recapture the lost prediction accuracy when using general models versus single paradigm models, we generated a multi-layer switching model (Figures (Figures7F–I).7Figure 7F–I). The switching model classified the subject's walking as a number of sub-behaviors. That is, while the monkey was performing variable direction walking, walking was sub-classified into forward walking and backward walking periods. The switching model had two layers. The primary layer contained several independent submodels for kinematic variables, each one trained on data from a single sub-behavior. The secondary layer was a director layer that predicted the current behavioral mode (here, walking direction) for the subject and selected the appropriate kinematic submodel. The switching model outperformed the standard linear model trained during sessions with both forward and backward walking. The switching model improved the predictions of all kinematic variables on average by 13.8.
Predicting multiple parameters simultaneously
Simultaneous extraction of many parameters essentially depended on the size of the neuronal population (Figures (Figures8A–C).8Figure 8A–C). Random subpopulations were pooled from the entire neuronal ensemble and used to predict several parameters simultaneously. For each parameter, prediction accuracy was normalized by its maximum value. The accuracy of simultaneous predictions was characterized by the normalized accuracy for the least well predicted parameter. Neuron dropping curves for this metric had less steep slopes when larger parameter sets were predicted (Figure (Figure8A).8Figure 8A). In other words, smaller neuronal populations could predict a few parameters simultaneously, and larger populations were required for predicting many parameters. This is illustrated in Figure Figure8B,8Figure 8B, which shows that the number of neurons required to achieve a given minimum accuracy (75%, 85% or 95% of maximum prediction accuracy) increased with the number of simultaneously predicted parameters.
Similar to the result for the simultaneous extraction of multiple parameters, larger neuronal populations were required to predict complex patterns of walking compared to more simple walking patterns. For example, the number of neurons required to achieve 95% of maximum prediction accuracy for the X position of the ankle clearly increased when walking conditions of increasing complexity were employed (Figure (Figure8C).8Figure 8C). On average, 60 neurons were sufficient for predicting constant-speed walking in the forward direction. However, 90 neurons were needed to achieve this level of accuracy for variable-speed, forward walking. As complexity increased, more neurons were needed to maintain the same prediction accuracy. Thus, 95 neurons were required for predicting backward walking at constant speed, while 115 neurons were required for predicting backward walking at variable speed. Predicting variable-speed bidirectional walking required 110 neurons.
Real-time predictions of leg kinematics using cortical neuronal ensembles
Real-time prediction of kinematic variables (Figure (Figure8D)8Figure 8D) was performed in 22 daily recording sessions in Monkey 1. In each session, the first 5 min of walking were used for model training data. These data were sent to a buffer which was used to calculate a set of linear models for the leg kinematic parameters. These models were then used to generate real-time predictions from neural spike data alone. The actual and extracted joint angles are shown in Figure Figure8E,8Figure 8E, and the actual and extracted leg configurations are shown in Figure Figure8F.8Figure 8F. Real-time predictions were not statistically different from offline predictions (real-time SNR = 2.38 ± 1.14, offline SNR 2.46 ± 1.57, P > 0.05, Student's t-test). The real-time system included a web link that could send the extracted kinematics to robotic appendages and return visual feedback (Figure (Figure8D).8Figure 8D). The operation of robotic devices under these conditions has been reported in abstract form (Cheng et al., 2007; Lebedev et al., 2007) and will be covered in detail in a future article.
Discussion
In this study, we extracted bipedal walking patterns from the modulations of discharge rates of monkey S1 and M1 neuronal ensembles. While such modulations were expected from previous studies in quadrupeds (Armstrong and Marple-Horvat, 1996; Beloozerova and Sirota, 1998; Drew, 1993; Prilutsky et al., 2005; Widajewicz et al., 1994), it was not clear until the present study whether cortical activity would be sufficient for accurate predictions of leg kinematics and EMGs during bipedal walking in primates. These findings have several implications. First, we have demonstrated the utility of using nonhuman primates for elucidating the neurophysiological mechanisms of bipedal locomotion. As such, we propose that chronic, multi-site cortical recordings could be used in the future to elucidate the neuronal mechanisms involved in upright posture control and bipedal walking. Second, as an implication for the BMI field, our findings provide the first proof of concept demonstration that, in the future, a neuroprosthetic for restoring bipedal walking in severely paralyzed patients can be implemented.
Cortical neuronal modulations during bipedal locomotion
Using cortical activity rather than subcortical locomotion centers for extracting walking patterns is not without controversy (Prochazka et al., 2000). Although locomotion is conventionally recognized as a highly automated motor activity, subserved by spinal central pattern generators (Dietz, 2001; Grillner, 2006; Grillner et al., 2008; Hamm et al., 1999; Kiehn et al., 2008; McCrea and Rybak, 2008; Yamaguchi, 2004), the involvement of the cerebral cortex in gait control has been reported in studies conducted in rodents (Hermer-Vazquez et al., 2004), cats (Armstrong and Marple-Horvat, 1996; Beloozerova and Sirota, 1993), and monkeys (Courtine et al., 2005b). Moreover, cortical involvement in locomotion seems to increase during more demanding tasks, especially those that require visual feedback (Armstrong and Marple-Horvat, 1996; Beloozerova and Sirota, 1993; Drew et al., 2008).
Sampling from large populations of M1 and S1 neurons allowed us to analyze activity modulations in hundreds of simultaneously recorded neurons. Both M1 and S1 neurons modulated their firing rate in relationship to the stepping cycle. Firing for each neuron peaked at a particular phase of the stepping cycle. Given the large number of neurons that we recorded, it was unrealistic to examine in detail the properties of each neuron in order to determine which motor or sensory responses determined its step cycle modulations. Therefore, we used a population based approach in which cycle modulations of individual neurons each contributed to the extraction of the step cycle at every time step, and more precise extractions were obtained by combining the information from many neurons. We also showed the benefits of utilizing multi-site recordings to predict locomotion kinematic parameters. This approach was beneficial because neuronal modulations from different cortical areas provided the diversity and richness of cortical signals needed for the accurate extraction of motor parameters under different behavioral conditions. In this context, we consistently observed superior performance of neuronal populations drawn from several cortical areas compared to the performance of populations drawn from a single area.
Extracting of multiple parameters of bipedal walking
Walking patterns were extracted using multiple linear models. As in previous studies in which predictions of arm movements were obtained, NDA showed that large neuronal ensembles were needed for accurate predictions of leg kinematic parameters. Thus, larger neuronal samples were required for simultaneous extraction of multiple walking parameters. Small populations of relatively specialized neurons could accurately extract only a subset of locomotion parameters, and additional neurons were required to improve the extraction of the other parameters. As a corollary to this finding, larger neuronal populations were required as task complexity increased. These results support our previous suggestion that, to efficiently control a multiple degree of freedom neuroprosthetic, recording the activity of large ensembles of cortical neurons will be necessary (Carmena et al., 2003; Lebedev et al., 2007; Wessberg et al., 2000).
M1 versus S1 contributions to the prediction of leg kinematics
Both M1 and S1 neurons contributed significantly to the predictions of leg kinematics. As expected, M1 modulations were more informative for predicting future values in the parameters of walking, whereas S1 modulations better predicted the past values of the parameters. However, this distinction was not absolute, particularly in Monkey 2 in which accurate predictions of future values of locomotion parameters could be obtained from S1 activity. While repetitive nature of the movement may be one contributor to S1 predictive power, predictions obtained from S1 activity could also reflect centrally generated signals related to communication between motor and sensory areas and efferent copies (Chapin and Woodward, 1982; Fanselow and Nicolelis, 1999; Lebedev et al., 1994).
Generalized training
The computational models of a BMI take advantage of correlations between neuronal firing modulations and behavioral parameters to generate their predictions. These correlations do not necessarily reflect a direct relationship or causation between neuronal discharges and the predicted parameters (Lebedev and Nicolelis, 2006). Consequently, a model trained under one set of conditions may not perform well when the conditions are changed and the correlation between the neuronal discharges and behavioral parameters are altered. We observed this effect when we attempted to predict different locomotion patterns (Figure (Figure7).7Figure 7). Models trained on forward walking did not accurately predict backward walking and vice versa. This drop in prediction performance occurred because of the differences in neuronal modulations during the step cycles of forward walking versus backwards walking (Figure (Figure4).4Figure 4). Such an effect was expected because leg dynamics change dramatically during backwards walking despite relatively small changes in leg kinematics (Thorstensson, 1986). It was only when the models were trained on a wider variety of patterns that they improved their ability to predict more diverse behaviors. Given that motor cortical neurons may represent muscle activity, limb kinematics and their combinations (Kakei et al., 1999), which all change in complex ways when behavioral paradigms are altered, this result is not entirely surprising. For instance, the pattern of muscle activation may become different for a given leg orientation in a new walking condition, and muscle-representing neurons would introduce errors in predicted kinematics. It is only when the linear models are trained on several behaviors that the kinematics can be accurately extracted. These results suggest that in order to build a neuroprosthetic that is both accurate and versatile in restoring locomotion patterns, it will be necessary to generate a training protocol which is varied enough so that the employed models learn to predict a large set of behaviors.
Switching model
While generalized training had the advantage of producing models that provided accurate predictions of the various behavioral paradigms included in the training data set, generalized models predicted with less accuracy than specific models designed for a single walking paradigm. To recapture some of this lost accuracy, we developed a switching model that selected the appropriate specific model once a switch in the walking paradigm was detected. Algorithms based on the idea of switching state spaces have been previously proposed for extracting arm movements from neuronal modulations (Kim et al., 2003; Wu et al., 2004). While the switching model has many advantages and it increased the accuracy of predictions throughout this experiment, it does carry some potential drawbacks. For variables that do not behave differently between behavioral paradigms, there is little lost accuracy to recapture, and segregating the data in the training set into smaller sections for each submodel can have a detrimental effect, that is, the lower quantity of training data can produce a less accurate model. Additionally, the performance of the switching model is heavily dependent on the performance of the state predicting model that chooses which kinematic submodel is selected
Real-time bmi for reproduction of locomotion patterns
Since one of the long-term goals of this research is to build a neuroprosthetic device for the restoration of locomotion in paralyzed subjects, we tested the performance of our decoding algorithm using a real-time BMI suite that incorporated MAP recording hardware (Plexon Inc., TX, USA), a custom wireless video tracking system and a set of real-time prediction algorithms. The BMI suite sampled the activity of up to 512 neuronal units and converted this neuronal activity into the predictions of multiple degree of freedom kinematic and behavioral parameters. The BMI suite streamed the predictions using an internet protocol. The successful implementation of this apparatus allowed us to obtain a proof of concept that a real-time BMI for the reproduction of locomotion can be implemented in the future as the core of a neuroprosthetic device aimed at restoring bipedal locomotion in severely paralyzed patients.
Restoration of locomotion behaviors using a BMI
Based on these results, we propose an approach to restore locomotion in patients with lower limb paralysis that relies on using cortical activity to generate locomotor patterns in an artificial actuator, such as a wearable exoskeleton (Figure (Figure8G;8Figure 8G; Fleischer et al., 2006; Hesse et al., 2003; Veneman et al., 2007). This approach may be applicable to clinical cases in which the locomotion centers of the brain are intact, but cannot communicate with the spinal cord circuitry due to spinal cord injury. The feasibility of employing a cortically driven BMI for the restoration of gait is supported by fMRI studies in which cortical activation was detected when subjects imagined themselves walking (Bakker et al., 2007, 2008; Iseki et al., 2008; Jahn et al., 2004) and when paraplegic patients imagined foot and leg movements (Alkadhi et al., 2005; Cramer et al., 2005; Hotz-Boendermaker et al., 2008). Event-related potentials also demonstrated cortical activations in similar circumstances (Halder et al., 2006; Lacourse et al., 1999; Muller-Putz et al., 2007). Further support for this idea comes from recent studies of EEG-based brain-computer interfaces for navigation in a virtual environment in healthy subjects (Pfurtscheller et al., 2006) and paraplegics (Enzinger et al., 2008).
While a cortical BMI based neuroprosthesis that derived all its control signals from the user would have to cope with the lack of signals normally derived from subcortical centers, such as the cerebellum, basal ganglia and brainstem (Grillner, 2006; Grillner et al., 2008; Hultborn and Nielsen, 2007; Kagan and Shik, 2004; Matsuyama et al., 2004; Mori et al., 2000; Takakusaki, 2008), these problems may be avoided by an approach which only derives higher level leg movement signals from brain activity, while allowing robotic systems to produce a safer, optimum output. The challenge of efficient low-level control could be overcome by implementing “shared brain–machine” control (Kim et al., 2006), i.e. a control strategy that allows robotic controllers to efficiently supervise low-level details of motor execution, while brain derived signals are utilized to derive higher-order voluntary motor commands (step initiation, step length, leg orientation).
A cortically driven BMI for the restoration of walking may become an integral part of other rehabilitation strategies employed to improve the quality of life of patients. In particular, it may supplement the strategy based on harnessing the remaining functionality and plasticity of spinal cord circuits isolated from the brain (Behrman et al., 2006; Dobkin et al., 1995; Grasso et al., 2004; Harkema, 2001; Lunenburger et al., 2006). Indeed, cortically driven exoskeletons may facilitate spinal cord plasticity, helping to recover locomotion automatisms. Additionally, cortically driven neuroprostheses may work in cohort with rehabilitation methods based on functional electrical stimulation (FES; Hamid and Hayek, 2008; Nightingale et al., 2007; Wieler et al., 1999; Zhang and Zhu, 2007). In such an implementation, the BMI output could be connected to a FES system that stimulates the subject's leg muscles. Finally, there is the intriguing possibility of connecting the BMI to an electrical stimulator implanted in the spinal cord, a strategy that may help induce plastic reorganization within these circuits.
Altogether, our results indicate that direct linkages between the human brain and artificial devices may be utilized to define a series of neuroprosthetic devices for restoring the ability to walk in people suffering from paralysis.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
This work was supported by the Anne W. Deane Endowed Chair Fund and by Telemedicine and Advanced Technology Research Center (TATRC) W81XWH-08-2-0119 to MALN. We are grateful to Gary Lehew for building the experimental treadmill, Timothy Hanson, Zheng Li and Joseph O'Doherty for programming the BMI software, Dragan Dimitrov, Laura Oliveira and Aaron Sandler for conducting the implantation surgery, Weiying Drake, Tamara Phillips, Benjamin Grant and Jenna Maloka for experimental support, and Susan Halkiotis for help with manuscript preparation.
References
  • Alexander R. M. (2004. ). Bipedal animals, and their differences from humans. J. Anat. 204, 321–330 doi: 10.1111/j.0021-8782.2004.00289.x. [PubMed]
  • Alkadhi H., Brugger P., Boendermaker S. H., Crelier G., Curt A., Hepp-Reymond M. C., Kollias S. S. (2005. ). What disconnection tells about motor imagery: evidence from paraplegic patients. Cereb. Cortex 15, 131–140 doi: 10.1093/cercor/bhh116. [PubMed]
  • Andriacchi T. P., Alexander E. J. (2000. ). Studies of human locomotion: past, present and future. J. Biomech. 33, 1217–1224 doi: 10.1016/S0021-9290(00)00061-0. [PubMed]
  • Armstrong D. M., Marple-Horvat D. E. (1996. ). Role of the cerebellum and motor cortex in the regulation of visually controlled locomotion. Can. J. Physiol. Pharmacol. 74, 443–455 doi: 10.1139/cjpp-74-4-443. [PubMed]
  • Azevedo C., Espiau B., Amblard B., Assaiante C. (2007. ). Bipedal locomotion: toward unified concepts in robotics and neuroscience. Biol. Cybern. 96, 209–228 doi: 10.1007/s00422-006-0118-0. [PubMed]
  • Babyak M. A. (2004. ). What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom. Med. 66, 411–421 doi: 10.1097/01.psy.0000127692.23278.a9. [PubMed]
  • Bakker M., de Lange F. P., Helmich R. C., Scheeringa R., Bloem B. R., Toni I. (2008. ). Cerebral correlates of motor imagery of normal and precision gait. Neuroimage 41, 998–1010 doi: 10.1016/j.neuroimage.2008.03.020. [PubMed]
  • Bakker M., de Lange F. P., Stevens J. A., Toni I., Bloem B. R. (2007. ). Motor imagery of gait: a quantitative approach. Exp. Brain Res. 179, 497–504 doi: 10.1007/s00221-006-0807-x. [PubMed]
  • Behrman A. L., Bowden M. G., Nair P. M. (2006. ). Neuroplasticity after spinal cord injury and training: an emerging paradigm shift in rehabilitation and walking recovery. Phys. Ther. 86, 1406–1425 doi: 10.2522/ptj.20050212. [PubMed]
  • Beloozerova I. N., Sirota M. G. (1993. ). The role of the motor cortex in the control of accuracy of locomotor movements in the cat. J. Physiol. 461, 1–25 [PubMed]
  • Beloozerova I. N., Sirota M. G. (1998. ). Cortically controlled gait adjustments in the cat. Ann. N. Y. Acad. Sci. 860, 550–553 doi: 10.1111/j.1749-6632.1998.tb09101.x. [PubMed]
  • Beloozerova I. N., Sirota M. G., Swadlow H. A. (2003. ). Activity of different classes of neurons of the motor cortex during locomotion. J. Neurosci. 23, 1087–1097 [PubMed]
  • Boonstra T. A., van der Kooij H., Munneke M., Bloem B. R. (2008. ). Gait disorders and balance disturbances in Parkinson's disease: clinical update and pathophysiology. Curr. Opin. Neurol. 21, 461–471 doi: 10.1097/WCO.0b013e328305bdaf. [PubMed]
  • Borelli G. A. (1680. ). De Motu Animalium. Rome: , Superiorum Permissu;
  • Capaday C. (2002. ). The special nature of human walking and its neural control. Trends Neurosci. 25, 370–376 doi: 10.1016/S0166-2236(02)02173-2. [PubMed]
  • Capaday C., Stein R. B. (1986. ). Amplitude modulation of the soleus H-reflex in the human during walking and standing. J. Neurosci. 6, 1308–1313 [PubMed]
  • Carmena J. M., Lebedev M. A., Crist R. E., O'Doherty J. E., Santucci D. M., Dimitrov D. F., Patil P. G., Henriquez C. S., Nicolelis M. A. (2003. ). Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 1, E42. doi: 10.1371/journal.pbio.0000042. [PubMed]
  • Carmena J. M., Lebedev M. A., Henriquez C. S., Nicolelis M. A. (2005. ). Stable ensemble performance with single-neuron variability during reaching movements in primates. J. Neurosci. 25, 10712–10716 doi: 10.1523/JNEUROSCI.2772-05.2005. [PubMed]
  • Chapin J. K. (2004. ). Using multi-neuron population recordings for neural prosthetics. Nat. Neurosci. 7, 452–455 doi: 10.1038/nn1234. [PubMed]
  • Chapin J. K., Woodward D. J. (1982. ). Somatic sensory transmission to the cortex during movement: gating of single cell responses to touch. Exp. Neurol. 78, 654–669 doi: 10.1016/0014-4886(82)90082-6. [PubMed]
  • Chatani K. (2003. ). Positional behavior of free-ranging Japanese macaques (Macaca fuscata). Primates 44, 13–23 [PubMed]
  • Cheng G., Fitzsimmons N. A., Morimoto J., Lebedev M. A., Kawato M., Nicolelis M. A. (2007. ). Bipedal locomotion with a humanoid robot controlled by cortical ensemble activity. Abstr. Soc. Neurosci. 517, 22.
  • Courtine G., Roy R. R., Hodgson J., McKay H., Raven J., Zhong H., Yang H., Tuszynski M. H., Edgerton V. R. (2005a. ) Kinematic and EMG determinants in quadrupedal locomotion of a non-human primate (Rhesus). J. Neurophysiol. 93, 3127–3145 doi: 10.1152/jn.01073.2004. [PubMed]
  • Courtine G., Roy R. R., Raven J., Hodgson J., McKay H., Yang H., Zhong H., Tuszynski M. H., Edgerton V. R. (2005b. ) Performance of locomotion and foot grasping following a unilateral thoracic corticospinal tract lesion in monkeys (Macaca mulatta). Brain 128, 2338–2358 doi: 10.1093/brain/awh604. [PubMed]
  • Cramer S. C., Lastra L., Lacourse M. G., Cohen M. J. (2005. ). Brain motor system function after chronic, complete spinal cord injury. Brain 128, 2941–2950 doi: 10.1093/brain/awh648. [PubMed]
  • Deliagina T. G., Beloozerova I. N., Zelenin P. V., Orlovsky G. N. (2008. ). Spinal and supraspinal postural networks. Brain Res. Rev. 57, 212–221 doi: 10.1016/j.brainresrev.2007.06.017. [PubMed]
  • Deliagina T. G., Orlovsky G. N. (2002. ). Comparative neurobiology of postural control. Curr. Opin. Neurobiol. 12, 652–657 doi: 10.1016/S0959-4388(02)00376-8. [PubMed]
  • Dickinson M. H., Farley C. T., Full R. J., Koehl M. A., Kram R., Lehman S. (2000. ). How animals move: an integrative view. Science 288, 100–106 doi: 10.1126/science.288.5463.100. [PubMed]
  • Dietz V. (2001. ). Spinal cord lesion: effects of and perspectives for treatment. Neural Plast. 8, 83–90 doi: 10.1155/NP.2001.83. [PubMed]
  • Dietz V., Colombo G. (2004. ). Recovery from spinal cord injury – underlying mechanisms and efficacy of rehabilitation. Acta Neurochir. Suppl. 89, 95–100 [PubMed]
  • Dobkin B. H., Harkema S., Requejo P., Edgerton V. R. (1995. ). Modulation of locomotor-like EMG activity in subjects with complete and incomplete spinal cord injury. J. Neurol. Rehabil. 9, 183–190 [PubMed]
  • Drew T. (1993. ). Motor cortical activity during voluntary gait modifications in the cat. I. Cells related to the forelimbs. J. Neurophysiol. 70, 179–199 [PubMed]
  • Drew T., Andujar J. E., Lajoie K., Yakovenko S. (2008. ). Cortical mechanisms involved in visuomotor coordination during precision walking. Brain Res. Rev. 57, 199–211 doi: 10.1016/j.brainresrev.2007.07.017. [PubMed]
  • Drew T., Prentice S., Schepens B. (2004. ). Cortical and brainstem control of locomotion. Prog. Brain Res. 143, 251–261 doi: 10.1016/S0079-6123(03)43025-2. [PubMed]
  • Enzinger C., Ropele S., Fazekas F., Loitfelder M., Gorani F., Seifert T., Reiter G., Neuper C., Pfurtscheller G., Muller-Putz G. (2008. ). Brain motor system function in a patient with complete spinal cord injury following extensive brain-computer interface training. Exp. Brain Res. 190, 215–223 doi: 10.1007/s00221-008-1465-y. [PubMed]
  • Fanselow E. E., Nicolelis M. A. (1999. ). Behavioral modulation of tactile responses in the rat somatosensory system. J. Neurosci. 19, 7603–7616 [PubMed]
  • Fetz E. E. (2007. ). Volitional control of neural activity: implications for brain-computer interfaces. J. Physiol. 579, 571–579 doi: 10.1113/jphysiol.2006.127142. [PubMed]
  • Fleischer C., Wege A., Kondak K., Hommel G. (2006. ). Application of EMG signals for controlling exoskeleton robots. Biomed. Tech. (Berl.) 51, 314–319 doi: 10.1515/BMT.2006.063. [PubMed]
  • Georgopoulos A. P., Grillner S. (1989. ). Visuomotor coordination in reaching and locomotion. Science 245, 1209–1210 doi: 10.1126/science.2675307. [PubMed]
  • Grasso R., Ivanenko Y. P., Zago M., Molinari M., Scivoletto G., Castellano V., Macellari V., Lacquaniti F. (2004. ). Distributed plasticity of locomotor pattern generators in spinal cord injured patients. Brain 127, 1019–1034 doi: 10.1093/brain/awh115. [PubMed]
  • Green L. B., Hurvitz E. A. (2007. ). Cerebral palsy. Phys. Med. Rehabil. Clin. N. Am. 18, 859–882, vii. doi: 10.1016/j.pmr.2007.07.005. [PubMed]
  • Grieve D. W., Gear R. J. (1966. ). The relationships between length of stride, step frequency, time of swing and speed of walking for children and adults. Ergonomics 9, 379–399 doi: 10.1080/00140136608964399. [PubMed]
  • Grillner S. (2006. ). Biological pattern generation: the cellular and computational logic of networks in motion. Neuron 52, 751–766 doi: 10.1016/j.neuron.2006.11.008. [PubMed]
  • Grillner S., Kozlov A., Dario P., Stefanini C., Menciassi A., Lansner A., Hellgren Kotaleski J. (2007. ). Modeling a vertebrate motor system: pattern generation, steering and control of body orientation. Prog. Brain Res. 165, 221–234 doi: 10.1016/S0079-6123(06)65014-0. [PubMed]
  • Grillner S., Wallen P. (2002. ). Cellular bases of a vertebrate locomotor system-steering, intersegmental and segmental co-ordination and sensory control. Brain Res. Brain Res. Rev. 40, 92–106 doi: 10.1016/S0165-0173(02)00193-5. [PubMed]
  • Grillner S., Wallen P., Saitoh K., Kozlov A., Robertson B. (2008. ). Neural bases of goal-directed locomotion in vertebrates – an overview. Brain Res Rev 57, 2–12 doi: 10.1016/j.brainresrev.2007.06.027. [PubMed]
  • Halder P., Curt A., Brem S., Lang-Dullenkopf A., Bucher K., Kollias S., Brandeis D. (2006. ). Preserved aspects of cortical foot control in paraplegia. Neuroimage 31, 692–698 doi: 10.1016/j.neuroimage.2005.12.018. [PubMed]
  • Hamid S., Hayek R. (2008. ). Role of electrical stimulation for rehabilitation and regeneration after spinal cord injury: an overview. Eur. Spine J. 17, 1256–1269 doi: 10.1007/s00586-008-0729-3. [PubMed]
  • Hamm T. M., Trank T. V., Turkin V. V. (1999. ). Correlations between neurograms and locomotor drive potentials in motoneurons during fictive locomotion: implications for the organization of locomotor commands. Prog. Brain Res. 123, 331–339 doi: 10.1016/S0079-6123(08)62868-X. [PubMed]
  • Harkema S. J. (2001. ). Neural plasticity after human spinal cord injury: application of locomotor training to the rehabilitation of walking. Neuroscientist 7, 455–468 [PubMed]
  • Hatanaka N., Nambu A., Yamashita A., Takada M., Tokuno H. (2001. ). Somatotopic arrangement and corticocortical inputs of the hindlimb region of the primary motor cortex in the macaque monkey. Neurosci. Res. 40, 9–22 doi: 10.1016/S0168-0102(01)00210-3. [PubMed]
  • Hatsopoulos N., Joshi J., O'Leary J. G. (2004. ). Decoding continuous and discrete motor behaviors using motor and premotor cortical ensembles. J. Neurophysiol. 92, 1165–1174 doi: 10.1152/jn.01245.2003. [PubMed]
  • Haykin S. (2002. ). Adaptive Filter Theory Upper Saddle River, NJ: Prentice Hall.
  • Hermer-Vazquez L., Hermer-Vazquez R., Moxon K. A., Kuo K. H., Viau V., Zhan Y., Chapin J. K. (2004. ). Distinct temporal activity patterns in the rat M1 and red nucleus during skilled versus unskilled limb movement. Behav. Brain Res. 150, 93–107 doi: 10.1016/S0166-4328(03)00226-2. [PubMed]
  • Hesse S., Schmidt H., Werner C., Bardeleben A. (2003. ). Upper and lower extremity robotic devices for rehabilitation and for studying motor control. Curr. Opin. Neurol. 16, 705–710 doi: 10.1097/00019052-200312000-00010. [PubMed]
  • Hirasaki E., Ogihara N., Hamada Y., Kumakura H., Nakatsukasa M. (2004. ). Do highly trained monkeys walk like humans? A kinematic study of bipedal locomotion in bipedally trained Japanese macaques. J. Hum. Evol. 46, 739–750 doi: 10.1016/j.jhevol.2004.04.004. [PubMed]
  • Hirokawa S. (1989. ). Normal gait characteristics under temporal and distance constraints. J. Biomed. Eng. 11, 449–456 doi: 10.1016/0141-5425(89)90038-1. [PubMed]
  • Hochberg L. R., Serruya M. D., Friehs G. M., Mukand J. A., Saleh M., Caplan A. H., Branner A., Chen D., Penn R. D., Donoghue J. P. (2006. ). Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 doi: 10.1038/nature04970. [PubMed]
  • Hotz-Boendermaker S., Funk M., Summers P., Brugger P., Hepp-Reymond M. C., Curt A., Kollias S. S. (2008. ). Preservation of motor programs in paraplegics as demonstrated by attempted and imagined foot movements. Neuroimage 39, 383–394 doi: 10.1016/j.neuroimage.2007.07.065. [PubMed]
  • Hultborn H., Nielsen J. B. (2007. ). Spinal control of locomotion – from cat to man. Acta Physiol. (Oxf) 189, 111–121 doi: 10.1111/j.1748-1716.2006.01651.x. [PubMed]
  • Iseki K., Hanakawa T., Shinozaki J., Nankaku M., Fukuyama H. (2008. ). Neural mechanisms involved in mental imagery and observation of gait. Neuroimage 41, 1021–1031 doi: 10.1016/j.neuroimage.2008.03.010. [PubMed]
  • Ivanenko Y. P., Cappellini G., Poppele R. E., Lacquaniti F. (2008. ). Spatiotemporal organization of alpha-motoneuron activity in the human spinal cord during different gaits and gait transitions. Eur. J. Neurosci. 27, 3351–3368 doi: 10.1111/j.1460-9568.2008.06289.x. [PubMed]
  • Jahn K., Deutschlander A., Stephan T., Strupp M., Wiesmann M., Brandt T. (2004. ). Brain activation patterns during imagined stance and locomotion in functional magnetic resonance imaging. Neuroimage 22, 1722–1731 doi: 10.1016/j.neuroimage.2004.05.017. [PubMed]
  • Kadaba M. P., Wootten M. E., Gainey J., Cochran G. V. (1985. ). Repeatability of phasic muscle activity: performance of surface and intramuscular wire electrodes in gait analysis. J. Orthop. Res. 3, 350–359 doi: 10.1002/jor.1100030312. [PubMed]
  • Kagan I., Shik M. L. (2004. ). How the mesencephalic locomotor region recruits hindbrain neurons. Prog. Brain Res. 143, 221–230 doi: 10.1016/S0079-6123(03)43022-7. [PubMed]
  • Kakei S., Hoffman D. S., Strick P. L. (1999. ). Muscle and movement representations in the primary motor cortex. Science 285, 2136–2139 doi: 10.1126/science.285.5436.2136. [PubMed]
  • Kiehn O., Quinlan K. A., Restrepo C. E., Lundfald L., Borgius L., Talpalar A. E., Endo T. (2008. ). Excitatory components of the mammalian locomotor CPG. Brain Res. Rev. 57, 56–63 doi: 10.1016/j.brainresrev.2007.07.002. [PubMed]
  • Kim H. K., Biggs S. J., Schloeb D. W., Carmena J. M., Lebedev M. A., Nicolelis M. A. (2006. ). Continuous shared control stabilizes reach and grasping with brain-machine interfaces. IEEE Trans. Biomed. Eng. 53, 1164–1173 doi: 10.1109/TBME.2006.870235. [PubMed]
  • Kim S. P., Sanchez J. C., Erdogmus D., Rao Y. N., Wessberg J., Principe J. C., Nicolelis M. (2003. ). Divide-and-conquer approach for brain machine interfaces: nonlinear mixture of competitive linear models. Neural Netw. 16, 865–871 doi: 10.1016/S0893-6080(03)00108-4. [PubMed]
  • Kim S. P., Sanchez J. C., Rao Y. N., Erdogmus D., Carmena J. M., Lebedev M. A., Nicolelis M. A., Principe J. C. (2006. ). A comparison of optimal MIMO linear and nonlinear models for brain-machine interfaces. J. Neural. Eng. 3, 145–161 doi: 10.1088/1741-2560/3/2/009. [PubMed]
  • Kimura H., Fukuoka Y., Cohen A. H. (2007. ). Biologically inspired adaptive walking of a quadruped robot. Philos. Transact. A Math. Phys. Eng. Sci. 365, 153–170 [PubMed]
  • Koditschek D. E., Full R. J., Buehler M. (2004. ). Mechanical aspects of legged locomotion control. Arthropod Struct. Dev. 33, 251–272 doi: 10.1016/j.asd.2004.06.003. [PubMed]
  • Lacourse M. G., Cohen M. J., Lawrence K. E., Romero D. H. (1999. ) Cortical potentials during imagined movements in individuals with chronic spinal cord injuries. Behav. Brain Res. 104, 73–88 doi: 10.1016/S0166-4328(99)00052-2. [PubMed]
  • Lebedev M. A., Carmena J. M., O'Doherty J. E., Zacksenhouse M., Henriquez C. S., Principe J. C., Nicolelis M. A. (2005. ). Cortical ensemble adaptation to represent velocity of an artificial actuator controlled by a brain-machine interface. J. Neurosci. 25, 4681–4693 doi: 10.1523/JNEUROSCI.4088-04.2005. [PubMed]
  • Lebedev M. A., Denton J. M., Nelson R. J. (1994. ). Vibration-entrained and premovement activity in monkey primary somatosensory cortex. J. Neurophysiol. 72, 1654–1673 [PubMed]
  • Lebedev M. A., Fitzsimmons N. A., Drake W. G., Lehew G., Dimitrov D., Nicolelis M. A. (2007. ). Decoding bipedal locomotion patterns from cortical ensemble activity in rhesus monkeys. Abstr. Soc. Neurosci. 517, 23.
  • Lebedev M. A., Nicolelis M. A. (2006. ). Brain-machine interfaces: past, present and future. Trends Neurosci. 29, 536–546 doi: 10.1016/j.tins.2006.07.004. [PubMed]
  • Lebedev M. A., O'Doherty J. E., Nicolelis M. A. (2008. ). Decoding of temporal intervals from cortical ensemble activity. J. Neurophysiol. 99, 166–186 doi: 10.1152/jn.00734.2007. [PubMed]
  • Lunenburger L., Bolliger M., Czell D., Muller R., Dietz V. (2006. ). Modulation of locomotor activity in complete spinal cord injury. Exp. Brain Res. 174, 638–646 doi: 10.1007/s00221-006-0509-4. [PubMed]
  • Maki B. E., McIlroy W. E. (2007. ). Cognitive demands and cortical control of human balance-recovery reactions. J. Neural. Transm. 114, 1279–1296 doi: 10.1007/s00702-007-0764-y. [PubMed]
  • Matsuyama K., Mori F., Nakajima K., Drew T., Aoki M., Mori S. (2004. ). Locomotor role of the corticoreticular-reticulospinal-spinal interneuronal system. Prog. Brain Res. 143, 239–249 doi: 10.1016/S0079-6123(03)43024-0. [PubMed]
  • McCrea D. A., Rybak I. A. (2008. ). Organization of mammalian locomotor rhythm and pattern generation. Brain Res. Rev. 57, 134–146 doi: 10.1016/j.brainresrev.2007.08.006. [PubMed]
  • Mori F., Nakajima K., Tachibana A., Takasu C., Mori M., Tsujimoto T., Tsukada H., Mori S. (2004. ). Reactive and anticipatory control of posture and bipedal locomotion in a nonhuman primate. Prog. Brain Res. 143, 191–198 doi: 10.1016/S0079-6123(03)43019-7. [PubMed]
  • Mori F., Tachibana A., Takasu C., Nakajima K., Mori S. (2001. ). Bipedal locomotion by the normally quadrupedal Japanese monkey, M. Fuscata: strategies for obstacle clearance and recovery from stumbling. Acta Physiol. Pharmacol. Bulg. 26, 147–150 [PubMed]
  • Mori S., Matsui T., Mori F., Nakajima K., Matsuyama K. (2000. ). Instigation and control of treadmill locomotion in high decerebrate cats by stimulation of the hook bundle of Russell in the cerebellum. Can. J. Physiol. Pharmacol. 78, 945–957 doi: 10.1139/cjpp-78-11-945. [PubMed]
  • Morimoto J., Endo G., Nakanishi J., Cheng G. (2008. ). A biologically inspired biped locomotion strategy for humanoid robots: modulation of sinusoidal patterns by a coupled oscillator model. IEEE Trans. Rob. 24, 185–191 doi: 10.1109/TRO.2008.915457.
  • Morris M. E. (2006. ). Locomotor training in people with Parkinson disease. Phys. Ther. 86, 1426–1435 doi: 10.2522/ptj.20050277. [PubMed]
  • Muller-Putz G. R., Zimmermann D., Graimann B., Nestinger K., Korisek G., Pfurtscheller G. (2007. ). Event-related beta EEG-changes during passive and attempted foot movements in paraplegic patients. Brain Res. 1137, 84–91 doi: 10.1016/j.brainres.2006.12.052. [PubMed]
  • Murray M. P., Drought A. B., Kory R. C. (1964. ). Walking patterns of normal men. J. Bone Joint Surg. Am. 46, 335–360 [PubMed]
  • Murray M. P., Kory R. C., Sepic S. B. (1970. ). Walking patterns of normal women. Arch. Phys. Med. Rehabil. 51, 637–650 [PubMed]
  • Nakanishi J., Morimoto J., Endoa G., Cheng G., Schaal S., Kawato M. (2004. ). Learning from demonstration and adaptation of biped locomotion. Rob. Auton. Syst. 47, 79–91 doi: 10.1016/j.robot.2004.03.003.
  • National Research Council. (1996. ). Guide for the Care and Use of Laboratory Animals. National Academy Press; , Washington, DC:
  • Nicolelis M. A. (2001. ). Actions from thoughts. Nature 409, 403–407 doi: 10.1038/35053191. [PubMed]
  • Nicolelis M. A., Dimitrov D., Carmena J. M., Crist R., Lehew G., Kralik J. D., Wise S. P. (2003. ). Chronic, multisite, multielectrode recordings in macaque monkeys. Proc. Natl. Acad. Sci. U.S.A 100, 11041–11046 doi: 10.1073/pnas.1934665100. [PubMed]
  • Nielsen J. B. (2003. ). How we walk: central control of muscle activity during human walking. Neuroscientist 9, 195–204 doi: 10.1177/1073858403009003012. [PubMed]
  • Nightingale E. J., Raymond J., Middleton J. W., Crosbie J., Davis G. M. (2007. ). Benefits of FES gait in a spinal cord injured population. Spinal Cord 45, 646–657 doi: 10.1038/sj.sc.3102101. [PubMed]
  • Orlovsky G. N., Deliagina T. G., Grillner S. (1999. ). Neuronal Control of Locomotion – From Mollusc to Man. Oxford: , Oxford University Press;
  • Ounpuu S. (1994. ). The biomechanics of walking and running. Clin. Sports Med. 13, 843–863 [PubMed]
  • Patil P. G., Carmena J. M., Nicolelis M. A., Turner D. A. (2004. ). Ensemble recordings of human subcortical neurons as a source of motor control signals for a brain-machine interface. Neurosurgery 55, 27–35; discussion 35.–28 [PubMed]
  • Pearson O. R., Busse M. E., van Deursen R. W., Wiles C. M. (2004. ). Quantification of walking mobility in neurological disorders. QJM 97, 463–475 doi: 10.1093/qjmed/hch084. [PubMed]
  • Peikon I. D., Hanson T. L., Fitzsimmons N. A., Lebedev M. A., Nicolelis M. A. (2007. ). Real-time three-dimensional video tracking system for kinematic analysis of animal behavior. Abstr. Soc. Neurosci. 624, 18.
  • Perry J. (1992. ). Gait Analysis: Normal and Pathological Function. Thorofare, SLACK Inc.
  • Pfurtscheller G., Leeb R., Keinrath C., Friedman D., Neuper C., Guger C., Slater M. (2006. ). Walking from thought. Brain Res. 1071, 145–152 doi: 10.1016/j.brainres.2005.11.083. [PubMed]
  • Prilutsky B. I., Sirota M. G., Gregor R. J., Beloozerova I. N. (2005. ). Quantification of motor cortex activity and full-body biomechanics during unconstrained locomotion. J. Neurophysiol. 94, 2959–2969 doi: 10.1152/jn.00704.2004. [PubMed]
  • Prochazka A., Clarac F., Loeb G. E., Rothwell J. C., Wolpaw J. R. (2000. ). What do reflex and voluntary mean? Modern views on an ancient debate. Exp. Brain Res. 130, 417–432 doi: 10.1007/s002219900250. [PubMed]
  • Rossignol S., Schwab M., Schwartz M., Fehlings M. G. (2007. ). Spinal cord injury: time to move? J. Neurosci. 27, 11782–11792 doi: 10.1523/JNEUROSCI.3444-07.2007. [PubMed]
  • Saibene F., Minetti A. E. (2003. ). Biomechanical and physiological aspects of legged locomotion in humans. Eur. J. Appl. Physiol. 88, 297–316 doi: 10.1007/s00421-002-0654-9. [PubMed]
  • Sanchez J. C., Kim S. P., Erdogmus D., Rao Y. N., Principe J. C., Wessberg J., Nicolelis M. A. (2002. ). Input-output mapping performance of linear and nonlinear models for estimating hand trajectories from cortical neuronal firing patterns. In: Proceedings of the 2002 12th IEEE Workshop on Neural Networks for Signal Processing, Valais, Switzerland, pp. 139–148.
  • Santucci D. M., Kralik J. D., Lebedev M. A., Nicolelis M. A. (2005. ). Frontal and parietal cortical ensembles predict single-trial muscle activity during reaching movements in primates. Eur. J. Neurosci. 22, 1529–1540 doi: 10.1111/j.1460-9568.2005.04320.x. [PubMed]
  • Schwartz A. B., Cui X. T., Weber D. J., Moran D. W. (2006. ). Brain-controlled interfaces: movement restoration with neural prosthetics. Neuron 52, 205–220 doi: 10.1016/j.neuron.2006.09.019. [PubMed]
  • Scivoletto G., Di Donna V. (2008. ). Prediction of walking recovery after spinal cord injury. Brain Res. Bull. 78, 43–51 doi: 10.1016/j.brainresbull.2008.06.002. [PubMed]
  • Serruya M. D., Hatsopoulos N. G., Paninski L., Fellows M. R., Donoghue J. P. (2002. ). Instant neural control of a movement signal. Nature 416, 141–142 doi: 10.1038/416141a. [PubMed]
  • Shiavi R., Bugle H. J., Limbird T. (1987. ). Electromyographic gait assessment, part 1, adult EMG profiles and walking speed. J. Rehabil. Res. Dev. 24, 13–23 [PubMed]
  • Sparrow W. A., Tirosh O. (2005. ). Gait termination: a review of experimental methods and the effects of ageing and gait pathologies. Gait Posture 22, 362–371 doi: 10.1016/j.gaitpost.2004.11.005. [PubMed]
  • Stevens N. J. (2006. ). Stability, limb coordination and substrate type: the ecorelevance of gait sequence pattern in primates. J. Exp. Zoolog. A Comp. Exp. Biol. 305, 953–963 doi: 10.1002/jez.a.336. [PubMed]
  • Tachibana A., Mori F., Boliek C. A., Nakajima K., Takasu C., Mori S. (2003. ). Acquisition of operant-trained bipedal locomotion in juvenile Japanese monkeys (Macaca fuscata): a longitudinal study. Motor Control 7, 388–410 [PubMed]
  • Takakusaki K. (2008. ). Forebrain control of locomotor behaviors. Brain Res Rev 57, 192–198 doi: 10.1016/j.brainresrev.2007.06.024. [PubMed]
  • Tanji J., Wise S. P. (1981. ). Submodality distribution in sensorimotor cortex of the unanesthetized monkey. J. Neurophysiol. 45, 467–481 [PubMed]
  • Taylor D. M., Tillery S. I., Schwartz A. B. (2002. ). Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 doi: 10.1126/science.1070291. [PubMed]
  • Thorstensson A. (1986. ). How is the normal locomotor program modified to produce backward walking? Exp. Brain Res. 61, 664–668 doi: 10.1007/BF00237595. [PubMed]
  • Vaughan C. L. (2003. ). Theories of bipedal walking: an odyssey. J. Biomech. 36, 513–523 doi: 10.1016/S0021-9290(02)00419-0. [PubMed]
  • Vaughan C. L., Davis B. L., O'Connor J. C. (1992. ). Dynamics of Human Gait. Human Kinetics; Champaign, IL:
  • Velliste M., Perel S., Spalding M. C., Whitford A. S., Schwartz A. B. (2008. ). Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101 doi: 10.1038/nature06996. [PubMed]
  • Veneman J. F., Kruidhof R., Hekman E. E., Ekkelenkamp R., Van Asseldonk E. H., van der Kooij H. (2007. ). Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng. 15, 379–386 doi: 10.1109/TNSRE.2007.903919. [PubMed]
  • Wessberg J., Stambaugh C. R., Kralik J. D., Beck P. D., Laubach M., Chapin J. K., Kim J., Biggs S. J., Srinivasan M. A., Nicolelis M. A. (2000. ). Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408, 361–365 doi: 10.1038/35042582. [PubMed]
  • Widajewicz W., Kably B., Drew T. (1994. ). Motor cortical activity during voluntary gait modifications in the cat. II. Cells related to the hindlimbs. J. Neurophysiol. 72, 2070–2089 [PubMed]
  • Wieler M., Stein R. B., Ladouceur M., Whittaker M., Smith A. W., Naaman S., Barbeau H., Bugaresti J., Aimone E. (1999. ). Multicenter evaluation of electrical stimulation systems for walking. Arch. Phys. Med. Rehabil. 80, 495–500 doi: 10.1016/S0003-9993(99)90188-0. [PubMed]
  • Winter D. A., Yack H. J. (1987. ). EMG profiles during normal human walking: stride-to-stride and inter-subject variability. Electroencephalogr. Clin. Neurophysiol. 67, 402–411 doi: 10.1016/0013-4694(87)90003-4. [PubMed]
  • Wise S. P., Tanji J. (1981a. ). Neuronal responses in sensorimotor cortex to ramp displacements and maintained positions imposed on hindlimb of the unanesthetized monkey. J. Neurophysiol. 45, 482–500 [PubMed]
  • Wise S. P., Tanji J. (1981b. ). Supplementary and precentral motor cortex: contrast in responsiveness to peripheral input in the hindlimb area of the unanesthetized monkey. J. Comp. Neurol. 195, 433–451 doi: 10.1002/cne.901950305. [PubMed]
  • Wood-Dauphinee S., Exner G., Bostanci B., Exner G., Glass C., Jochheim K. A., Kluger P., Koller M., Krishnan K. R., Post M. W., Ragnarsson K. T., Rommel T., Zitnay G. (2002. ). Quality of life in patients with spinal cord injury – basic issues, assessment, and recommendations. Restor. Neurol. Neurosci. 20, 135–149 [PubMed]
  • Wu W., Black M. J., Mumford D., Gao Y., Bienenstock E., Donoghue J. P. (2004. ). Modeling and decoding motor cortical activity using a switching Kalman filter. IEEE Trans. Biomed. Eng. 51, 933–942 doi: 10.1109/TBME.2004.826666. [PubMed]
  • Yamaguchi T. (2004. ). The central pattern generator for forelimb locomotion in the cat. Prog. Brain Res. 143, 115–122 doi: 10.1016/S0079-6123(03)43011-2. [PubMed]
  • Yogev-Seligmann G., Hausdorff J. M., Giladi N. (2008. ). The role of executive function and attention in gait. Mov. Disord. 23, 329–342; quiz 472. doi: 10.1002/mds.21720. [PubMed]
  • Zajac F. E., Neptune R. R., Kautz S. A. (2002. ). Biomechanics and muscle coordination of human walking. Part I: introduction to concepts, power transfer, dynamics and simulations. Gait Posture 16, 215–232 doi: 10.1016/S0966-6362(02)00068-1. [PubMed]
  • Zatsiorky V. M., Werner S. L., Kaimin M. A. (1994. ). Basic kinematics of walking. Step length and step frequency. A review. J. Sports Med. Phys. Fitness 34, 109–134 [PubMed]
  • Zhang D., Zhu K. (2007. ). Modeling biological motor control for human locomotion with functional electrical stimulation. Biol. Cybern. 96, 79–97 doi: 10.1007/s00422-006-0107-3. [PubMed]
Articles from Frontiers in Integrative Neuroscience are provided here courtesy of
Frontiers Research Foundation
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph