Effects of fMRI-EEG Mismatches in Cortical Current Density Estimation Integrating fMRI and EEG: A Simulation Study

doi:10.1016/j.clinph.2006.03.031

Journal List > NIHPA Author Manuscripts

Clin Neurophysiol. Author manuscript; available in PMC 2007 August 13.

Published in final edited form as:

Clin Neurophysiol. 2006 July; 117(7): 1610–1622.

Published online 2006 June 9. doi: 10.1016/j.clinph.2006.03.031.

PMCID: PMC1945186

NIHMSID: NIHMS15327

Effects of fMRI-EEG Mismatches in Cortical Current Density Estimation Integrating fMRI and EEG: A Simulation Study

Zhongming Liu, Fedja Kecman, and Bin He^*

Department of Biomedical Engineering, University of Minnesota, MN, USA

*Correspondence: Bin He, Ph.D., University of Minnesota, Department of Biomedical Engineering, 7-105 BSBE, 312 Church Street, Minneapolis, MN 55455, Phone: 612-626-1115, Email: binhe/at/umn.edu

The publisher's final edited version of this article is available at Clin Neurophysiol.

See other articles in PMC that cite the published article.

Abstract

Objective

Multimodal functional neuroimaging by combining functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) has been studied to achieve high-resolution reconstruction of the spatiotemporal cortical current density (CCD) distribution. However, mismatches between these two imaging modalities may occur due to their different underlying mechanisms. The aim of the present study is to investigate the effects of different types of fMRI-EEG mismatches, including fMRI invisible sources, fMRI extra regions and fMRI displacement, on the fMRI-constrained cortical imaging in a computer simulation based on realistic-geometry boundary-element-method (BEM) model.

Methods

Two methods have been adopted to integrate the synthetic fMRI and EEG data for CCD imaging. In addition to the well-known 90% fMRI-constrained Wiener filter approach (Liu AK, Belliveau JW and Dale AM, PNAS, 95: 8945–8950, 1998), we propose a novel two-step algorithm (referred to as “Twomey algorithm”) for fMRI-EEG integration. In the first step, a “hard” spatial prior derived from fMRI is imposed to solve the EEG inverse problem with a reduced source space; in the second step, the fMRI constraint is removed and the source estimate from the first step is re-entered as the initial guess of the desired solution into an EEG least squares fitting procedure with Twomey regularization. Twomey regularization is a modified Tikhonov technique that attempts to simultaneously minimize the distance between the desired solution and the initial estimate, and the residual errors of fitness to EEG data. The performance of the proposed Twomey algorithm has been evaluated both qualitatively and quantitatively along with the lead-field normalized minimum norm (WMN) and the 90% fMRI-weighted Wiener filter approach, under repeated and randomized source configurations. Point spread function (PSF) and localization error (LE) are used to measure the performance of different imaging approaches with or without a variety of fMRI-EEG mismatches.

Results

The results of the simulation show that the Twomey algorithm can successfully reduce the PSF of fMRI invisible sources compared to the Wiener estimation, without losing the merit of having much lower PSF of fMRI visible sources relative to the WMN solution. In addition, the existence of fMRI extra sources does not significantly affect the accuracy of the fMRI-EEG integrated CCD estimation for both the Wiener filter method and the proposed Twomey algorithm, while the Twomey algorithm may further reduce the chance of occurring spurious sources in the extra fMRI regions. The fMRI displacement away from the electrical source causes enlarged localization error in the imaging results of both the Twomey and Wiener approaches, while Twomey gives smaller LE than Wiener with the fMRI displacement ranging from 1-cm to 2-cm. With less than 2-cm fMRI displacement, the LEs for the Twomey and Wiener approaches are still smaller than in the WMN solution.

Conclusions

The present study suggests that the presence of fMRI invisible sources is the most problematic factor responsible for the error of fMRI-EEG integrated imaging based on the Wiener filter approach, whereas this approach is relatively robust against the fMRI extra regions and small displacement between fMRI activation and electrical current sources. While maintaining the above advantages possessed by the Wiener filter approach, the Twomey algorithm can further effectively alleviate the underestimation of fMRI invisible sources, suppress fMRI spurious sources and improve the robustness against fMRI displacement. Therefore, the Twomey algorithm is expected to improve the reliability of multimodal cortical source imaging against fMRI-EEG mismatches.

Significance

The proposed method promises to provide a useful alternative for multimodal neuroimaging integrating fMRI and EEG.

Keywords: multimodal neuroimaging, EEG, fMRI, lead-field normalized minimum norm, point spread function, Twomey regularization, Wiener estimation, boundary element method

1. Introduction

In the past decade, tremendous efforts have been made to integrate information across multiple neuroimaging modalities during the same task with the aim to characterize brain function with high resolution in both spatial and temporal domains. Electroencephalography (EEG), as well as magnetoencephalography (MEG), can detect the rapid change of neurophysiologic processes but they suffer from poor spatial resolution due to their inherent mathematical difficulties. Functional magnetic resonance imaging (fMRI), by measuring hemodynamic responses related to brain activation, has the advantage of revealing anatomical details of neural activation but is limited by its low temporal resolution on the order of seconds (Kwong et al., 1992; Ogawa et al., 1992; Bandettini et al, 1992). Hence, combining the complementary information from EEG (or MEG) and fMRI holds potential for high-resolution spatiotemporal mapping of brain activity (Dale and Halgren, 2001; He and Lian, 2002).

A common strategy for fMRI-EEG integration is to use the results of fMRI analysis as a priori knowledge for imaging the continuous distribution of EEG sources over the entire cortical surface, namely fMRI-constrained cortical current density estimation (which we shall call cortical imaging subsequently) (Liu et al., 1998; Babiloni et al., 2003). This approach implies that cortical electrical sources can be modeled by hundreds or thousands of current dipoles evenly disposed over a triangulated cortical surface with the dipole orientation perpendicular to its local patch (Dale and Sereno, 1993). The problem of EEG-based cortical imaging is thus to estimate the strengths of these dipoles from the recorded electrical potentials over the scalp (He et al., 2002). Considering the close coupling between local hemodynamic response and neural activity as observed in animal and human experimental studies (Puce, 1995; Logothetis et al., 2001), it is expected that an enhanced spatial resolution of EEG-based cortical imaging should be achieved by incorporating the information from fMRI. However, the physical and physiological basis that accounts for the correlation between fMRI signal and neural electrical activity is not yet well understood, and the existing approaches for fMRI-EEG data fusion are mainly based on variations of weighted minimum norm methods. Typically, the fMRI “hotspots” (locations with significant hemodynamic change) are preferred in the fMRI-constrained cortical imaging, which can be accomplished, for example, by encoding the fMRI spatial information into the source covariance matrix and constructing an optimal linear estimator in the form of Wiener filter (Dale and Sereno, 1993; Liu et al, 1998). Previous simulation and experimental studies have discussed the efficacy of using fMRI constraints to enhance the spatial resolution of EEG- or MEG-based cortical imaging (Liu et al., 1998; Babiloni et al., 2003; Ahlfors and Simpson, 2004), and applications of this method have already advanced our understanding of the spatiotemporal pattern of brain activity and connectivity underlying perception, motion and cognition (Dale et al., 2000; Bonmassar et al., 2001; Jaaskelainen et al, 2004; Babiloni et al., 2005).

However, since EEG (or MEG) and fMRI measure physically different aspects of brain activities and they usually involve a variety of experimental setup and complicated mathematical procedures, it is important to consider possible misspecifications between multimodal signals, such as the presence of fMRI extra sources, fMRI invisible (missing) sources and displacement of fMRI activation from electrical current sources. The fMRI extra sources are the regions that are deemed as fMRI activations but produce no observable EEG/MEG signals, which may likely happen as some of the EEG sources are activated at certain time window while some others are activated at other time windows, but the fMRI activation map may include all of them together as it pools the activity over time due to its inherent lack of temporal resolution. The fMRI invisible (missing) sources are the generators of bioelectromagnetic signals that are not detected by fMRI. Typically, the fMRI invisible sources may happen if neurons are not activated long enough to induce a detectable increase of cerebral blood flow; or they may also happen if a cortical patch generates an EEG signal simply by increasing the firing synchronicity of a small percentage of neurons with little modification of its metabolic consumption (Babiloni and Cincotti, 2004). Also fMRI, by applying statistical methods, is assumed to reflect the integral energy consumption of local neural firing during an entire time period, while cortical imaging can actually be performed instant-by-instant since EEG is, in principle, capable of monitoring brain function virtually at every single time point. A cortical patch of little hemodynamic response can be interpreted as non-active in fMRI in the sense that the mean power of local electrical activity during the time course of the “event” is too small to induce significant BOLD change. But one can hardly say such a cortical region must be always “silent” during the whole process; it is likely that its instantaneous activity can give rise to signals observable in EEG. Additionally, the possible difference in the locations of neurons and the involved blood vessels can cause slight displacement of fMRI “hotspots” away from neural electrical generators.

To deal with the possible fMRI-EEG (or MEG) mismatches, previous studies suggested using fMRI partial constraint, namely without excluding non-fMRI locations from having electrical sources, to avoid over-constraining the solution (Liu et al, 1998; Dale et al, 2000). With the aim to provide the best empirical value of fMRI partial weighting, Liu et al performed a Monte Carlo simulation on MEG-fMRI combined cortical imaging with randomized source placement and in consideration of both valid and invalid fMRI spatial priors. They found that the estimation error as measured by “crosstalk” metric was substantially higher for fMRI invisible sources than for fMRI visible sources. Their results also suggested that using 90% partial fMRI constraint provided the statistically best compromise (among several other fMRI weighting values) between sensitivities to crosstalk from fMRI visible and invisible sources at realistic signal-to-noise-ratio (SNR) and fMRI extent (Liu et al, 1998). An independent simulation study (Babiloni et al, 2003) on EEG-fMRI integration concluded consistently an optimal fMRI strength equal to 10, which was equivalent to 90% partial fMRI weighting reported in (Liu et al, 1998), while more simulation factors (variable SNRs, sensors numbers and different computational schemes for fMRI priors) were taken into account. More recently, a qualitative simulation study (Ahlfors and Simpson, 2004) also showed that the setting of 90% fMRI weighting gave smallest error of source reconstruction. Other than these computer simulation studies, the 90% fMRI weighting strategy has been successfully applied in a variety of experimental studies combining fMRI with EEG (or MEG) (Dale et al., 2000; Bonmassar et al., 2001; Jaaskelainen et al, 2004; Babiloni et al., 2005).

In this paper, we propose an alternative algorithm using Twomey regularization (Twomey, 1963), instead of applying 90% partial fMRI weighting, to correct the fMRI-biased source estimates from being distorted by the presence of fMRI-EEG mismatches. Unlike conventional Tikhonov regularization which imposes a side constraint on magnitudes or derivatives of the solution, this approach minimizes the difference between the desired solution and a rough estimate of the solution, as well as the residual errors in the least square sense. In the context of fMRI-EEG data fusion, Twomey algorithm consists of two steps. In the first step, a “hard” spatial prior derived from fMRI is imposed to solve the EEG inverse problem with a reduced source space; in the second step, the fMRI constraint is removed and the source estimate from the first step is re-entered as the initial guess of the desired solution into an EEG least square fitting procedure with Twomey regularization.

In this paper, we re-addressed the effect of fMRI-EEG mismatches on fMRI-constrained cortical current density imaging by computer simulation based on a realistic-geometry boundary element method (BEM) model of human head. For the sake of comparison and evaluation, the simulation in the present study was conducted in a similar fashion as (Liu et al, 1998), in which we employed repeated and randomized source placement and synthetic fMRI regions with or without introducing mismatches between fMRI activation and EEG source locations. Also similarly, we used the point spread function (PSF) to evaluate the performance of two fMRI-EEG integration algorithms (90% fMRI-weighted Wiener filter approach and the proposed Twomey algorithm) against the presence of fMRI invisible and extra sources as in (Liu et al, 1998). Moreover, we imposed displacement (less than 2-cm) of fMRI activation away from current sources and assessed, by means of the localization error (LE), the effects of displaced fMRI spatial priors on localization of electrical sources.

2. Methods

2.1 Cortical Current Density Estimation

Cortical current density (CCD) estimation, known as an EEG inverse problem, is to solve the following linear equation:

(1)

where Φ is the vector of instantaneous EEG recordings, J is the vector of unknown dipole moments, b is the noise vector, and L is the transfer matrix with each column corresponding to the scalp potential pattern generated by a given dipole and each row specifying the sensitivity pattern of a given sensor.

Such an EEG inverse problem is underdetermined and ill-posed; therefore additional constraints and/or a priori information are always needed in order to obtain a unique and stable solution. In particular, if a priori information exists about the auto-covariance matrix of source and noise, the optimal linear inverse operator T in the form of Wiener filter can be written as (2):

(2)

where C is the noise covariance matrix and R is the source covariance matrix.

2.2 90% fMRI-constrained Wiener Filter

The integration of EEG and fMRI works under the hypothesis that the regions with great BOLD-fMRI activation have larger possibility of being electrically active over the time period of interest. This hypothesis is reasonable in that neural activity, modulating neuronal firing and generating EEG signals, increases the demands for oxygen and induces larger cerebral blood flow, and consequently produces larger fMRI responses (Babiloni et al, 2005).

The coupling between brain electrical activity and fMRI measurement suggests that we can use fMRI spatial information to bias the EEG source estimation to those locations of significant BOLD response. This can be accomplished by using the fMRI response as the prior estimate of the local electrical activity integrated over time. Since a voxel in fMRI must be either active or non-active, the fMRI-derived source covariance can be written as (3)

(3)

where the diagonal elements of R_f are set to a positive value f only for those dipoles whose locations are deemed active in fMRI, while other diagonal elements are set to 1, and r is a regularization parameter. The inverse operator in Eq.(2) is also the minimizer of (4)

(4)

The parameter f controls the amount of bias towards fMRI active locations, which depends on the confidence in the hypothesis that neuronal and hemodynamic activities are co-located. Liu et al previously suggested a 90% fMRI partial weighting strategy, which is equivalent to f =10, to accommodate the possible fMRI-EEG mismatch (Liu et al, 1998). From their results of Monte Carlo simulation, they concluded that 90% fMRI partial weighting provided the best (among other values of fMRI weighting) trade-off between a high spatial resolution and a minimal distortion due to fMRI-EEG mismatches. Consistent conclusion was recently obtained in independent simulation studies (Babiloni et al, 2003; Ahlfors and Simpson, 2004). The 90% fMRI-constrained Wiener filter approach was also successfully applied in a variety of experimental studies (Dale et al, 2000; Bonmassar et al, 2001; Jaaskelainen et al, 2004; Babiloni et al, 2005).

Let f =10, the regularization matrix R_f in (4) can be straightforwardly derived from fMRI map. The regularization parameter r can be chosen by use of the “L-curve” method (Hansen, 1992), which plots the first term of residual norm versus the regularized norm of the solution in log-log scale. Finally, the solution of 90% fMRI-constrained Wiener estimation can be written as Eq.(5):

(5)

2.3 Twomey Algorithm

Unlike the 90% fMRI-constrained Wiener filter approach, the Twomey algorithm consists of two steps. In the first step, a “hard” fMRI constraint is imposed by setting the fMRI weighting factor f as a large value 100. Such a hard spatial constraint effectively reduces the source space to the regions deemed as active in fMRI, while all the other non-fMRI locations are highly penalized. As a consequence, the source reconstruction J^fMRI (obtained from Eq.(6)) has a spatial resolution as high as fMRI.

(6)

However, this overconstrained high-resolution solution is highly sensitive to the existence of fMRI-EEG mismatches. To overcome the possible distortion in the second step, J^fMRI is re-entered as an initial estimate into the EEG least square fitting procedure with Twomey regularization. Instead of imposing constraints on the magnitude of the solution or on its derivatives, this method minimizes the difference between the desired solution Ĵ and a rough estimate J^fMRI , as well as the residual error in the least square sense. The Twomey regularized least square estimation can be written as (7)

(7)

Here, the weighting matrix C^−1/2 is still introduced to compensate for different levels of noise contamination in the sensor space (Fuchs et al, 1998; Babiloni et al, 2003). We can see that the influence from fMRI spatial priors has been softened in the second step of Twomey algorithm, which yields the robustness of the estimated solution in the face of the possible fMRI-EEG misspecifications. If λ, the regularization parameter that balances the two terms in the cost function, is chosen to be a large value, the solution is forced to be very close to J^fMRI , and it should come as no surprise to expect that the solution is dominated by the fMRI spatial prior and may be still sensitive to the presence of fMRI-EEG mismatches sources. On the contrary, if λ is small, the solution tends to shift away from J^fMRI in a way to reduce the residual norm, as a result, the source estimation has good chance to be corrected against the influence from fMRI-EEG mismatches through better fitting to EEG recordings. And if λ is virtually close to zero, then the solution turns to a purely least square inverse solution, suffering from unstableness and low spatial resolution. Clearly, the trade-off is controlled by λ.

Similar to the way adopted in 90% fMRI-constrained Wiener filter approach, the regularization parameter in (7) was obtained by the “L-curve” approach (Hansen, 1992), which plots the first term of residual norm versus the norm of the discrepancy between the desired solution and the initial solution in log-log scale. The value of regularization parameter at the corner of the L-shaped curve represents the desired setting that compromises the minimization of the two terms in (7). Given λ is known, the solution to (7) is

(8)

where J^fMRI = T^fMRIΦ. In fact, this solution is still a linear inverse solution and the inverse operator is written as (9)

(9)

2.4 Computer Simulation

Through computer simulation, the performance of the proposed method, abbreviated as Twomey, was examined both qualitatively and quantitatively in comparison with lead field normalized minimum norm approach (also known as “weighted minimum norm”; Lawson and Hanson, 1974) and the 90% fMRI-constrained Wiener estimation, denoted as WMN and Wiener, respectively. The simulation was conducted on a realistic-geometry BEM-model reconstructed from the T1-weighted MRI images of a human subject (128 slices, 256×256 pixels; 1.6 mm thickness, 1.17×1.17 mm in-plane resolution). The conductivities of the scalp, skull and brain were taken as 0.33, 0.0165 and 0.33 S/m, respectively (Oostendorp et al., 2000; Lai et al., 2005). The cortical surface was also segmented from the MRI images obtaining a triangulated mesh with about 20,199 surface triangles. The cortical surface was down-sampled to 3,906 dipole locations. The forward problem for each dipole at these locations was solved using the BEM model (Hamalainen and Sarvas, 1989; He et al., 1987), and the transfer matrix was calculated accordingly.

The simulation was based on randomized placement of EEG dipole point sources and synthetic fMRI activations in consideration of three types of fMRI-EEG mismatches, described in the Introduction section. The simulation of these fMRI and EEG signals was summarized as below.

fMRI invisible sources

Five EEG sources were randomly selected from the entire cortical surface. The fMRI activation map was simulated to cover 4 or 3 out of these five sources, leaving 1 or 2 EEG sources to behave as fMRI invisible (missing) sources. The simulated fMRI activation had a fixed extent of 1-cm radius and it was centered at the corresponding “true” source location.

fMRI extra regions

Five cortical locations were randomly selected from the entire cortical surface. Five fMRI activated areas were simulated to respectively be centered at the selected cortical locations and have a fixed extent of 1-cm radius. Either 5, or 4 or 3 current dipole sources were placed at the centers of different fMRI regions, and consequently accurate fMRI coverage, or 1 or 2 fMRI extra regions were simulated.

fMRI displacement

An EEG point source was randomly placed on the cortical surface. The fMRI activation with fixed extent of 1-cm radius was either centered at the EEG source location, or was displaced by 0.5, 1.0, 1.5 or 2.0 cm.

All the placed dipole sources have equal unitary strength. The instantaneous EEG measurements were generated through forward calculation with additive Gaussian white noise at several SNRs (3, 5 or 10). And for each simulation condition described above, the random source placement was repeated for 100 times.

To assess the effects of different types of fMRI-EEG mismatches and to compare the performance of different imaging algorithms, we adopted two performance metrics for the evaluation purpose. We used “point spread function” (PSF) in the simulation for fMRI missing or extra regions, and used “localization error” (LE) in the simulation for fMRI displacement.

Due to the linearity of both the forward and inverse problems, a measure of estimation error defined as “point spread function” (PSF) (Pascual-Marqui, 1999; Dale et al, 2000; Liu et al, 2002) can be calculated for each source location. Mathematically,

(10)

where (TL)_ji describes the sensitivity of the estimate at a location j to activity at location i . In language, the PSF specifies the ratio of the energy that arises from the actual current dipole at a given location but spreads onto the estimates at all other locations to the energy that only contributes to the source estimate at the same location. It is a measure of the spatial blurring of the true activity at any given position. Therefore, a location with lower PSF is expected to have a smaller spatial extent and higher estimation accuracy.

To assess the induced error of localization of a point-like current source due to fMRI displacement, “localization error” (LE) was calculated as distance between the location of the current dipole and the center of gravity of the source estimate (Baillet et al, 2001; Im et al, 2003). The distributed source image was thresholded such that 80% of the power of the reconstructed source distribution was remained.

3. Results

Effects of fMRI invisible sources

An example of cortical current density results using three different inverse algorithms is illustrated in Fig. 1

. Five dipole sources with unitary strength were randomly sampled from the cortical surface. EEG measurements at 128 electrodes were simulated at SNR=5, which was a value consistent with typical EEG experiments. From Fig. 1

.C), WMN tended to produce extended areas of current density so that two dipoles that were proximal to each other could hardly be differentiated in the reconstructed CCD map. When one of the five dipole sources was invisible in fMRI as shown in Fig. 1.D

), the 90% fMRI-constrained Wiener filter approach could substantially improve the spatial resolution of imaging fMRI visible sources, but the fMRI invisible source was considerably underestimated. In contrast, the Twomey solution had high resolution for fMRI visible sources comparable to that in the Wiener estimation, and noticeably it also accentuated and improved the estimation of activity associated with the fMRI invisible source. The similar phenomena were also observed when two of the five sources were invisible in fMRI as shown in the third row of images in Fig 1

Figure 1

A) Source and electrode configuration. A cortical current density source model is used in the inverse problem. Five dipoles with unitary strength and outward normal direction to the cortical surface were sampled randomly and used in the forward problem; (more ...)

Fig. 2

shows the PSF values for all three algorithms (WMN, Wiener and Twomey) in 100 random trials, in each of which 5 unitary point sources were randomly sampled from the cortical surface to generate scalp potential measurements at SNR=5. The simulated fMRI activations covered 4 out of 5 dipole sources, leaving one source to behave as an fMRI missing source. Fig. 2.A

) shows the mean PSF averaged over the four fMRI visible sources in each trial. Clearly, both Wiener and Twomey resulted in much smaller PSF of fMRI visible sources than WMN did. On the other hand, the trial-wise PSF of the fMRI missing source in the 90% fMRI weighted Wiener estimation was considerably larger PSF in most random trials than the EEG-alone WMN solution (as shown in Fig. 2.B

). Obviously, the Twomey algorithm had superior performance over the Wiener estimation in terms of revealing the fMRI-missed EEG source, as we can see that in most random trials, the values of PSF of the fMRI invisible source for Twomey was much smaller than those for Wiener, and in many trials they were even smaller (or at least comparable) than in the WMN solution. Fig. 2.C

) illustrates the PSF of the fMRI invisible source for both the Wiener and Twomey algorithms as a function of the respective PSF value in the WMN solution in the same trial. The PSF for the Twomey solution trended to decrease (or increase) with the drop (or rise) of the respective PSF in the WMN solution, meaning that the level by which the fMRI invisible sources could be revealed by the Twomey algorithm was dependent on whether or not they could be effectively imaged in the EEG-alone WMN solution. Moreover, as for the fMRI invisible sources that were more “visible” (with smaller PSF values) in the WMN solution, it was more obvious to observe the improvement due to the use of the Twomey algorithm relative to the Wiener estimation in imaging fMRI missing sources. If the fMRI invisible sources could hardly be imaged even in the non-fMRI-biased WMN solution, the Twomey algorithm was also less helpful in revealing such fMRI-missed sources. But under such situations, these sources were usually even more severely underestimated in the 90% fMRI weighted Wiener estimation.

Figure 2

The values of PSF for either fMRI visible sources or fMRI invisible sources in random source configurations (SNR=5, 5 dipoles, 1 fMRI missing source and fixed fMRI extent with 1-cm radius). A) The scatter plot of the mean PSF averaged over four fMRI visible (more ...)

The effects of fMRI invisible sources (either 1 or 2 out of 5) at different SNR (3, 5 and 10) were assessed by the PSF (or averaged PSF) of the fMRI invisible source(s). The mean PSF averaged over 100 random trials are listed in Table 1. For all three methods, higher SNR reduced the PSF of fMRI invisible sources, most dramatically for the Wiener estimation. The underestimation of fMRI invisible sources in the Wiener solution was again confirmed by the fact that the average PSF of fMRI invisible sources in the Wiener solution was always larger than in the WMN solution regardless of the number of fMRI missing sources (either 1 or 2) or different levels of SNR (3, 5 and 10). Conversely, the average PSF of fMRI invisible sources in the Twomey solution was considerably smaller than in the Wiener estimation, especially at low SNR (e.g. 3 and 5). The results of one-way ANOVA and its following multiple comparisons are also listed in Table 1. The performance of these three imaging algorithms (WMN, Wiener and Twomey) was significantly different at all three levels of SNR. In terms of imaging fMRI invisible sources, WMN was significantly better than Wiener for all different SNRs and different numbers of fMRI missing sources. At low SNR (3 and 5), there was no significant difference between WMN and Twomey, while Twomey was significantly better than Wiener. At high SNR=10, WMN was significantly better than Twomey when 2 sources are missing in fMRI, but that was not the case for only 1 missing source, whereas no significant improvement could be concluded for Twomey vs. Wiener for both cases.

Table 1

Effects of fMRI invisible sources assessed by PSF

Effects of fMRI extra sources

We further investigated the effects of the extra fMRI areas that did not cover any EEG sources on imaging the current density distribution at a single instant. Fig. 3

shows an example where three unitary dipole sources were randomly placed on the folded cortical surface. Similar to Fig.1

, the WMN solution exhibited the extended and smooth source image as shown in Fig. 3.B

. When the fMRI active regions (with 1-cm radius) exactly covered the EEG sources, both the Wiener and Twomey algorithms gave rise to similar CCD reconstruction with much more focalized source image around real source locations than the WMN solution. When one or two extra fMRI regions existed as shown in Fig. 3.F and 3.I

, the estimation of the fMRI visible sources by Wiener still maintained a high resolution, suggesting that the extra fMRI regions did not affect the imaging of real EEG sources that were covered by the fMRI active areas. However, slightly spurious sources in the Wiener solution may appear in the extra fMRI areas, particularly as shown in Fig. 3.J

. By use of the Twomey algorithm, the spurious sources induced by the fMRI extra areas appeared with smaller intensity in the reconstructed CCD map. Therefore, the fMRI extra sources did not significantly distort the fMRI-EEG integrated CCD estimation using either the new Twomey algorithm or the conventional Wiener filter approach, although slightly spurious sources may occur within the extra fMRI regions in the Wiener solution, which could be further diminished by using the alternative Twomey algorithm.

Figure 3

A) Source and electrode configuration. Three dipoles with unitary strength and outward normal direction to the cortical surface were sampled randomly and used in the forward problem. Scalp potentials were simulated at SNR=5. B) Cortical current density (more ...)

To confirm the above finding, we also conducted the repeated randomized simulation at several SNRs (3, 5 or 10). Specifically, five fMRI regions with fixed 1-cm-radius extent were randomly chosen from the cortical surface. Either 5, or 4 or 3 current dipoles were placed at the centers of each fMRI activation area, simulating accurate fMRI location priors, or 1 (or 2) extra fMRI coverage(s) that had no associated EEG sources. We calculated the mean PSF averaged over all the locations where the EEG sources were placed to quantify the amount of distortion introduced by the fMRI extra sources upon imaging the real EEG sources.

The results averaged over 100 random source placements are summarized in Table 2. Combining fMRI spatial information in both the Wiener and Twomey algorithms resulted in significantly smaller PSF of fMRI visible sources relative to the EEG-alone WMN solution, regardless of the existence or lack of the extra fMRI regions or different levels of SNR. The results of one-way ANOVA and its following multiple comparisons strongly concluded that the factor of METHODS (WMN, Wiener and Twomey) had a significant effect on the PSF for all of the simulation conditions. Both Wiener and Twomey were significantly better than WMN, while there was no significant difference between Wiener and Twomey, for all the levels of SNR and numbers of fMRI extra regions considered in the simulation. Three-way ANOVA with the factors (METHODS, SNR and number of fMRI extra regions), also concluded that METHODS (F=491.21, p<10⁻⁹) and SNR (F=21.12, p<0.0001) had significant effects, while the number of fMRI extra regions was insignificant (F=1.32, p=0.21).

Table 2

Effects of fMRI extra regions assessed by PSF

Effects of fMRI displacement

Fig. 4

summarizes the effects of displacement of fMRI active region relative to the associated electric current dipole source on the localization error of fMRI-EEG integrated imaging approaches (Wiener and Twomey), in comparison with the EEG-alone WMN solution. The bars in Fig. 4

represent the mean LE averaged over 100 random trials for each of the three imaging methods, with or without different levels of fMRI displacement (5, 10, 15 or 20 mm). Without fMRI prior, the EEG-alone WMN solution had a mean localization error of around 1-cm, and the error tended to decrease with higher SNR (e.g. LE was around 9-mm at SNR=10). With accurate fMRI priors, the fMRI-constrained solutions in both Wiener and Twomey had much smaller mean LE (about 4 to 5-mm) than WMN, and Wiener gave slightly smaller LE than Twomey. For both Wiener and Twomey, the LE also tended to decrease with higher SNR.

Figure 4

For all three imaging algorithms, the mean localization errors averaged over 100 random trials are shown as color-coded bars. The localization error is illustrated as a function of METHODS (Twomey, Wiener and WMN) and fMRI displacement (distance of fMRI (more ...)

If the fMRI activation was displaced away from the electrical source, the LEs for both Wiener and Twomey increased when the distance of displacement became larger. With a smaller fMRI displacement (e.g. 5-mm), the LEs for Wiener and Twomey were very close. But with a relatively larger fMRI displacement (e.g. >1.0cm), Twomey gave smaller LE than Wiener. Moreover, even with the fMRI displacement as large as 2-cm, Wiener and Twomey still have smaller LE than the WMN solution, except at SNR=10 where Wiener had larger LE than WMN. This suggests that the incorporation of fMRI spatial prior with even less than 2-cm displacement was still helpful to localize the electrical sources, compared to the WMN solution without combining fMRI.

4. Discussion

The aim of the integration of fMRI and EEG shall be to provide a more reliable functional neuroimaging technique and achieve high-resolution spatiotemporal mapping of brain activities. The existing methods are mainly based on fMRI-constrained Wiener estimation (or fMRI-weighted minimum norm), namely assigning different weighting factors to the source location covered or not covered by the fMRI active areas. However, the presence of fMRI-EEG mismatches (especially for the fMRI invisible sources) leads to distorted current density reconstruction, which limits the reliability of such a multimodal approach. Previous efforts have been made in finding a fixed fMRI-weighting factor that possesses the best trade-off between utilization of the high spatial resolution provided by fMRI and sensitivity to the possible fMRI-EEG (or MEG) mismatches. In contrast, we proposed, in the present study, to apply a two-step Twomey algorithm in an attempt to correct such distortion without compromising the high resolution provided by the consistent fMRI constraints. Previously, similar concept has been adopted to include the constraint of time progress in electrocardiography inverse problem (Oster and Rudy, 1992). It has been clearly shown in the present computer simulation that Twomey regularization reduces the point spread function from fMRI invisible sources without affecting the estimation accuracy for fMRI visible sources. Our simulation results also suggest that the presence of fMRI extra sources has little effect on the accuracy of fMRI-EEG integrated CCD imaging, although it may bring slightly larger chance of having spurious sources in the extra fMRI regions, which is consistent with previous stimulation studies (Liu et al, 1998; Ahlfors and Simpson, 2004). Nevertheless, under this situation, Twomey regularization can also help diminish the spurious sources. Our simulation on effects of fMRI displacement shows that the possible distance between fMRI activation and EEG sources causes larger error for localizing EEG sources using fMRI-EEG integrated approaches, and the higher fMRI displacement results in larger localization error. However, even with a limited fMRI displacement (less than 2-cm), the fMRI prior is still helpful to enhance the EEG source localization capability, compared with the WMN solution without combining information from fMRI. Also notably, Twomey algorithm exhibits higher robustness than 90% fMRI-constrained Wiener filter approach against the distortion due to fMRI displacement.

Overall, our simulation results indicate that the most problematic fMRI-EEG mismatch is fMRI invisible EEG sources. Although applying Twomey regularization can alleviate the distortion due to fMRI invisible sources, our simulation results also indicate that fMRI invisible sources are still underestimated since the PSF of fMRI invisible sources is still higher than that of fMRI visible sources and even higher than the PSF in the weighted minimum norm solution at some cases. Therefore, more efforts are still required to further reduce estimation error due to this type of fMRI-EEG mismatch. But considering the dominant advantage of high-resolution estimation of fMRI visible sources, the overall performance of fMRI-EEG integrated cortical current density estimation superiors to that of using EEG data alone, as long as fMRI spatial information is, in general, consistent with locations of neural activity.

The proposed Twomey algorithm is a two-step approach to combine the fMRI and EEG data. The solution with hard fMRI constraint resulting from the first step is re-entered to the second step as the initial estimate in a modified Tikhonov formulation utilizing Twomey regularization. The reason why adding the second step can enhance the robustness of the CCD imaging against inconsistent fMRI information is because that in the second step fMRI constraint is not directly involved any more. Noticeably, the eventual inverse operator is still a linear one. This property essentially differentiates this algorithm from other iterative algorithms such as FOCUSS (Gorodnitsky and Rao, 1997; Rao and Kreutz-Delgado, 1999), in which the previous solution is re-entered to the current estimation procedure as a weighting matrix and eventually arrives at a nonlinear inverse operator equivalent to L-p norm (Rao and Kreutz-Delgado, 1999).

From the perspective of Bayesian estimation, the two-step Twomey algorithm provides a different way of rendering the prior Gaussian probability density of current density at each source location, by controlling its source variance and mean separately in two steps. In the first step, which is identical to the conventional Wiener filter method, the probability densities are assumed to be zero-mean with the variance depending on whether the source location is inside the fMRI activation or not. Specifically, the source variance is larger for fMRI active locations than that for fMRI non-active locations. As a consequence, the prior probability distribution for fMRI invisible locations are much shaper and more concentrated to its mean value 0 while the prior probability distribution for fMRI visible locations have much wider spread. Therefore, the large amplitude of non-zero current density estimate is penalized in the fMRI invisible locations but rewarded in the fMRI visible locations. That is also the reason why the fMRI invisible sources tend to be underestimated in the Wiener filter method. However, in the second step, the prior probability distribution for different source locations is assumed to have the identical variance but different means as given by the initial source estimate in the first step. The use of Twomey regularization will yield more robust solution in that the location-wise hyperparameters of the prior probability are the means in the second step, which are carried by the initial estimate from the first step and convey the integrated information from both fMRI and EEG. Since the locations with the prior means far away from zero are more likely having a larger absolute amplitude of current density estimate, the source locations that are favored by the fMRI spatial constraint and have larger estimate in the first step are still indirectly favored in the second step, resulting in the low PSF for fMRI visible sources for the Twomey algorithm. However, the relaxation from the constraints on the variance makes the prior probability distribution equally spread out, providing better chance for the non-fMRI locations to have a large non-zero source estimate so that the fMRI-invisible sources may be recovered.

In the present study, we compared the proposed Twomey algorithm with the Wiener filter approach with fixed 90% fMRI partial weighting. The 90% fMRI weighting resulted from a series of simulations previously reported in (Liu et al, 1998; Babiloni et al, 2003; Ahlfors and Simpson, 2004). However, the fMRI-constrained current density estimation can also be formulated as a least squares estimation plus two L-2 norm side constraints and two associated regularization parameters (Ahlfors and Simpson, 2004), in which the fMRI-weighting factor can be considered as a regularization parameter. Recent development on data-driven restricted maximum likelihood (ReML) method (Phillips et al, 2002; Phillips et al, 2005; Mattout et al, in press) for jointly choosing multiple regularization parameters may be applied to adaptively select the value of fMRI partial weighting instead of fixing it empirically. Moreover, for both Wiener filter and Twomey algorithm, a variety of methods can be used to choose the regularization parameter, which is apparently an influential factor on the accuracy of inverse solutions. Addressing the difference and superiority of one method over the others is a complex issue and beyond the scope of the present study, therefore we choose to use a well-known heuristic L-curve method for selecting regularization parameters in both Wiener and Twomey algorithms.

Although point sources are not physiologically realistic, they are included in our current study to allow for repeated random source placement and a simple quantitative comparison between different linear inverse algorithms. In our future study, we also plan to try distributed spatiotemporal activation patterns to simulate the real brain sources, and the relative error or the correlation coefficient, or ROC analysis can be used to measure the estimation accuracy.

Acknowledgments

We are grateful to the reviewers for constructive comments. This work was supported in part by NSF-BES-0411898 and NIH EB00178, and by the Biomedical Engineering Institute of the University of Minnesota.

Footnotes

Acknowledgements: This work was supported in part by NSF BES-0411898, NIH EB00178, and by the Biomedical Engineering Institute of the University of Minnesota.

References

Ahlfors SP, Simpson GV. Geometrical interpretation of fMRI-guided MEG/EEG inverse estimates. NeuroImage. 2004;22:323–332. [PubMed]
Babiloni F, Babiloni C, Carducci F, Romani GL, Rossini PM, Angelone LM, Cincotti F. Multimodal integration of high-resolution EEG and functional magnetic resonance imaging data: a simulation study. NeuroImage. 2003;19:1–15. [PubMed]
Babiloni F, Cincotti F. Multimodal imaging from neuroelectromagnetic and functional magnetic resonance recordings. In: He B, editor. Modeling and Imaging of Bioelectrical Activity Principle and Applications. 2004. pp. 251–280.
Babiloni F, Cincotti F, Babiloni C, Carducci F, Mattia D, Astolfi L, Basilisco A, Rossini PM, Ding L, Ni Y, Cheng J, Christine K, Sweeney J, He B. Estimation of the cortical functional connectivity with the multimodal integration of high-resolution EEG and fMRI data by directed transfer function. NeuroImage. 2005;24:118–131. [PubMed]
Baillet S, Riera JJ, Marin G, Mangin JF, Aubert J, Garnero L. Evaluation of inverse methods and head models for EEG source localization using a human skull phantom. Phys Med Biol. 2001;46:77–96. [PubMed]
Bandettini PA, Wong EC, Hinks RS, Tikofsky RS, Hyde JS. Time course EPI of human brain function during task activation. Magn Reson Med. 1992;25:390–397. [PubMed]
Bonmassar G, Schwartz DP, Liu AK, Kwong KK, Dale AM, Belliveau JW. Spatiotemporal brain imaging of visual-evoked activity using interleaved EEG and fMRI recordings. NeuroImage. 2001;13:1035–1043. [PubMed]
Dale AM, Sereno MI. Improved localization of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction: A linear approach. J Cogn Neurosci. 1993;5:162–176.
Dale AM, Liu AK, Fischl BR, Buckner RL, Belliveau JW, Lewine JD, Halgren E. Dynamic statistical parametric mapping: combining fMRI and MEG for high-resolution imaging of cortical activity. Neuron. 2000;26:55–67. [PubMed]
Dale AM, Halgren E. Spatiotemporal mapping of brain activity by integration of multiple imaging modalities. Curr Opin Neurobiol. 2001;11:202–207. [PubMed]
Foster M. An application of the Wiener-Kolmogorov smoothing theory to matrix inversion. J Soc Indust Appl Math. 1961;9(3):387–392.
Fuchs M, Wagner M, Wischmann HA, Kohler T, Theiben A, Drenckhahn R, Buchner H. Improving source reconstructions by combining bioelectric and biomagnetic data. Electroenceph Clin Neurophysiol. 1998;107:93–107. [PubMed]
Fujimaki N, Hayakawa T, Nielsen M, Knosche TR, Miyauchi S. An fMRI-constrained source analysis with procedures for dividing and grouping activation. NeuroImage. 2002;17:324–343. [PubMed]
Gorodnitsky IF, Rao BD. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans Signal Proc. 1997;45(3):600–616.
Hamalainen MS, Sarvas J. Realistic conductivity geometry model of the human head for interpretation of neuromagnetic data. IEEE Trans Biomed Eng. 1989;36:165–171. [PubMed]
Hansen PC. Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 1992;34:561–580.
He B, Musha T, Okamoto Y, Homma S, Nakajima Y, Sato T. Electric dipole tracing in the brain by means of the boundary element method and its accuracy. IEEE Trans Biomed Eng. 1987;34:406–414. [PubMed]
He B. Brain Electric Source Imaging: Scalp Laplacian Mapping and Cortical Imaging. Crit Rev Biomed Eng. 1999;27:149–188. [PubMed]
He B, Yao D, Lian J. High-resolution EEG: on the cortical equivalent dipole layer imaging. Clin Neurophysiol. 2002;113:227–235. [PubMed]
He B, Lian J. High-resolution spatio-temporal functional neuroimaging of brain activity. Crit Rev Biomed Eng. 2002;30:283–306. [PubMed]
Im C-H, An K-O, Jung H-K, Kwon H, Lee Y-H. Assessment criteria for MEG/EEG cortical patch tests. Phys Med Biol. 2003;48(15):2561–2573. [PubMed]
Jaaskelainen IP, Ahveninen J, Bonmassar G, Dale AM, Ilmoniemi RJ, Levanen S, Lin FH, May P, Melcher J, Stufflebeam S, Tiitinen H, Belliveau JW. Human posterior auditory cortex gates novel sounds to consciousness. Proc Natl Acad Sci USA. 2004;101(17):6809–6814. [PubMed]
Kwong KK, Belliveau JW, Chesler DA, Goldberg IE, Weisskoff RM, Poncelet BP, Kennedy DN, Hoppel BE, Cohen MS, Turner R. Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. Proc Natl Acad Sci USA. 1992;89:5675–5679. [PubMed]
Lai Y, van Drongelen W, Ding L, Hecox KE, Towle VL, Frim DM, He B. Estimation of in vivo human brain-to-skull conductivity ratio from simultaneous extra- and intra-cranial electrical potential recordings. Clin Neurophysiol. 2005;116:456–465. [PubMed]
Lawson CL, Hanson RJ. Solving least square problems. Englewood Cliffs; NJ Prentice Hall: 1974.
Lin FH, Witzel T, Hamalainen MS, Dale AM, Belliveau JW, Stufflebeam SM. Spectral spatiotemporal imaging of cortical oscillations and interactions in the human brain. NeuroImage. 2004;23(2):582–595. [PubMed]
Liu AK, Belliveau JW, Dale AM. Spatiotemporal imaging of human brain activity using functional MRI constrained magnetoencephalography data: Monte Carlo simulations. Proc Natl Acad Sci USA. 1998;95:8945–8950. [PubMed]
Liu AK, Dale AM, Belliveau JW. Monte Carlo simulation studies of EEG and MEG localization accuracy. Human Brain Mapp. 2002;16:47–62.
Mattout J, Phillips C, Penny WD, Rugg MD, Friston KJ. MEG source localization under multiple constraints: an extended Bayesian framework. NeuroImage. in press.
Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature. 2001;412:150–157. [PubMed]
Ogawa S, Tank DW, Menon R, Ellermann JM, Kim SG, Merkle H, Ugurbil K. Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. Proc Natl Acad Sci USA. 1992;89:5951–5955. [PubMed]
Oostendorp TF, Delbeke J, Stegeman DF. The conductivity ratio of the human skull: results of in vivo and in vitro measurement. IEEE Trans Biomed Eng. 2000;47:1487–1492. [PubMed]
Oster HS, Rudy Y. The use of temporal information in the regularization of the inverse problem of electrocardiography. IEEE Trans Biomed Eng. 1992;39:65–75. [PubMed]
Pascual-Marqui Review of methods for solving the EEG inverse problem. Int J Bioelectromagn. 1999;1(1):75–86.
Phillips C, Rugg MD, Friston KJ. Systematic regularization of linear inverse solutions of the EEG source localization problem. NeuroImage. 2002;17:287–301. [PubMed]
Phillips C, Mattout J, Rugg M, Maquet P, Friston KJ. An empirical Bayesian solution to the source reconstruction problem in EEG. NeuroImage. 2005;24:997–1011. [PubMed]
Rao BD, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Trans Signal Proc. 1999;47(1):187–200.
Puce A. Comparative assessment of sensorimotor function using functional magnetic resonance imaging and electrophysiological methods. J Clin Neurophysiol. 1995;12:450–459. [PubMed]
Sekihara K, Poeppel D, Marantz A, Koizumi H, Miyashita Y. Noise covariance incorporated MEG-MUSIC algorithm: a method for multiple-dipole estimation tolerant of the influence of background brain activity. IEEE Trans Biomed Eng. 1997;44:839–847. [PubMed]
Twomey S. On the numerical solution of Fredholm integral equations of the first kind by the inversion of the linear system produced by qudarature. J Assoc Comput Mach. 1963;10:97–101.
Wagner M, Fuchs M, Kastner J. Proc 12th Int Conf Biomag. 2000. fMRI-constrained dipole fits and current density reconstructions. CD-ROM.

Formats:

PubMed articles by these authors

PubMed related articles

Recent Activity

Links

Simple NCBI Directory

Getting Started

Resources

Popular

Featured

NCBI Information