• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jamiaJAMIA - The Journal of the American Medical Informatics AssociationInstructions for authorsCurrent TOC
J Am Med Inform Assoc. 2005 Sep-Oct; 12(5): 530–536.
PMCID: PMC1205601

Effect of an Internet-Based System for Doctor-Patient Communication on Health Care Spending


We studied the effect of a structured electronic communication service on health care spending, comparing doctor office and laboratory spending for a group of patients before and after the service became available to them relative to changes in a control group. In the treatment group, doctor office spending and laboratory spending fell in the period after the service became available, relative to the control group (p < 0.05). A rough estimate is that average doctor office spending per treatment group member per month fell $1.71 after availability of the service, and laboratory spending fell roughly $0.12. Spending associated with use of the electronic service was $0.29 per member per month. We conclude that use of structured electronic visits can reduce health care spending.

Use of the Internet for health care has generated considerable interest in recent years.1,2,3,4,5,6 While concerns continue to persist about issues like privacy and the quality of Internet-based information and communication,3,7,8,9,10,11,12,13,14,15 at their best, electronic technologies promise an efficient mechanism by which information can be disseminated, communication enhanced, and improvements in health care facilitated. One possibility is that harnessing the Internet to enhance communication between physicians and patients will lead to improved efficiency in medical care and reduced costs. Effective electronic communication between patients and physicians may even be able to reduce the need for office visits in some circumstances and generate spending reductions. On the other hand, expanded electronic communication may increase workloads for physicians without improving health or might even lead to reductions in care effectiveness. Understanding the effects of expanded communication is increasingly important as efforts to use more electronic communication, aided by policy changes like the addition of codes to allow compensation for electronic communication, expand.

This study evaluated the effects on health care spending of an Internet-based physician-patient communication system. We studied the impact of the service offered by RelayHealth, a health care firm in the San Francisco Bay area. They offer a service whereby patients can communicate with their physicians using free-text messaging as well as an Internet-based structured patient interview called a webVisit, which provides a structured set of diagnostically oriented questions that depend on patients' conditions and responses to previous questions. When both patients and their physicians are registered for the service, patients' free-text messages or webVisits are transmitted via the RelayHealth Web site to the physician for response or further action. The service contains security provisions to protect the privacy of the communication. RelayHealth also makes available at its Web site an appointment request service, a referral request service, a prescription renewal request service, a laboratory and test results service, and a mechanism for physicians to provide prescriptions (“e-scripts”). We focus our analysis on the free-text messaging and webVisit services.

In 2001–2002, with the intent of creating a setting that would facilitate a study of the effects of the service, RelayHealth in cooperation with Blue Shield of California conducted a pilot implementation, making the service available to more than 2,000 Blue Shield of California patients and their physicians. This paper analyzes this pilot implementation and its impact on health care spending. We use detailed claims data to track spending for key health care services likely to be affected by the availability of the Internet service, comparing spending in the time period before the service was available to spending in the time period after it was available. We incorporate spending data from a matched control group to account for contemporaneous trends in spending.

While this study focuses on one particular method of communication in one population, it provides useful first evidence about the impacts of using the Internet for doctor-patient communication on spending. While electronic communication between physicians and patients has attracted considerable attention for its benefits as well as its risks,16,17,18,19,20,21 we are aware of no systematic evidence of the impacts of electronic doctor-patient communication on health care use and spending.

Study Setting and Claims Data

Beginning in February 2001, all 1,742 physicians participating in Blue Shield of California Preferred Provider Organization (PPO) who had at least 75 PPO patients were invited to participate in the study. Of these, 176 (10.1%) agreed to participate. Those who agreed to participate received a one-time payment of $100 and thereafter had the ability to use the RelayHealth service. Physicians faced no charges for their use of the service and were eligible for $20 payments from Blue Shield of California for each webVisit they conducted. Nonclinical administrative use of the service was available but was not reimbursed.

Blue Shield of California patients were recruited for the study from practices of participating physicians beginning in March 2001. Recruitment was by mail, e-mail, or telephone contact or through direct communication with their doctor. A total of 2,357 patients agreed to participate and to have their health expenditures tracked as part of the study. The majority of the patients participating in the study were adults, although some were children whose parents could use the service for their health care. Although concentrated in metropolitan areas, individuals from all regions of California were included. Patients who enrolled could begin using the service in April 2001. Most patients could use the service free of charge, although a minority of physicians charged small copayments when the Blue Shield of California benefit packages included a copayment for office visits.

The study examined health care spending by analyzing Blue Shield of California health insurance coverage information as well as detailed and comprehensive claims data spanning the period June 1, 2000 through May 31, 2002 for the enrolled patients. Of the 2,357 patients, 369 could not be uniquely linked to Blue Shield of California records, leaving 1,988 patients for analysis. These 1,988 patients provide 42,452 patient-months of data during the June 1, 2000 through May 31, 2002 period. In addition to the insurance data, we obtained data on the use of webVisits, free text messages, and other services by enrolled patients from RelayHealth.

The 1,988 enrolled patients form the treatment group for our study. For comparison, we obtained data on a control group of patients. To be useful, the control group should have approximately the same characteristics as the treatment group. In this case, it is important that the control group not only match on demographic, geographic, health plan, and health care characteristics, but also that they have Internet access.

The control group was selected using a multistep process. First, patients enrolled in Blue Shield of California PPOs but not in the practice of a physician who had elected to use the RelayHealth service were identified. Second, the individuals were eliminated from the set of potential controls if they did not reside in the same zip codes as the treatment group. This ensured that the controls had approximately the same socioeconomic status, access to the same community and health care services, and the same environmental influences. Third, propensity score matching was used to select a subset of patients from this group that best matched the treatment group on demographic and health care use characteristics. To implement the propensity score matching approach, we estimated a logit model of the probability that an individual was in the treatment group as a function of age, gender, inpatient episodes, outpatient visits, and drug expenditures in the year prior to the intervention. The results from this model were then used to predict the propensity (probability) that an individual in the set of potential controls would have been a treatment group patient. Working with the set of potential controls and their propensity scores, we identified a set of controls with propensity scores that fell within the range of propensity scores in the treatment data.

In the final step, the households of members of the propensity score–matched group were surveyed by telephone and asked about Internet access. The response rate to this telephone survey was less than 20%. Patients in households that reported having access to the Internet were retained as controls for this study. Enrollment and claims data were obtained for this group from Blue Shield of California. In all, we have data on 3,007 control patients, providing 68,345 patient-months of data.

For the treatment and control groups, we used the claims data to compute three measures of spending for each patient on a monthly basis. We hypothesized that the RelayHealth service is most likely to influence use of outpatient care for evaluation and management of relatively routine conditions, so we focused on spending for services in doctor offices and laboratories. We measured total office spending by summing claims where codes for the place of service and the attending provider indicate services performed in a physician's office. This includes some laboratory services that are performed in the doctor's office but not those performed in an external laboratory. This measure includes claims for a number of services that we do not expect to be directly influenced by the RelayHealth service, so we also developed a narrower measure of doctor office spending (which we term “MD office spending”) that excludes spending on chiropractors, acupuncturists, physical therapists, and occupational therapists. Finally, we measured spending on laboratory services provided by laboratories external to physician offices.

For each measure for each patient, we used the claims data to compute spending by month for each month the person was covered by Blue Shield of California. We computed measures of spending based on the allowed amounts in the claims data. Since these data are the basis for payment, the allowed amounts are closely monitored and should nearly always accurately reflect the amount of spending Blue Shield of California will allow for a given service. In general, the allowed amounts reflect fee schedules and discounts that the plan may have in place with different providers, before considering patient cost sharing or other similar factors that would reduce the amount that the plan ultimately pays. Spending measures do not include the costs of RelayHealth use, which we accounted for separately.

Based on the enrollment date in the study, for each individual in the treatment group, we categorized the monthly spending data as being in either the “preservice” period, when the service was not available to them, or the “service” period, when the service was available to them. Enrollment dates vary across individuals, ranging from April 2001 to February 2002. The majority of the registrations occurred between April 2001 and July 2001. Individuals who enrolled in the middle of the month have a partial month in the preservice period and a partial month in the service period, weighted appropriately in the analysis. Of the 42,452 patient months in the treatment group, 23,691.9 are in the preservice period, and 18,760.1 are in the service period.

Examination of the distributions of the monthly spending data revealed that the distributions were very skewed, with a small number of observations at very high spending levels. Because these observations seem largely driven by health care demands and spending patterns for a small number of patients that are quite different from those of the vast majority of patients in the data set, we excluded from analyses for a given measure any individuals who had any monthly spending observation that was more than four standard deviations above the mean for the measure. This excluded 11.5% of the patient months for the office spending measures and 9.6% of the patient-months for the laboratory spending measure. We examined the robustness of our results to including all observations and found generally consistent results, although somewhat smaller and with less statistical significance (as might be expected given the increase in variance associated with including the high outliers). For example, including all observations in our models of office spending would reduce the predicted savings amounts by 45%. In some ways, this is not surprising since patients with very high spending are unlikely to be substantially affected by the RelayHealth service, and the inclusion of their very high spending data has an important impact on the regression estimates.

Statistical Modeling

Our objective was to study spending changes in the treatment group associated with the availability of the service, relative to contemporaneous trends in the control group. We approached this as an intent-to-treat analysis, comparing spending in the groups that did and did not have access to the service rather than attempting to study the subset of the treatment group who did in fact use the service.

We began with simple comparisons of spending in the treatment group before and after the service became available. These comparisons are instructive but are not adjusted for trends in spending, do not control for any characteristics of the individuals, and are disproportionately influenced by observations at the high end of the distribution even after excluding individuals with observations more than four standard deviations above the mean.

To incorporate the control group data, account for individual characteristics, and better manage the skewed distributions, we adopted a regression modeling strategy commonly used in health economic analyses of spending data.22 This model, referred to as a “two-part model,” is based on two regressions: the first examining whether a given individual had any spending and the second part predicting the logarithm of spending among those who had any spending. The first part of the model is expressly aimed at accounting for the analytic problem posed by the large number of monthly spending observations with zero spending.

Specifically, in the first part, we estimated a probit model (a model appropriate for dichotomous dependent variables, closely related to a logit model) in which the dependent variable is a 0–1 indicator for whether a given patient-month has any spending. We estimated this model at the level of the patient-month. In addition to an indicator variable distinguishing members of the treatment group during the service period, we included the following control variables: gender, age (six groups: < 20, 20–29, 30–39, 40–49, 50–59, 60+), three-digit zip code, a control for membership in the treatment group, and indicator variables for each month between June 2000 and May 2002. Although the control group was matched to the treatment group, as we show below, the matching was not perfect, so including the controls in the modeling helped to account for any differences in observable characteristics between the treatment and control groups. The indicator for membership in the treatment group accounted for any baseline differences in spending between the two groups.

Essentially, this method allows the control group observations to trace out a time path in the spending measures and allows us to learn whether the probability of spending in a treatment group patient-month during the service period is different from that in the preservice period, relative to contemporaneous changes for the control group. We expected the results from part 1 of the model to be of particular interest since they provide a mechanism for examining the hypothesis that use of the RelayHealth service reduces the number of encounters. Although we do not have a direct measure of encounters available in these data, if the number of encounters is reduced, the probability of having no spending in any given month should increase with the availability of the Internet service.

For presentation, we used the part 1 results to compute the predicted probability that any given patient-month in the treatment group has nonzero spending during the preservice and service periods, holding the controls for age, gender, and geographic area fixed at their sample means.

In the second part of the model, we used ordinary least squares regression to estimate a model in which the dependent variable is the log of spending for patient-months in which there is any spending. We included the same control variables as in the first part. Here, in a way analogous to the first part of the model, the results indicate whether there are changes in the amount of spending given any spending between the preservice and service periods for the treatment group, relative to simultaneous changes for the control group. For presentation, we used the results to compute predicted mean spending for the treatment group in the preservice and service periods, holding the control variables fixed at their sample means. Since this regression was estimated using the log of spending as the dependent variable, predicted values are average log spending values. We derived a predicted value for the mean of the level of spending from the predicted value for mean log spending using Duan's smearing estimator, a nonparametric approach to this transformation, which is more accurate than other parametric approaches if the distribution of spending is not precisely log normal.23

Traditionally computed regression standard errors rely on the assumption that all observations are independent. Since our data contained multiple monthly observations for each individual, there may be some correlation across observations from the same people, violating the independence assumption. To account for this, we computed robust standard errors that explicitly allow for the potential for this correlation. We used these standard errors in our tests for statistical significance. We interpreted p-values of 0.05 or less as statistically significant.


Sample Characteristics

[triangle] summarizes demographic and baseline spending data for the treatment and control groups for each of the measures (the samples for each measure vary slightly because we excluded the set of high outliers specific to each measure). The treatment and control groups appear to be relatively well, but not perfectly, matched on key characteristics. The treatment groups have a slightly higher share of males and are slightly older. While the differences in the age and gender distributions are statistically significant, the sizes of the differences are not large. We also compared average per-member per-month spending amounts in the period before April 1, 2001 (the first possible date that study patients could use the service) for our spending measures. There is some evidence that the treatment group has lower average spending than the control group on physician office spending measures. We included control variables in our analyses below to account for these observable differences in characteristics and baseline spending.

Table 1.
Characteristics of Treatment and Control Groups

Use of the service among treatment group patients, who were all eligible to use the service, was relatively limited. RelayHealth personnel reviewed patterns of use by the treatment group patients and reported that there were 215 reimbursable physician-patient interactions during the service period.

[triangle] summarizes the distribution of the per-member per-month spending levels for our three spending measures, combining the treatment and control groups, emphasizing that all the spending measures are skewed, even after elimination of the highest outliers.

Table 2.
Summary Measures for the Per-Member Per-Month Spending Variables

Average Spending Comparisons

A straightforward way to look for differences associated with availability of the service is to examine mean spending in the treatment group before and after the service was available. [triangle] reports mean per-member per-month spending for the treatment group in the preservice and service periods for the various spending measures. There are sizable declines in mean spending between the preservice and service periods in the treatment group.

Table 3.
Differences in Mean Spending for the Treatment Group

Modeling Spending Changes

To incorporate the control group data, account for baseline differences between the treatment and control groups, and more explicitly incorporate the skewed distributions into our approach, we used a two-part spending model. [triangle] presents the key results from the first part of the two-part model (more complete results can be found in Appendix 1). Here, we estimate a model of the probability that a patient-month had any spending. For all office spending, the coefficient from the probit regression model is negative, suggesting that the probability of having any spending is lower for the treatment group in the service period relative to the preservice period, but the coefficient is not statistically significant. The rightmost two columns of [triangle] show the predicted probabilities of having any spending in the preservice and service periods, derived from the regression results. For the more focused measures of MD office and laboratory spending, the results show a statistically significant reduction in the probability of having any spending. In part 2 of the model, which models the log of spending given any spending, we found no statistically significant relationships between the availability of the service and spending amounts ([triangle] and more complete results in Appendix 1).

Table 4.
Results From Part 1 Models for the Probability of Any Spending
Table 5.
Results From Part 2 Models of Log (Spending) for Patient-Months with Any Spending

If the probability of having any spending in a given month is reduced, average spending per member per month will also be reduced. Since we observed no statistically significant change in the level of spending given any spending, the expected amount of net spending change is the reduction in the probability of having any spending multiplied by the amount of spending expected if there is any spending. Using the sample average levels of the amount of spending, given any spending, we could derive a rough estimate of the net savings associated with the availability of service.

For the narrower MD office spending measure, we observed a statistically significant reduction in the probability of any spending from 31.7% to 30.5%. Using the mean value of $142.43 for spending in this category, among those who had any spending, this would imply a reduction of per-member per-month spending of $1.71. Using the broader office spending measure produced a similar estimated saving, but the results were not statistically significant. For laboratory spending, with a mean of $39.28 among those who had spending, the estimate of net per member per month spending reduction is $0.12.

Offsetting RelayHealth Costs

The estimates shown here do not include spending on RelayHealth consultations, which must be considered to get a complete picture of the cost effects of the service. In practice, the exact cost of the service would depend on the amount that providers are paid for Web-based consultations, which would likely vary from one situation to the next. In this case, we can estimate the costs using the amounts paid in the pilot implementation that we study here. Among the patients studied here, between June 2001 and May 31, 2002, there were 215 qualifying interactions. In this implementation, these were paid at $25 per visit, $20 for the provider performing the consultation and $5 for RelayHealth. At $25 per visit, 215 interactions would have cost $5,375. There are 18,760.1 patient-months in the study period in the treatment group. An estimate of the per-member per-month spending on webVisits is thus $5,375 divided by 18,760.1, which equals $0.29.


This study has some limitations. Because the selection of the control group required not only matching of demographic and health system characteristics but also completion of a survey to assess Internet access, the final control group differs from the treatment group in some ways. These differences necessarily raise some questions about the comparability of the two groups, although many of these differences, while statistically significant, are small.

There are also issues of self-selection. The treatment group comprised patients from the practices of physicians who elected to join the study. The patients themselves also had to elect to join the study. Thus, the experience of the treatment group here is best regarded as reflecting the experience of a group of doctors and patients who have some interest in this type of service. On the one hand, this group of doctors and patients may be useful to study in that they may resemble those most likely to take up future opportunities for electronic communication. On the other hand, this sample is not representative of the population more generally, so these results may not reflect what would happen if the service were made available to less interested patients and physicians. It seems plausible that if the service were made available to a less motivated population, it would produce smaller savings and would not be likely to result in increased spending. We do not, however, have the data to test this directly.

Since we studied insurance claims, these results should be interpreted as reflecting the impacts of the service on payments by the insurance company. We did not track impacts on other spending, like out-of-pocket spending by patients either in the form of copayments and deductibles, or spending for other services not covered by insurance.

The goal of this study was to assess implications for health care spending and not to study health outcomes. The claims data that we studied are not well equipped to evaluate health outcomes. A full evaluation of the effects of expanded electronic communication will require careful assessment of health care delivery and outcomes.

The financial specifics of this study may also be important. Physicians were paid $20 per interaction, and most patients could use the service for free. With these parameters, we observed savings, but if the specific circumstances were to change, it is not clear whether the same savings would be observed. For example, consider the hypothetical case of increasing the per-use payment to physicians to more than $20. We do not have the data to estimate the impact empirically, but one possibility is that a higher payment could give physicians some additional interest in the service, which could promote more use of the service. The cost of running the service would be increased, both because more would be paid per use and because there would be more use, but the higher cost could be offset if expanded use also led to further reductions in spending on office visits or other medical care by patients. The ultimate impact on spending in this scenario would depend on whether the reductions in medical care costs would be large enough to offset the increased payments to doctors. Another possibility is that higher payments to doctors would not change the amount of system use, in which case the predicted savings would be clearly lower since there would be more payments to doctors but not a chance for offsetting reductions in medical care use.

Another important consideration is the free use by patients. If patients were required to pay for the service, we speculate that they would be less likely to use it, which would, on the one hand, limit the impact of the service on their health care spending and, on the other hand, increase revenues to the health plan that could offset some of the cost of running the service and paying doctors. The net result would depend on which effect is larger.

Finally, the issue of conflict of interest should be noted. The initial work that underlies this analysis, including the data collection and development of the main features of the analytic approach, was conducted while some of the authors were retained by RelayHealth to examine the impact of its Web service on health spending. While the specific analyses presented in this paper were conducted after those relationships ended, some of the analyses here are similar to work performed while they were in place. While we do not believe these relationships impair the validity of the results, readers should be aware of these circumstances.


This study examined the effects of making available a Web-based consultation service to a large group of patients. This is the first study that we are aware of that examines the impacts of Internet-based health care provision on spending in a large-scale setting. An important strength of the study is that it incorporates strong and comprehensive claims data from a large set of patients and includes a control group matched to the treatment group to account for spending pattern trends.

Mounting studies of the impact of electronic physician-patient communication is costly and complex. Thus, despite the fact that this study does have some limitations, we believe it to be a useful opportunity to gain valuable information about the impacts of Web-based communication on spending. The results clearly suggest the potential for savings associated with Internet communication. In particular, we find evidence that the probability of having any spending on physician office services was reduced with the availability of the service. Our rough estimate of the reduction in office visit spending associated with availability of the service is $1.71 per member per month, while the cost of the service was approximately $0.29 per member per month. We lack the data to say exactly how this came about—the claims data do not allow us to clearly identify encounters, for example—but one plausible possibility is that use of the service reduced the number of visits and thus increased the probability of having no visits in any given month.

We also found evidence that laboratory spending is also reduced, again because the probability of having any laboratory spending falls with the introduction of the service. Our rough estimate of the reduction in spending on laboratory services is $0.12 per member per month, which should be evaluated in addition to reductions for physician office visits. If office visits generate laboratory tests, a plausible explanation for this result is that the service reduces the number of office visits, which in turn reduces the number of laboratory tests ordered. Taking these estimates of the reduction in MD office claims and the reduction in laboratory spending together would yield an estimate of a reduction of $1.83 per member per month for these two categories. Netting out the $0.29 per-member per-month cost of providing the service leaves $1.54. While in absolute magnitude this may seem a relatively limited reduction, we note that this is approximately 3.3% of the total average MD office spending and laboratory spending observed in our sample, and many other health plan activities produce smaller percentage reductions in per-member per-month spending and yet are highly valued by health plans.

Readers should be aware that there is some uncertainty in the estimated net spending reductions. That the results from the part 1 models are statistically significant for MD office spending and laboratory spending provides a measure of security for the finding of spending reductions and suggests that a 95% confidence interval for projections of the dollar amount of savings would not include zero. However, the derivation of net spending reduction figures involves combining multiple estimated effects, each of which is subject to some uncertainty, so that the potential for variation in the final result may be important. It is difficult to precisely compute a standard error for the composite net estimates, but it need not be implausible to consider variations of 50% or more.

The Internet clearly has considerable promise to improve health care delivery and possibly patient health as well. Continued attention to developing Internet resources for information dissemination and for communication could provide important benefits for patients and doctors.

Appendix 1. Detailed Results From Part 1 and Part 2 Spending Models

All Office Spending
MD Office Spending
Lab Spending
Part 1 (probit) (1)Part 2 (OLS) (2)Part 1 (probit) (3)Part 2 (OLS) (4)Part 1 (probit) (5)Part 2 (OLS) (6)
Service available−0.030 (0.016)−0.002 (0.018)−0.032 (0.016)*−0.025 (0.018)−0.092 (0.040)*−0.051 (0.068)
Treatment group0.010 (0.011)−0.088 (0.012)−0.025 (0.011)*−0.103 (0.012)0.020 (0.028)0.068 (0.048)
Male−0.216 (0.009)−0.050 (0.010)−0.204 (0.009)−0.042 (0.010)−0.268 (0.023)0.079 (0.042)
    20–29 y0.045 (0.020)*0.087 (0.022)0.027 (0.020)0.049 (0.022)*0.689 (0.049)0.165 (0.096)
    30–39 y0.113 (0.015)0.100 (0.016)0.074 (0.015)0.075 (0.016)0.557 (0.044)0.204 (0.089)*
    40–49 y0.136 (0.012)0.161 (0.013)0.114 (0.013)0.139 (0.013)0.634 (0.039)0.301 (0.080)
    50–59 y0.252 (0.013)0.172 (0.013)0.225 (0.013)0.159 (0.013)0.662 (0.039)0.315 (0.079)
    60 y or over0.405 (0.016)−0.175 (0.020)0.395 (0.017)−0.185 (0.020)0.683 (0.046)0.384 (0.088)
June 20000.080 (0.031)−0.080 (0.035)*0.084 (0.031)−0.078 (0.035)*−0.053 (0.074)−0.152 (0.136)
July 20000.034 (0.031)−0.085 (0.036)*0.033 (0.031)−0.094 (0.036)−0.098 (0.075)−0.053 (0.131)
August 20000.120 (0.031)−0.017 (0.034)0.127 (0.031)−0.008 (0.034)−0.143 (0.076)−0.150 (0.130)
September 20000.104 (0.031)−0.076 (0.034)*0.116 (0.031)−0.068 (0.034)*−0.125 (0.075)−0.089 (0.123)
October 20000.153 (0.030)−0.084 (0.035)*0.160 (0.030)−0.091 (0.035)−0.075 (0.073)0.150 (0.120)
November 20000.176 (0.030)−0.088 (0.034)*0.179 (0.030)−0.105 (0.034)−0.075 (0.073)−0.100 (0.116)
December 20000.093 (0.030)−0.127 (0.035)0.107 (0.030)−0.140 (0.034)−0.112 (0.074)−0.058 (0.114)
January 20010.223 (0.030)−0.048 (0.032)0.228 (0.030)−0.055 (0.032)−0.172 (0.075)*0.022 (0.126)
February 20010.139 (0.030)−0.069 (0.033)*0.148 (0.030)−0.060 (0.032)−0.084 (0.073)−0.001 (0.125)
March 20010.225 (0.030)−0.051 (0.033)0.229 (0.030)−0.058 (0.032)−0.066 (0.072)0.055 (0.110)
April 20010.025 (0.029)−0.075 (0.034)*0.038 (0.030)−0.078 (0.033)*−0.158 (0.074)*−0.069 (0.127)
May 20010.034 (0.029)−0.091 (0.033)0.044 (0.029)−0.093 (0.033)−0.067 (0.071)0.104 (0.108)
June 20010.005 (0.029)−0.077 (0.033)*0.023 (0.029)−0.078 (0.033)*−0.195 (0.075)−0.134 (0.144)
July 2001−0.007 (0.029)−0.061 (0.034)−0.001 (0.030)−0.062 (0.034)−0.052 (0.072)−0.035 (0.123)
August 20010.028 (0.029)−0.039 (0.034)0.026 (0.030)−0.046 (0.033)0.057 (0.069)−0.116 (0.109)
September 2001−0.041 (0.030)−0.039 (0.034)−0.041 (0.030)−0.042 (0.034)−0.006 (0.071)−0.022 (0.112)
October 20010.057 (0.029)−0.042 (0.034)0.064 (0.029)*−0.038 (0.034)0.061 (0.069)0.146 (0.108)
November 20010.042 (0.030)−0.116 (0.035)0.048 (0.030)−0.119 (0.035)0.059 (0.069)0.054 (0.109)
December 2001−0.012 (0.030)−0.099 (0.035)−0.009 (0.030)−0.103 (0.035)−0.042 (0.072)0.099 (0.114)
January 20020.059 (0.030)*0.039 (0.033)0.066 (0.030)*0.039 (0.033)0.098 (0.068)−0.099 (0.113)
February 20020.047 (0.030)−0.028 (0.033)0.061 (0.030)*−0.030 (0.033)0.036 (0.071)−0.011 (0.105)
March 20020.048 (0.030)−0.029 (0.034)0.057 (0.030)−0.033 (0.034)0.061 (0.070)0.048 (0.110)
April 20020.047 (0.030)−0.015 (0.035)0.059 (0.030)−0.003 (0.034)0.049 (0.071)0.013 (0.116)
Constant−0.669 (0.039)4.513 (0.048)−0.676 (0.039)4.529 (0.048)−2.556 (0.093)2.804 (0.149)

Note: Robust standard errors in parentheses.

*Significant at 5%.
Significant at 1%. Models also contain 36 dummy variables for 3-digit zip codes. Models are weighted to allow for partial preservice and service period months for treatment group patients who enrolled in the study mid-month.


The initial research underlying this manuscript was supported by the RelayHealth Corporation. The authors are grateful for support from Blue Shield of California in obtaining the data necessary for this study. They also acknowledge research support from Tiago Ribeiro of UC Berkeley and from Bruce Deal and Armando Levy of Analysis Group, Inc.


1. Kassirer JP. The next transformation in the delivery of health care. N Engl J Med. 1995;332:52–4. [PubMed]
2. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: caveat lector et viewor—let the reader and viewer beware. JAMA. 1997;277:1244–5. [PubMed]
3. Purcell GP, Wilson P, Delamothe T. The quality of health information on the Internet. BMJ. 2002;324:557–8. [PMC free article] [PubMed]
4. Blumenthal D. Doctors in a wired world: can professionalism survive connectivity? Milbank Q. 2002;80:525–46. [PMC free article] [PubMed]
5. Gingrich N, Magaziner I. Two old hands and the new new thing: interview by Rob Cunningham. Health Aff. 2000;19:33–40. [PubMed]
6. Council NR. Networking health: prescriptions for the Internet. Washington, DC: National Academy Press, 2000.
7. Kleinke JD. Vaporware.com: the failed promise of the health care Internet. Health Aff. 2000;19:57–71. [PubMed]
8. Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ. 1997;314:1875–9. [PMC free article] [PubMed]
9. Culver JD, Gerr F, Frumkin H. Medical information on the Internet: a study of an electronic bulletin board. J Gen Intern Med. 1997;12:466–70. [PMC free article] [PubMed]
10. Pealer LN, Dorman SM. Evaluating health-related Web sites. J School Health. 1997;67:232–5. [PubMed]
11. Wyatt JC. Commentary: measuring quality and impact of the World Wide Web. BMJ. 1997;314:1879–81. [PMC free article] [PubMed]
12. Griffiths KM, Christiansen H. Quality of Web based information on treatment of depression: cross sectional survey. BMJ. 2000;321:1511–5. [PMC free article] [PubMed]
13. Meric F, Bernstam EV, Mirza NQ, Hunt KK, Ames FC, Ross MI, et al. Breast cancer on the world wide Web: cross sectional survey of quality of information and popularity of Websites. BMJ. 2002;324:577–81. [PMC free article] [PubMed]
14. Pennbridge J, Moya R, Rodrigues L. Questionnaire survey of California consumers' use and rating of sources of health care information including the Internet. West J Med. 1999;171:302–5. [PMC free article] [PubMed]
15. Fox S, Rainie L, Horrigan J, et al. The online health care revolution: how the Web helps Americans take better care of themselves. Washington, DC: Pew Internet and American Life Project; 2000.
16. Borowitz SM, Wyatt JC. The origin, content, and workload of e-mail consultations. JAMA. 1998;280:1321–4. [PubMed]
17. Eysenbach G, Diepgen TL. Responses to unsolicited patient email requests for medical advice on the world wide Web. JAMA. 1998;280:1333–5. [PubMed]
18. Spielberg AR. On call and online: sociohistorical, legal, and ethical implications of e-mail for the patient-physician relationship. JAMA. 1998;280:1353–9. [PubMed]
19. Couchman GR, Forjuoh SN, Rascoe TG. E-mail communications in family practice: what do patients expect? J Fam Pract. 2001;50:414–8. [PubMed]
20. Mandl KD, Feit S, Pena BM, Kohane IS. Growth and determinants of access in patient e-mail and Internet use. Arch Pediatr Adolesc Med. 2000;154:508–11. [PubMed]
21. Moyer C, Stern D, Dobias K. Bridging the electronic divide: patient and provider perspectives on e-mail communication in primary care. Am J Managed Care. 2002;8:427–33. [PubMed]
22. Duan N, Manning WG, Morris CN, Newhouse JP. A comparison of alternative models for the demand for medical care. J Business Econ Stat. 1983;1:115–26.
23. Duan N. Smearing estimate: a nonparametric retransformation method. J Am Stat Assoc. 1983;78:605–10.

Articles from Journal of the American Medical Informatics Association : JAMIA are provided here courtesy of American Medical Informatics Association
PubReader format: click here to try


Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


  • PubMed
    PubMed citations for these articles