NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Gibbons MC, Wilson RF, Samal L, et al. Impact of Consumer Health Informatics Applications. Rockville (MD): Agency for Healthcare Research and Quality (US); 2009 Oct. (Evidence Reports/Technology Assessments, No. 188.)

Cover of Impact of Consumer Health Informatics Applications

Impact of Consumer Health Informatics Applications.

Show details

Executive Summary

Many people are excited about the potential to improve the health of the public by using health information technology (health IT) and eHealth solutions that are tailored to consumers. Despite growing interest in this field referred to as consumer health informatics (CHI), the value of CHI applications has not been rigorously reviewed. The objectives of this report were to review the literature on the evidence of the influence of currently developed CHI applications on health and health care process outcomes, to identify the gaps in the CHI literature, and to make recommendations for future CHI research. For the purposes of this review, CHI is defined as any electronic tool, technology, or electronic application that is designed to interact directly with consumers, with or without the presence of a health care professional that provides or uses individualized (personal) information and provides the consumer with individualized assistance, to help the patient better manage their health or health care.

The specific Key Questions were:

  1. What evidence exists that CHI applications impact:
    1. Health care process outcomes (e.g., receiving appropriate treatment) among users?
    2. Intermediate health outcomes (e.g., self-management, health knowledge, and health behaviors) among users?
    3. Relationship-centered outcomes (e.g., shared decisionmaking or clinician-patient communication) among users?
    4. Clinical outcomes (including quality of life) among users?
    5. Economic outcomes (e.g., cost and access to care) among users?
  2. What are the barriers that clinicians, developers, consumers, and their families or caregivers encounter that limit utilization or implementation of CHI applications?
  3. What knowledge or evidence exists to support estimates of cost, benefit, and net value with regard to CHI applications?
  4. What critical information regarding the impact of CHI applications is needed to give consumers, their families, clinicians, and developers a clear understanding of the value proposition particular to them?

The best evidence available to answer Key Question 1 is found in randomized controlled trials (RCTs). However, RCTs are not the best study design for addressing Key Question 2, so for this question we included articles on any study that was designed to look at barriers to use of CHI, including but not limited to the RCTs that addressed Key Question 1. Key Question 3 addressed knowledge and evidence deficits regarding needed information to support the estimation of costs, benefits, and value regarding CHI applications. Key Question 4 addresses critical information regarding the effect of CHI applications needed to give consumers, their families, clinicians, and developers a clear understanding of the value of CHI applications.

To identify articles that addressed Key Question 1, we searched computerized literature databases using terms relevant to our definition of CHI applications, combined with terms relevant to our definition of “consumer,” combined with terms identifying RCTs as the study design of interest. To search for articles that were relevant to Key Question 2, we used terms relevant to our definition of CHI applications, combined with terms relevant to barriers; the search was not limited by study design. Our comprehensive search included electronic searching of MEDLINE®, EMBASE®, The Cochrane Library, ScopusTM, and CINAHL® databases. We also looked for eligible studies by reviewing the references in pertinent reviews, by querying our experts, and by searching grey literature sources such as conference proceedings.

Studies were eligible for inclusion in the review if they applied to Key Question 1 or 2 and did not have one of the following reasons for exclusion: no health informatics application, health informatics application does not apply to the consumer, health informatics applications is for general information only (e.g., general Web site) and is not tailored to individual consumers, study of a “point of care” device (defined as requiring a clinician to use or obtain and is part of the regular provision of care), or no original data.

We assessed the eligible studies on the basis of the quality of their reporting of relevant data. For the RCTs, we used the study quality scoring system developed by Jadad et al. For the other studies, we used a form to identify key elements that should be reported when reporting results. The quality assessments were done independently by paired reviewers.

We then created a set of detailed evidence tables containing information extracted from the eligible studies. We stratified the tables according to the applicable Key Question and subquestion (for Key Question 1). We did not quantitatively pool the data for any of the outcomes because of the marked heterogeneity of target conditions of interest and the wide variety of outcomes studied.

Data were abstracted by one investigator and entered into online data abstraction forms using SRS (Mobius Analytics, Inc., Ottawa, Ontario, CA) Second reviewers were generally more experienced members of the research team, and one of their main priorities was to check the quality and consistency of the first reviewers’ answers.

At the completion of our review, we graded the quantity, quality, and consistency of the best available evidence for each type of outcome in each clinical area, using an evidence grading scheme recommended by the GRADE Working Group and modified for use by the Evidence-based Practice Centers (EPC) Program. For each outcome of interest, two investigators independently assigned a grade, and then the entire team discussed their recommendations and reached a consensus.

Throughout the project, the core team sought feedback from external experts with expertise in systematic reviews, CHI, consumer advocacy, decision aids, and ethics. A draft of the report was sent to the external experts. The EPC team addressed the comments of the external experts before submitting the final version of the evidence report.

Results

Our literature search identified 146 articles that were eligible for inclusion in this report: 121 for Key Question 1 and 31 for Key Question 2; 6 articles were eligible for both Key Question 1 and Key Question 2. All of the Key Question 1 eligible studies were RCTs. The 31 articles addressing barriers to use of CHI applications fell under a variety of study designs and data collection types. Data on barriers was collected mostly in non-validated surveys and qualitative studies from trial data.

In terms of types of applications studied, 55 percent of studies evaluated interactive Web-site–based applications or Web-based tailored educational Web sites. Another 15 percent of studies evaluated computer-generated tailored feedback applications. Interactive computer programs and personal monitoring devices were evaluated in approximately 8 percent of studies each. Finally, health risk assessments, decision aids, cell phones, laptops, CD ROMs, personal digital assistants (PDA/smartphones), short message system texting (SMS/text), discussion/chat groups and computer-assisted imagery were evaluated in less than 5 percent of studies each. In terms of participant age groups, 77 percent (76/99) of studies reporting age of participants targeted adult CHI users. Approximately 12 percent of studies targeted adolescents/teens, 3 percent of studies targeted seniors and another 3 percent of studies targeted children. Five percent of studies targeted participants from overlapping age groups. In terms of intervention delivery setting or location, 58 percent of studies reporting delivery location evaluated CHI applications that were used in the home or residence. A minority of evaluations were completed in schools (15 percent), clinical settings (17 percent), communities (3 percent), online (5 percent) or kiosks (2 percent). Finally, of studies reporting the race of the participants 92 percent (49/53) of the studies employed populations that were greater than 50 percent white/Caucasian. There was only one study with greater than 50 percent African-American participants and no studies with a majority of participants who were Hispanic, American Indian/Alaska Native, or Asian/Pacific Islander.

Key Question 1: What is the evidence of impact of CHI applications on health outcomes?

First, we sought to understand the impact of CHI applications on health care process outcomes (Key Question 1a). There were only five studies that met the inclusion-exclusion criteria and thus were available to shed light on this question. Five of these studies focused on asthma and one additional study focused on contraceptive medication utilization. All of the asthma studies showed a significant positive effect of the CHI application on at least one health care process measure. The oral contraceptive medication use application failed to reduce contraceptive discontinuation. No study found any evidence of harm.

This review identified 108 studies that addressed the influence of CHI applications on intermediate health outcomes (Key Question 1b). These 108 studies evaluated the effects of CHI applications on intermediate outcomes in the context of nine categories of diseases or health conditions. Intermediate outcomes were evaluated related to breast cancer in three studies, diet, exercise, physical activity, not obesity in 32 studies, alcohol abuse in seven studies, smoking cessation in 19 studies, and obesity in 11 studies, diabetes mellitus (or diabetes with associated conditions) in seven studies, mental health in eight studies, asthma/chronic obstructive pulmonary disease (COPD) in four studies, and miscellaneous health conditions in another 15 studies.

With regard to breast cancer, evaluated intermediate outcomes included social support, information competence, level of conflict, and satisfaction. All three studies reported significant positive effect on at least one intermediate health outcome. No study found any evidence of harm.

In terms of diet, exercise, physical activity, not obesity, evaluated intermediate outcomes included self-management, knowledge, program adherence, and change in health behaviors. Eighty-nine percent of these studies demonstrated significant positive effect on at least one intermediate health outcome related to diet, exercise, and physical activity. No study found any evidence of harm.

Evaluated intermediate outcomes related to alcohol abuse included self-management, knowledge attainment, and change in health behaviors. All studies found significant positive effect on at least one intermediate outcome related to alcohol abuse. No study found any evidence of harm.

With regard to smoking cessation, intermediate outcomes assed in these smoking cessation CHI trials included self-management, knowledge attainment, and change in health behaviors. Fifty-seven percent of these studies demonstrated a positive effect on at least one intermediate outcome related to smoking cessation. No study found any evidence of harm.

Evaluated intermediate outcomes of interest related to obesity included weight loss behaviors and body composition. Only 36 percent of studies demonstrated positive effect on intermediate outcomes related to obesity. No study found any evidence of harm.

Seven studies were identified to evaluate the influence of CHI on intermediate outcomes related to diabetes mellitus. Intermediate outcomes of interest included perceived self-efficacy, satisfaction, and readiness to change, perceived competence, exercise minutes per day, and self-reported global health. All seven studies found evidence of effect of CHI applications on one or more intermediate outcomes related to diabetes mellitus. No study found any evidence of harm.

Eight studies were identified to evaluate the effect of CHI applications on intermediate outcomes related to mental health issues. Intermediate outcomes of interest included work and social adjustment, perceived stress, self-rated self-management, sleep quality, mental energy, and concentration. Seven of the eight studies found evidence of positive effect of CHI applications on at one or more intermediate outcomes related to mental health. No study found any evidence of harm.

Four studies were identified to evaluate the effect of CHI applications on intermediate outcomes related to asthma/COPD. Intermediate outcomes of interest included adherence, knowledge, change in behavior, dyspnea knowledge, and self-efficacy. Only one of the four studies demonstrated a significant effect on any intermediate outcome related to asthma/COPD. No study found any evidence of harm.

Two studies were identified to evaluate the effect of CHI applications on intermediate outcomes related to menopause or hormone replacement therapy (HRT). Only one study found evidence of significant effect on an intermediate outcome related to menopause/HRT utilization.

Finally, an additional 15 studies were identified to evaluate the influence of intermediate health outcomes in other clinical areas. These intermediate outcomes were in health areas related to arthritis, back pain, behavioral risk factor control, contraception, cardiovascular disease, cancer, caregiver decisionmaking, fall prevention, health behavior change, headache, HIV/AIDS, and adolescent risk behaviors. Each of these studies found evidence of significant effect of the CHI application on intermediate outcomes related to the health condition under study. No study found evidence of harm.

Another subquestion of this key question this review sought to answer was regarding the effect of CHI applications on relationship centered outcomes (Key Question 1c). Eight studies were identified that met the inclusion-exclusion criteria. Relationship centered outcomes of interest included social support, quality of life, decisionmaking skill, social support, positive interaction with the provider, and satisfaction with care. These relationship centered outcomes were evaluated in the context of HIV/AIDS, cancer, osteoarthritis, and pregnancy. Just over 60 percent (5/8) of studies demonstrated significant effect of CHI on at least one aspect of relationship centered care. No study found any evidence of harm.

Twenty-eight studies addressed the question about the impact of CHI applications on clinical outcomes (Key Question 1d). Clinical outcomes evaluated in the identified studies included disease-specific outcomes in the context of cancer (three studies), diabetes mellitus (three studies), mental health (seven studies), diet, exercise, or physical activity (five studies), and Alzheimer’s disease, arthritis, asthma, back pain, aphasia, COPD, HIV/AIDS, headache, obesity, and pain (one study each). Over 80 percent of studies found significant influence of CHI applications on at least one clinical outcome. Three studies evaluated the effect of CHI applications on breast cancer clinical outcomes, but only one found any evidence of significant CHI impact. Of the five studies that evaluated the effect of CHI applications on clinical outcomes related to diet, exercise or physical activity, four studies found a significant positive effect on one or more clinical outcomes. Among the seven studies that evaluated the effect of CHI applications on mental health clinical outcomes, all seven found evidence of significant effect of CHI on one or more clinical outcomes. Three studies evaluated the effect of CHI applications on diabetes mellitus clinical outcomes. All three studies found evidence of significant effect of CHI on at least one clinical outcome. The remaining nine studies evaluated a CHI application in different health areas including Alzheimer’s disease, arthritis, asthma, back pain, aphasia, COPD, headache, HIV/AIDS, and general pain. With the exception of the general pain study, the eight remaining studies all found evidence of significant effect of CHI on one or more clinical outcomes. None of these 27 studies found any evidence of harm attributable to a CHI application.

The fifth subquestion of this key question was about the evidence of impact of CHI applications on economic outcomes (Key Question 1e). Three studies addressed this question. Economic outcomes evaluated in these studies included cost of program delivery, cost of computer information system with manual data extraction versus cost of the computer system with use of the electronic patient record, materials costs, total costs, and incremental cost-effectiveness. These outcomes were evaluated in the context of asthma, cancer, and obesity. Each of these studies used different economic metrics and methodologies. One study failed to provide any cost estimates for the control group. One study was done in an adult population, another in a pediatric population, and the third study did not provide any details regarding the age of study participants. Given the very small number of studies and the significant limitations and heterogeneity of these studies, no conclusions regarding the economic impact of CHI applications can be made.

Key Question 2: What are the barriers that clinicians, developers, consumers, and their families or caregivers encounter that limit utilization or implementation of CHI applications?

Thirty-one studies addressed the barriers to CHI applications. Studies focused on a wide variety of clinical conditions including cancer, HIV/AIDS (and sexually transmitted disease), mental health, physical activity/diet/obesity, smoking cessation, prostate cancer, and hypertension. The methodology used to identify barriers included validated and nonvalidated surveys, and qualitative and empirical research. Because CHI applications involve the participation of consumers, their caregivers, clinicians, and often developers, barriers can apply to any of the participants and the type and impact of the barrier may vary significantly between providers, developers, patients, and their caregivers. Thus, this analysis of the barriers included barriers that impede participation of any of the above groups.

In terms of systems-level barriers, six studies addressed Internet access at home or in the community and six found this to be a barrier. One study identified hardware requirements and another study identified mobile device shape/design/configuration as a systems-level barrier. Another five studies cited incompatibility with current health care as a barrier.

Identified individual-level barriers included clinic staff who feared increased workloads, lack of built-in social support, forgotten passwords, automated data entry inability to allow for back entry of old data, lack of adequate user customization, and substantial financial investment. Nineteen studies queried application usability or user-friendliness and all 19 found evidence of this barrier. Eleven studies explored patient knowledge, literacy, and skills to use the CHI application. All found these deficits to be barriers while one study found no evidence that literacy or knowledge deficits were a barrier. Six studies considered the possibility that users would find the application too time-consuming and five of these studies cited the evidence in the results section, while the one additional study cited too many emails to participants as a barrier. Utilization fees were also identified as a barrier. Five studies sought information about privacy concerns and four reported concerns over privacy as a barrier. These studies also found concerns over the control of information or lack of trust to be barriers. Only two studies queried for potential cultural barriers and one study found evidence of this. The expectations of consumers including acceptability, usefulness, credibility, expectations, and goals were found to be barriers in eight studies. Cost was mentioned as a barrier in only one study and only one study found evidence that physical or cognitive impairment resulted in barriers to the use of CHI applications. Finally, anxiety over the use of computers, complaints about lack of personal contact with clinicians and the belief that health IT would not be an improvement to current care were mentioned in two studies as barriers.

Key Question 3: What knowledge or evidence deficits exist regarding needed information to support estimates of cost, benefit, and net value with regard to consumer health informatics applications?

The literature was at a very early stage of development. Many questions have only been evaluated by one study. Thus, confirmatory studies have generally not been done. In addition, no high quality studies have been conducted regarding several important questions. Broadly, these questions can be grouped into at least one of four categories: patient-related questions; CHI utilization factors; technology-related issues (i.e., hardware, software, and platform related issues, and health-related questions).

Patient-related questions. The literature is relatively silent on the question of whether or not significant differences in patient preferences, knowledge, attitudes, beliefs, needs, utilization and potential benefits exists across gender, age and race/ethnicity. The same could be said for potential gender and race or ethnicity-based differences. Beyond these demographic differences, the field of CHI is developing within the context of a global emergence of technology based realities including Web 2.0/Web 3.0 and ubiquitous computing which are enabling an unprecedented level of user determined interactivity and functionality. The degree to which this functionality could be harnessed for the health benefit of consumers is unknown. The targeted uses of CHI applications must increasingly be focused on more than just the index patient. The role of sociocultural and community factors will likely exert significant effect on access, usability, desirability and benefit of CHI applications. Issues related to trust, security, confidentiality need to be further explored. Because the bulk of the currently available research has been conducted on the 18-to 65-year-old adult population, more work needs to be done among the populations that may have the most potential for using CHI applications. Seniors may stand to benefit from those applications that reduce social isolation and independence. Adolescents are some of the most intense technology users. Their natural affinity for technology may prove advantageous to CHI applications that could be developed in the future. Finally, most of the currently CHI research is being conducted among predominately white/Caucasian populations. Early evidence suggests that differential utilization patterns and preferences exist by race. Such differences could potentially lead to differential efficacy of emerging CHI applications. This could have the unintended consequence of enhancing rather than reducing some racial and ethnic disparities in health care. Age and race/ethnicity subgroup differences need to be netter understood and those differences incorporated into the development of emerging applications to ensure efficacy among all population subgroups.

CHI utilization factors. Despite a rapid increase in access to broadband services among all population groups, age groups and geographic regions of the country, differential access to broadband internet access may have significant implications in terms of health benefits that may be derived from these tools and applications. While many in the younger generations become very technically savvy at an early age, many Americans still have limited health literacy. These CHI utilization factors suggest the need for a more robust evaluation of the epidemiology of broadband access and technology literacy in the United States.

Technology-related issues. The majority of CHI applications are designed for use on personal computers as Web-based applications. Many more potential platforms exist that have not been evaluated. In addition, emerging evidence is suggesting that the CHI applications and functionality that consumers want and need are not always what health care practitioners think they need. As a result, important sociocultural and human computer interface design elements may not get incorporated adequately into emerging CHI applications and therefore lead to CHI applications with limited efficacy.

Health-related questions. Finally, most CHI applications that have been evaluated tend to focus on one or more domains of chronic disease management. Insufficient attention has been given to the role of CHI applications in addressing acute health problems. The role of CHI applications in primary, secondary, and tertiary prevention also needs to be more adequately explored. Sociocultural factors are increasingly important determinants of health care outcomes. The potential influence on social factors including social isolation and social support and perhaps even broader social determinants of health need to be evaluated and may prove useful in helping consumers address specific health concerns in the home and community-based setting.

Key Question 4: What critical information regarding the impact of consumer health informatics applications is needed in order to give consumers, their families, clinicians, and developers a clear understanding of the value proposition particular to them?

Several critical information needs must be addressed to enable a clear understanding of the value proposition of CHI applications. It is likely that the knowledge gaps needed to establish a value proposition, while overlapping, are not identical across all potential stakeholders. Because providers are often most concerned about clinical outcomes and costs, it seems reasonable that questions of the impact of CHI applications on provider or health care processes, costs, and outcomes as addressed in this report will need to be more definitively characterized. In addition, the potential liability a provider might incur from a patient using a CHI application will also need to be addressed.

Patients often cite convenience and anonymity as the primary reasons the Internet has become such a major source of health information. It is likely that the more these elements can be incorporated into emerging CHI applications, the more likely they will be considered of value by consumers. Other related factors such as usability, portability, and patient-centered functionality are likely important characteristics of CHI applications that may help drive utilization. Those technologies that exist and enable consumers to accomplish tasks (empower) without further complicating individuals’ lives may ultimately prove to be the most widely valued CHI applications. By expanding the number of platforms available to consumers, CHI applications may become more appealing to a broader consumer base and thus prove valuable to those consumers who could most benefit, but may not otherwise use a more traditional CHI application.

Discussion

Overall, despite the significant heterogeneity and limited nature of the literature, the following themes were suggested by the studies included in this review. First, there may be a role for CHI applications to reach consumers at a low cost and obviate the need for some activities currently performed by humans. In addition, the data suggest that CHI applications may also be used to enhance the efficacy of interventions currently delivered by humans. Several studies compared the use of a CHI application and traditional therapy against traditional therapy alone. Many found that the group receiving traditional therapy with a CHI application had more benefit than traditional therapy alone. Thirdly, the studies evaluated in this review tended to support the finding that at least three critical elements are most often found in those CHI applications found to exert a significant effect on health outcomes. These three factors are (1) individual tailoring, (2) personalization, and (3) behavioral feedback. Personalization involves designing the intervention to be delivered in a way that makes it specific for a given individual. Tailoring refers to building an intervention in part on specific knowledge of actual characteristics of the individual receiving the intervention.

Finally, behavioral feedback refers to providing consumers with messages regarding their progression through the intervention. Interestingly, it is not clear from this literature that CHI-derived behavioral feedback is any better than feedback originating from human practitioners or others. Rather, it appears that the feedback must happen with an appropriate periodicity, in a format that is appealing and acceptable to the consumer, not just the provider. Finally, despite the paucity of studies in many areas of this emerging field and because of the methodological limitations found in many of the studies, the body of the available scientific evidence suggests that CHI applications may hold significant future promise for improving outcomes across a wide variety of diseases and health issues. In terms of health care processes and relationship centered outcomes, the literature is positive but very limited. Most of the currently available research has evaluated the impact of CHI applications on intermediate health outcomes. Due in part to the number of studies conducted to date, the evaluation of both short-term and longer-term outcomes, the utilization of significant sample sizes, appropriate statistics, the near uniformity of dependent variables across studies, and cogent articulation of the theoretic bases of the CHI content and methodology in most studies, the literature appears strongest for CHI applications targeting intermediate outcomes related to smoking cessation. In terms of clinical outcomes, the weight of the evidence appears strongest for the use of CHI applications on mental health outcomes. Evidence-based conclusions regarding economic outcomes can not be made at this time.

Despite the positive nature of some of the available evidence, significant research opportunities and knowledge gaps exist in terms of understanding the role of CHI applications targeting children, adolescents, the elderly, and specifically nontraditional (family members, friends, allied health workers) patient caregivers. The role of Web 2.0, social networking, and health gaming technology in CHI has not been adequately evaluated. Much more work needs to be done to understand consumer desires and needs versus provider perceptions of patient desires and needs in terms of emerging CHI applications and tools. Similarly, much more work is needed to explicate the effect of CHI applications on health outcomes among racial and ethnic minority populations, low-literate populations, and the potential effect of these applications on health care disparities.

Finally, CHI research would be greatly enhanced with standardization and widespread utilization of a transdisciplinary CHI nomenclature and a CHI evaluation registry to facilitate uniform reporting and synthesis of results across emerging CHI applications, interventions, and evaluations.

PubReader format: click here to try

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (3.3M)

In this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...