Future Research

Need for High Quality Evidence

High quality evidence is lacking in many MMIT phases, care settings, and populations (Table 29). Despite the fact that many RCTs exist in the MMIT literature, they are concentrated in certain areas: 77 of 88 RCTs evaluated CDSSs. The prescribing and monitoring phases have a strong base of studies and systematic reviews. Almost completely lacking were studies in the other phases. For this report we provide the numbers of studies and research methods used (Table 29). In addition, we used the bibliographies and summaries from more than 100 systematic and narrative review articles for this report.

Almost half of the articles that met the content criteria for MMIT did not include a comparison group, hypotheses testing or qualitative methods. These articles were not formally evaluated but are listed in the bibliography. We need well-designed research studies with control groups and appropriate analysis. Research needs center on issues related to:

• Appropriate methods for MMIT and the needs of the stakeholders. Important issues are multicentered studies, research teams with a broad experience or consultation in research methods, statistical analyses, clinical care, and informatics, and that represent the interests of all groups involved in the use of the MMIT system.
• Recognition that MMIT applications are complex interventions and their evaluation must reflect their use in real life settings and situations (i.e., complex interventions methods and pragmatic trials).
• Qualitative and mixed methods studies to understand issues related to workflow, communications, interdisciplinary collaboration and care processes, and patient use and values.

We also found that certain technologies (CDSSs, either stand-alone or integrated with CPOE systems) are well-studied. Although much data exist, as these CDSS and CPOE systems evolve and have greater penetration among health care settings, we should continue to evaluate their effectiveness. The tools that are outside the prescribing phase of medication management and the health IT tools that pharmacists, nurses, other health care professionals, and patients use are less well-studied—fewer studies and weaker methods.

Need for Well-Designed Research

Despite having 88 RCTs in the MMIT literature base, many of the studies have weaknesses. This is shown by the low quality scores, most of which were in the range of four to five out of nine points. In addition, we saw errors and poor methods in published studies. For example, most of the RCTs with clinical outcomes (n = 28) used cluster randomization methods to allocate clinical units or clinicians to study groups, but analyzed and reported results based on patients or medication events. Many authors did not test or adjust for clustering so that complex analyses could be accomplished appropriately. We also identified problems with poor application of methods in most other research studies.

Training informaticians in research methodology and statistical methods is crucial. Many programs sponsored by the U.S. National Library of Medicine and other institutions are graduating health informaticians. Training programs are content-rich because of the breadth of the field. Formal training and experience in the research methods and statistical analyses components of the training initiatives might be useful to determine what is being taught and if it is sufficient to produce researchers who are competent in evaluating MMIT and other health IT systems.

By settings and levels of care. Adult hospitals were relatively well-studied and we have sufficient evidence to show that MMIT systems for ordering medications improve processes and reduce medication errors. Adult ambulatory care clinics were also well-represented in the literature, although studies of errors and error prevention have not been done. Additional studies are especially needed in the nursing home setting, where some 1.6 million people receive care annually, and concern continues about the quality of pharmaceutical care, the frequency of polypharmacy, and an insufficient health care workforce with a poorly developed safety culture. Other long-term care settings such as assisted living and home-based primary care also need more research. The number of older adults continues to increase rapidly and they frequently have multiple comorbidities resulting in complex medication regimens, polypharmacy, and ADEs. Studies conducted in pediatric hospitals are warranted because these patients are particularly vulnerable to medication errors and those medication errors that do occur have three times the potential to cause harm.⁸⁴⁷ Community pharmacies and the newer mail-order and online pharmacy services were not studied. Evaluating these settings may be problematic because of their commercial nature. Homes and other residential or community settings will become more important to study with the spread of patient-centered medicine and associated technologies such as PHRs and remote MMITs.

Monitoring. Our data suggest that interventions that focused on laboratory-based medication monitoring (22 of 29 studies) were associated with the most number of interventions, and showed statistically significant changes in at least half of its main endpoints. We recommend additional research in this area especially because laboratory data are readily available in most clinical settings, and studies in the acute, ambulatory, and nursing home settings suggest that failure to act on available laboratory information accounts for a substantial number of ADEs.^848–850 With the integration of more health IT systems and the move to more patient directed care, systematic monitoring will become even more important.

Practitioners and patients. Nurses and pharmacists are not studied as thoroughly as physicians. Mental health professionals and other health care workers who prescribe, including dentists, are studied even less than nurses and pharmacists. Each group of health professionals reports different needs for their MMIT and health IT tools (e.g., specialist physicians compared with primary care, nurses compared with physicians in hospital wards need compatible but different MMIT tools for ordering, dispensing, and administering). These differences need to be studied and applied in building, evaluating, and implementing MMIT applications. Nurse practitioners, advance practice nurses and physician assistants, and allied and other health professionals should also be the target of MMIT interventions, especially because they play an increasing role in providing primary and subspecialty care, especially in the United States. The move to patient-centered care and chronic disease management also make the study of patients and their informal caregivers an important area for research and development.

Unique needs of evaluation of health IT. MMIT applications are neither simple nor isolatable components that can be easily studied as such. Research methods to evaluate MMIT applications should be based on principles of complex interventions. For example, the U.K. Medical Research Council provides a framework for individuals to consider when planning complex intervention projects (http://www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC003372).

In addition, MMIT applications are often being changed and modified to address problems or to implement fixes or upgrades. This ever changing aspect of health IT poses challenges for health researchers. Classical evaluation and research methods dictate that what is being evaluated needs to be stable over the time period of the study. “Fast” analysis methods may need to be developed.

MMIT systems, as for any IT system, are easiest to study in laboratory settings. Because health care is so complex, the study of MMIT systems must be done in real settings. Pragmatic trials methods (trials done in real life situations) may also need to be applied in many research projects related to MMIT systems.

Commercial interests also complicate the evaluation of MMIT and health IT. This makes research harder to do and provides barriers to the most common government-based funding sources. AHRQ has provided funding for many unique and valuable health IT applications that have included much evaluation of MMIT. They should be encouraged to continue to provide leadership in this domain.

Another challenge to research methods is that often the existing evaluations have been done by system developers or implementers. This ownership can cloud an evaluator’s vision (i.e., bias) or at least stand in the way of publication of negative findings. Some evidence exists that evaluation of one’s own system contributes to biases towards the system being found to be positive.⁷²⁵

Knowledge translation/translational research. Because many of the studies in MMIT are small descriptive studies and often done by developers, generalizabity or transferability are often not ideal. Researchers in this domain of “getting research into practice” (i.e., knowledge translation or translational research) can provide tools and insights in two areas. The first is how to harness existing knowledge tool development such as building the knowledge base behind health IT systems such as CDSS, CPOE systems, and other knowledge summaries such as clinical practice guidelines and order sets. Second, those involved in knowledge translation can assist in the dissemination of studies of MMIT applications, or provide leadership in how we can disseminate studies of health IT.⁷²⁵

MMIT applications do not stand alone. MMIT implementation has had substantial impact on communication, interpersonal, and inter-professional relationships, and development of unintended consequences. In some cases, issues such as rage against the machine, guilt, embarrassment associated with reminders and alerts, and frustration have been reported. The qualitative and mixed methods studies summarized in the KQ1: Effectiveness section are examples of studies that have shown how MMIT and health IT can and do affect individuals on a personal level. More of these studies of the effects of these technologies on people, clinicians, and individuals need to be done in various settings and with all technologies. Workflow and communication are ideally studied using qualitative and mixed methods.

Sustainability. Sustainability is tremendously important to MMIT and other health IT applications. See also KQ5: Sustainability. We could not find an agreed upon definition and used one from Australia: “the ability of a health service to provide ongoing access to appropriate quality care in a cost effective and health-effective manner.” The informatics domain needs to have an agreed upon definition of sustainability. Once this is established, research needs to be done to identify our current “sustained” systems and determine the factors that are associated with them. Qualitative and quantitative studies are essential and they need to be done by people with strong content and methods background and sufficient financial backing. Partnerships among Federal groups (e.g., AHRQ, Office of the National Coordinator for Health Information Technology, Health Resources and Services Administration, and Centers for Medicare and Medicaid Services), vendors, professional organizations (e.g., Healthcare Information and Management Systems Society, American Medical Informatics Association, and the major pharmacy associations such as the American Pharmacists Association, American Society of Health System Pharmacists, Academy of Managed Care Pharmacy, American College of Clinical Pharmacy, American Society of Consultant Pharmacists, National Community Pharmacists Association, and National Association of Chain Drug Stores), researchers, and others could work together to address the sustainability challenge. We also need studies of successful MMIT applications as well as systematic study of failures. Perhaps the HITECH Act of 2009 will lead to improvements and sustainability of health IT applications that specifically support the medication management continuum through meaningful use.

Standards and certification. We were asked to provide the evidence on the influence of standards and certification and how they affect MMIT systems. This evidence is sorely lacking. Standards are necessary for interoperability and smooth functioning of existing systems and large scale integration of data at State and national levels. Leadership, probably more than research efforts, continues to be needed in this domain.

Measurement and definitional issues. Other gaps in the evidence that need addressing are definitional or measurement issues. Because health IT is an interdisciplinary field, standard definitions are crucial. Producers and users of research and evaluations function best when everyone is using the same terms with the same parameters. One simple example we found is a formal working definition of the difference between an e-Prescribing system and a medication-based CPOE system. Some European literature described a system as e-Prescribing, while the same system in the United States would be classified as COPE. Another idea that seemed to cause confusion among authors and readers is the use of EMR or EHR systems, and hospital information systems. Consistency in reporting and communicating MMIT information is also important.

Clinical practice guidelines, CPOE, and CDSS. One final issue that seems unresolved centers on the evidentiary nature and strength of the knowledge base that forms the foundation of CDSS applications, order sets, and most other MMIT systems. Some systems linked to established clinical practice guidelines, but we did not find studies that addressed the strength of the evidence base of MMIT systems. We feel that a strong, reliable, consistent, fully disseminated, and continually updated evidence base for MMIT and other health IT systems is vital. More emphasis has been placed on the mechanics of these systems than content. Establishment of standards and content for the knowledge base is something that is potentially more important than the mechanics of these decision-support systems. The U.S. National Library of Medicine could provide leadership here. They have already built strong knowledge management tools such as their Unified Medical Language System that knits together multiple vocabularies in a machine processable form. They have also developed other information handling and processing tools, and techniques such as natural language processing capabilities of medical text, RxNorm (a standardized electronic nomenclature for clinical drugs and drug delivery devices), and codified drug allergy information provision and transfer. Their work in genomics and proteomics is also important once an individual’s genetic information is ready for useful integration into our health IT and MMIT systems to provide individualized medicine.