• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information

Evidence Table 10KQ1: primary qualitative outcomes for all technologies across phases

Article InformationMM Phase(s)HIT studied
Integrated systems
SettingsResultsConclusions
Agostini (2007)319
Design: Qualitative
N = 36 house officers most of whom were PGY1
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
PrescribingCDSS/CDS/CCDS/reminders
Integrated CPOE/POE system
EHR/EMR system
Acute care/tertiary, AcademicBenefits and barrier themes were identified. Benefits include awareness of patient safety risks (delirium, falls, and general patient safety risks), usefulness of computer technology, and value of educational content of the reminder (geriatrics pharmacology review and nonpharmacologic treatment options). Barriers were related to demands of reading the reminder, role of clinical experience, and information content of the reminder.Both barriers and benefits of computer-based reminders were identified by house officers dealing with the elderly patients with insomnia.
Ahearn (2003)320
Design: Qualitative
N = 22 general practitioners
Implementation: 00/0000
Study Start: 04/2002
Study End: 05/2002
Prescribinge-RxAmbulatory care7 main themes emerging from the focus groups;
  1. reaction to prompts;
  2. concerns and potential problems re: comprehensiveness and accuracy of alerts;
  3. effects on prescribing behaviour;
  4. need for training;
  5. helpful CDSS features e.g. sensitivity settings, alerts in red, etc;
  6. suggested improvements; and
  7. attitudes to evidence-based guidelines.
GPs believed that important interactions may be missed because of desensitization from too many alerts (which also intrude on workflow); that interaction alerts need to be severity graded and only significant ones should appear; and that improved computer-user interface design could enhance the usefulness of the decision support systems.
Arar (2005)321
Design: Observational study
N = 50 clinical encounters with patients
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Education of patients and clinicians but not pre-professional educatione-Rx
Integrated EHR/EMR system, Imaging systems, Laboratory system
Ambulatory careDirect observation and content analysis showed that the EMR/e-Rx facilitated communication with respect to the process of care that included checking active and inactive prescriptions and new and refill prescriptions, names of medication, and other medication themes (mail- order issues, adherence, self regulation, alternate OTC issues).The EMR improved communication between physicians and patients in relation to medication issues.
Ash (2004)322
Ash (2003)323
Sittig (2005)324
Ash (1999)325
Ash (2000)326
Ash (2003)327
Ash (2001)328
Design: Qualitative
N = 58 physicians, nurses, administrators, IT professionals
Implementation: 1966 onwards
Study Start: 00/1998
Study End: 00/2003
PrescribingCPOE/POE systemAcute care/tertiary, General Hospital, Academic324 Negative emotional responses were more prevalent than positive or neutral.323 Four high-level themes were identified:
  1. organizational issues such as collaboration, pride, culture, power, politics, and control;
  2. clinical and professional issues involving adaptation to local practices, preferences, and policies;
  3. technical/implementation issues, including usability, time, training and support;
  4. issues related to the organization of information and knowledge, such as system rigidity and integration.
Relevant differences between teaching and nonteaching hospitals include extent of collaboration, staff longevity, and organizational missions.322 Themes included: language and misunderstandings, context matters and it affects the way of doing things, benefits and tradeoffs, ‘contrasts, conflicts and contradictions’, collaboration and trust, customization and organization of information, defining boundaries of CPOE, ongoing nature of implementation.327 Explores the theme of leaders and bridgers-administrative; clinical; bridgers/support staff; skills and training.326 Physicians, admin and IT have different perspectives of the technical and organizational aspects of CPOE; the multiple perspectives model was used to offer structure to the results.325 Themes relating to housestaff perceptions of CPOE included education; benefits; problems; feelings about; implementation strategies and the future of CPOE.
324 Designers need to recognize that CPOE features and implementation strategies can increase negative emotions and impact success of implementation. Positive feedback might alleviate some of the problems.323 An organizational culture characterized by collaboration and trust and an ongoing process that includes active clinician engagement in adaptation of the technology were important elements in successful implementation of physician order entry at the institutions that we studied.322 Publication of the results of these iterative inquiries served to promote a realization that implementation of CPOE is not easy and that the negatives must be weighed against the positives.327 Understanding multiple perspectives should be undertaken, with insights used to form strategic implementation plans.325 house officers felt that CPOE assists patient care but may undermine education; it works best when tailored to fit local and individual workflow; implementation strategies should include mechanisms for engaging housestaff in decision process.
Avery (2005)329
Design: Survey
N = 21 experts (Delphi panel members)
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Monitoring including patient adherence and compliance, Prescribinge-RxNot specifiedKey Themes:
  1. importance of computerized alerts;
  2. need to minimize spurious alerts;
  3. making it difficult to override critically important alerts;
  4. having audit trails of such overrides;
  5. support for safe repeat prescribing;
  6. effective computer– user interface;
  7. importance of call and recall;
  8. need to be able to run safety reports.
The high level of agreement among the expert panel members indicates clear themes and priorities that need to be addressed in any further improvement of safety features in primary care computing systems.
Banet (2004)198
Design: Before-after
N = 55 nurses
Implementation: 05/2003
Study Start: 00/0000
Study End: 00/0000
AdministeringCPOE/POE system, e- MAR, e-Medication administration system (e-MAR, e-TAR)
Integrated Imaging systems, Laboratory system, Pharmacy
ED, AcademicFor the open-ended question on ease of CPOE documentation, responses fell into the following themes: improvements in the clarity of orders, system helps organize and time their tasks, positive responses about efficiency and standardization of documentation provided by templates, general improvement in ED processes, decreased number of verbal orders and time searching for charts. For the open ended question for suggestions for improvements, themes included: additional terms and phrases for templates, process issues not affected by the ED application, complaints regarding technical problems with the system, suggestions for additional functionality, comments about the medication order icon on the tracking board.The findings from this study indicate that users perceived no change in the total amount of time spent on documentation, a perception that was corroborated by the results of the time-motion studies. Nurses also perceived that certain processes, such as laboratory and radiology tests, were accomplished more efficiently after the implementation.
Bastholm Rahmner (2004)330
Design: Qualitative
N = 21 physicians
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Prescribinge-Rx
Integrated CDSS/CDS/CCDS/reminders Pharmacy
General Hospital4 categories for possibilities and obstacles.
  1. possibilities related to access to patient drug history (which is not met by the new system), increased pharmacological knowledge from alerts etc., access to information more readily and time saved;
  2. obstacles centered around technical problems given current problems with the EMR and too frequent alerts, computer shortages within the ED, altering routines and habits and the resulting diminishing patient contact since they need to leave the consulting room to enter the prescriptions;
Gaining access to patient drug history enables physicians to carry out work in a professional way. Alerts and producer-independent drug information are valuable in reducing workload. However, technical prerequisites form the base for a successful implementation. Time must be given to adapt to new ways of working.
Beuscart-Zephir, (2010)331
Design: Qualitative
N = Not Specified Nurse
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
AdministeringCPOE/POE system
Integrated EHR/EMR system
Acute care/tertiary, 3,000 Beds Academic
3.

standard ethnographic methods were used to support the analysis of the current work system and work situations, coupled with cognitive task analysis methods and documents review;

4.

usability inspection (heuristic evaluation) and both in-lab (simulated tasks) and on-site (real tasks) usability tests were performed for the evaluation of the CPOE candidate

. The study focused on the nurses’ tasks of preparing and administering oral route drugs to the patients, with a particular attention to the nurses’ needs in terms of information necessary to efficiently and safely support their tasks.
The analysis of the work situations identified different work organizations and procedures across the hospital’s departments. The most important differences concerned the doctor–nurse communications and cooperation modes and the procedures for preparing and administering the medications. The assessment of the medication CPOE functions uncovered a number of usability problems, including severe ones which could be impossible to detect.
Boonstra (2004)332
Design: Qualitative
N = 36 GPs
Implementation: 00/2001
Study Start: 00/2001
Study End: 00/0000
PrescribingCDSS/CDS/CCDS/reminders e-Rx
Integrated CDSS/CDS/CCDS/reminders EHR/EMR system, Formulary, Insurance
Ambulatory careFive factors related to the perceived advantages and disadvantages of the system.
  1. system: usability issues and features of the system viewed as positive by some (user friendly, integrated) and negative by others (unfamiliar with the disease codes system, lack of flexibility, lack of computer resources);
  2. finance: though the software was free, the government reaped the economic benefits of using it and the GPs were required to keep their EHR systems up to date;
  3. system in consultation process: some felt it was more efficient during consultation and led to better quality; others felt it took longer and took away from patient focus;
  4. cultural factors: users tended to have a culture of professional quality, non- users tended to focus on human relations;
  5. policy environment: helps doctors become more cost conscious, but benefit only for insurers, and focused solely on costs.
Designing a system that met the diverse needs of users more satisfactorily, in being more compatible with their diverse cultures, may have encouraged wider and more creative use, and thus achieved more savings than the present arrangements have achieved.
Buhrer (2008)333
Design: Qualitative
N = 67 Nurses
Implementation: 11/2007
Study Start: 10/2007
Study End: 12/2007
AdministeringBCMAAcute care/tertiaryPre-BCMA:
  1. scheduled medication passes take longer (35/35);
  2. system is overwhelming (25/35);
  3. the system would direct nurses’ attention away from patients (23/35);
  4. nurses expected system to improve patient safety (30/35).

Post-BCMA:
  1. liked working with the system (19/32);
  2. BCMA improves safety (28/32);
  3. overwhelmed at beginning of the implementation (12/32);
  4. more focused on system than the patient and found this annoying (20/32);
  5. would like to switch back to the previous, paper- based system.

Negative attitude:
  1. computer carts: too heavy and too big and some without storage drawer;
  2. scanners: too few wireless scanners;
  3. batteries: unreliable power indicators and weak batteries;
  4. lost orders: sometimes disappeared from the medication schedule, causing confusion;
  5. documentation: required launching a separate cumbersome application.

Positive attitude:
  1. organization: nurses found BCMA system’s scheduling function helpful;
  2. carts: some use cart as a “portable desk”.
Implementation of BCMA into the active process of medication administration was a significant source of negative attitudes in nurses. Qualitative examination of users’ attitudes (negative and positive) toward specific attributes can result in improved design of both technology and implementation strategies.
Campbell (2009)334
Design: Qualitative
N = 32 semi- structured interviews=43 hours; 400 hours of observation shadowing 95 clinical providers
Implementation: 00/0000
Study Start: 08/2004
Study End: 04/2005
PrescribingCPOE/POE system
Integrated EHR/EMR system
Acute care/tertiary, General Hospital, 340 (Wishard); 893 (Mass.); 150 (Faulkner); 725 (Brigham); 238 (Alamance) Beds AcademicThemes: CPOE systems can affect clinical work by:
  1. introducing or exposing human/computer interaction problems;
  2. altering the pace, sequencing, and dynamics of clinical activities;
  3. providing only partial support for the work activities of all types of clinical personnel;
  4. reducing clinical situation awareness;
  5. poorly reflecting organizational policy and procedure.
CPOE systems are tools intended to support and improve the delivery of care, and are not solutions for all problems related to clinical practice. Workflow issues resulting from CPOE can be mitigated by iteratively altering both clinical workflow and the CPOE system until a satisfactory fit is achieved.
Cross (2009 )335
Design: Qualitative
N = 10 patients
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Monitoring including patient adherence and complianceCDSS/CDS/CCDS/reminders home automated tele- management
Integrated electronic messaging system, Personal health records systems
Ambulatory care, Home, AcademicPatients’ perceptions:
  1. constant communication: assist them in monitoring the symptoms of disease, both from a medical provider and a patient perspective;
  2. use of computer was not difficult;
  3. improved safety;
  4. keep the patient and provider up to date on changes in symptoms.

Analysis of the responses were sorted into three topic areas:
  1. user attitudes about the interface;
  2. user attitudes about the content of self- testing;
  3. user attitudes about the self-testing process.
Pilot testing of a tele-management system customized for UC revealed a high level of acceptance and interest among patients. The results suggest that implementation of a tele-management system will be feasible on a long-term basis with only minor modifications.
Crosson (2007)336
Design: Qualitative
N = 47 clinicians
Implementation: 10/2006
Study Start: 03/2006
Study End: 11/2006
Prescribing, Transmission, order communicatione-Rx, e-Transmission of the prescription to/from doctor to pharmacy Handheld
Integrated, EHR/EMR system
Ambulatory carePractices which successfully implemented the e-Rx system exhibited greater familiarity with the capabilities of the systems and had more realistic expectations of the benefits. Physicians in these practices tended to have positive attitudes about and previous experiences with e-Rx or EMR, participation in continuing education courses relating to e-Rx, and plans for the future use of other HIT. Physicians in the 3 practices where the programs were successfully installed but unevenly implemented had high expectations about the ease of implementation, but at the same time reported concerns about how e-Rx might affect their clinical independence or undermine their authority with patients. Prescribers and staff members in the 2 practices that successfully installed, but then discontinued use of the program exhibited very little advance knowledge of program functions or the potential effect on prescription workflow. Two practices failed to install e-Rx; physicians and support staff in these practices expected that e-Rx would lead to greater efficiency and safety but, at the same time, had little specific knowledge of program functionality.Practice leaders should plan implementation carefully, ensuring that practice members prepare for the effective integration of e-Rx technology into clinical workflow.
Feldstein (2004)337
Design: Qualitative
N = 20 Clinicians
Implementation: 00/1996
Study Start: 00/0000
Study End: 00/0000
PrescribingCDSS/CDS/CCDS/reminders CPOE/POE system
Integrated CDSS/CDS/CCDS/reminders EHR/EMR system
Ambulatory careThe study found some common theme with respect to prescribers’ frustrations with CPOE systems:
  1. alerts that contained low-priority information;
  2. intrusive alerts presented at the wrong time in the workflow;
  3. difficult-to-interpret alerts;
  4. delays caused by the alert;
  5. redundant and repetitive alerts.
Although alerts may slow the work process, busy clinicians generally find them helpful. Safety alerts need to be concise and relevant, have clear action steps, and provide options for users with different experience levels and work styles. Health care decisionmakers should prioritize safety-related alerts and educational programs to facilitate the implementation of CDSS at CPOE.
Fernando (2009)338
Design: Qualitative
N = 9 ED specialists and registrars
Implementation: 01/2006
Study Start: 05/2006
Study End: 12/2006
PrescribingCPOE/POE system
Integrated EHR/EMR system, Imaging systems, Laboratory system
Acute care/tertiary, 66 ED bedsThree major issues emerged from the findings:
  1. the implementation of the new system was accompanied by major shifts in ED work responsibilities and tasks;
  2. the appearance of dysfunctional consequences related to the excess time it took to electronically order and the usability of some features of the new system;
  3. doctors’ concerns that their views and opinions about design and implementation of the new system had not been adequately addressed
The implementation of electronic ordering has important implications for ED functioning and the delivery of patient care. The complexity of the ED makes it vulnerable to disruption caused by inadequate system design and ineffective channels of communication across the hospital.
Fields (2007)339
Design: Qualitative
N = 17 Health care providers
Implementation: 08/2006
Study Start: 00/0000
Study End: 00/0000
PrescribingCPOE/POE system
Integrated Hospital information system
Acute care/tertiary, Critical care units (CCU, ICU, NICU) 19 beds in the MICU BedsFour themes were suggested:
  1. ease of use;
  2. speed;
  3. trust;
  4. hopefulness.
Participants valued CPOE potential and were hopeful that future systems would be easy to use, decrease error potential, be more customizable for individual users, and contain concise physician order sets to foster medication order safety.
Participants valued CPOE potential although they commented on improvements and challenges.
Franklin (2007)50
Donyai (2008)51
Barber (2007)52
Franklin (2008)53
Franklin (2007)54
Design: Before-after
N = 4,803 medication orders
Implementation: 06/2003
Study Start: 00/0000
Study End: 00/0000
Administering, PrescribingAutomated Dispensing Machine, e-Medication administration system (e-MAR, e-TAR) e-Rx
Integrated Pharmacy
Acute care/tertiary, 28 surgery bed ward of a teaching hospital Beds Inpatient hospital based, AcademicThe system was successfully implemented on the ward, and remained in operation for over 2 years. Many of the technical components of the system initially showed problems, but evolved with increased functionality and improved performance. Attitudes to the system in the early stages were mixed. Over time, staff attitudes changed to become more balanced and the potential benefits of the system became clearer to most. The system structured the work of staff, sometimes unexpectedly.This theory-led evaluation offers valuable insights into a critical contemporary policy area. Technical systems are never perfect, and they require time and effort to become embedded into any particular clinical context. The effectiveness of ICT changes and develops over time, have quite different effects in different settings. For this reason a sophisticated evaluation framework is necessary.
Georgiou (2009)340
Design: Qualitative
N = 50 hospital employees
Implementation: 00/0000
Study Start: 01/2006
Study End: 03/2006
Prescribing, Transmission, order communicationCPOE/POE system
Integrated EHR/EMR system, e- MAR
Acute care/tertiary, Not specified, Inpatient hospital based, AcademicThe 20 recurring themes were grouped into 4 major constructs: Will it help?, Will it work?, Will it impair existing interaction?, and Will we cope?The hospital employees had major concerns before implementation of a CPOE system. The elucidation and understanding of these concerns and worries can help to inform and strengthen implementation strategies.
Graham (2008)341
Design: Qualitative
N = 7 physicians
Implementation: 00/0000
Study Start: 00/2006
Study End: 00/2007
PrescribingCDSS/CDS/CCDS/reminders
Integrated EHR/EMR system, Formulary
Emergency department, AcademicCoding categories for identifying usability problems from the analysis of video-based data included:
  1. interface problems;
  2. content problems;
  3. slips and mistakes.
From 56 recorded sessions, a total of 422 events were recorded. The events were further grouped into seven main categories:
  1. negative comments;
  2. positive comments;
  3. neutral comments;
  4. application events;
  5. problems;
  6. slips;
  7. mistakes.
This study provides a framework for evaluating CDSS applications in a clinical environment and has identified specific areas for improvement in the applications utilized. A number of interface issues that could lead directly to adverse medical events that were identified raises concerns about the potential for similar undocumented problems in other clinical applications currently in use or being developed for implementation. Application of usability engineering principles can help identify interface problems that may lead to medical adverse events, and need to be incorporated early in the design phase to ensure that such problems can be corrected while there is still time and it is cost effective to do so.
Grossman (2007)342
Design: Qualitative
N = 26 organizations
Implementation: 00/0000
Study Start: 11/2005
Study End: 03/2006
Prescribing, Transmission, order communicatione-Rx, e-Transmission- of the prescription to/from doctor to pharmacy
Integrated Stand-Alone, EHR/EMR system
Ambulatory careQualitative data were narratively analyzed from 44 telephone interviews with 26 medical practices, 21 with e-Rx.Barriers were reported related to maintaining complete lists of patients and their medications, use of CDSS, getting patient-specific formulary data, and EDI. Factors associated with these issues related to product limitations, external implementation challenges (e.g., communication with pharmacists and vendor support), and physician preferences on specific product features.
Holden (2010)318
Design: Qualitative
N = 20 attending Physicians
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
PrescribingCPOE/POE system, EMR
Integrated EHR/EMR system
Acute care/tertiary, 400+ BedsBehavioral beliefs:
  1. performance outcomes;
  2. productivity and efficiency outcomes;
  3. patient outcomes;
  4. financial, organizational, and other outcomes;
  5. affective outcomes;
External Normative beliefs; Control beliefs: controllability; self-efficacy.
EMR and CPOE were commonly believed to both improve and worsen the ease and quality of personal performance, productivity and efficiency, and patient outcomes. Physicians felt encouraged by employers and others to use the systems but also had personal role-related and moral concerns about doing so. Perceived facilitators and barriers were numerous and had their sources in all aspects of the work system.
Hurley (2007)241
Design: Mixed methods
N = 1,087 nurses
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
AdministeringBCMA, e-Medication administration system (e-MAR, e-TAR)
Integrated CPOE/POE system, EHR/EMR system, Pharmacy
Acute care/tertiary, AcademicInterview questions followed the same subscales as the satisfaction scale. Nurses found the new system more time consuming but acknowledged that the extra time was wisely spent to assure verification. They viewed saving time on handwritten, paper- based medication sheets transcribing as a positive change. They felt there was an increased sense of safety for the patients and the nurses and that the system helped with the 5 rights. In terms of access, they appreciated greater access to medications and information (policies, guidelines, drug resources, patient files, etc.), but felt there were still some delays in getting medications from pharmacy.A medication administration system that nurses view as being effective, by promoting efficacy, safety, and easy access, will support their nursing practice. Results of this study can give confidence to nurse executives that nurses can be satisfied with technology to make medication administration safer and more efficient and provide easier access to system components.
Johansson (2010)343
Design: Mixed methods
N = 15 home-care nurses
Implementation: 00/2007
Study Start: 12/2007
Study End: 3/2008
AdministeringBCMA, CDSS/CDS/CCDS/reminders
Integrated Handheld, Drug reference software
Home, AcademicMedication profiles: Most of the nurses had access to the mobile information and the possibility to obtain a profile of the patients’ medication regarding drug–drug interactions, therapeutic duplications and warnings for drugs unsuitable for elderly. Usability: The nurses discussed that it was a time consuming learning threshold, but once used to the LIFe-reader®s’ functions, they were regarded as fast. The nurses experienced that the keyboard of the LIFe-reader® was too small and not suited to the Swedish language, and that the pen was not easy to use. Usefulness: The nurses believed that it would be different to use the PDA once they started to work as district nurses. Some nurses thought they would have used the LIFereader ® in a different way if they could have had it for a longer time or if they knew they could have kept it. The drug reference text in the LIFe- reader® had the highest priority but there was also a potential for more functions and featuresWe found that the LIFe-reader® has the potential to be a useful and user-friendly MDSS for nurses in home care when obtaining profiles of the patients’ medication regarding drug–drug interactions, therapeutic duplications and warnings for drugs unsuitable for elderly patients.
Johnson (2010)75
Design: RCT
N = 3,285 patients
Implementation: 00/0000
Study Start: 04/2007
Study End: 08/2007
Prescribing, Transmission, order communicationCDSS/CDS/CCDS/reminders e-Rx
Integrated EHR/EMR system
Ambulatory care, Pharmacy, Not specified, AcademicImproving communication between prescribers and dispensers; Decreases callbacks in some cases; Pediatric dosing information helps check for potential errors; Increases callbacks in some cases; Need more information to be included in annotations; New SYW feature requestComments suggested that SYW increased callbacks where necessary and decreased them in other situations, but did not contribute to unnecessary callbacks. These findings support the continued and potentially expanded use of SYW by e-Rx systems to enhance communication with pharmacists.
Kazemi (2008)344
Design: Qualitative
N = 19 physicians
Implementation: 00/0000
Study Start: 12/2006
Study End: 01/2007
PrescribingCPOE/POE system
Integrated Hospital information system
General Hospital, 234 Beds Academic3 themes emerged on current prescription process:
  1. decision- making errors;
  2. transcription errors;
  3. over confidence errors.
3 themes were identified in the expected benefits category:
  1. confidentiality issues;
  2. reduction of medication errors;
  3. educational benefits.
4 themes emerged in the perceived obstacles category:
  1. high cost;
  2. social and cultural barriers;
  3. data entry time;
  4. problems with technical support.
Prescription patterns in Iranian teaching hospitals are physician centered, top-down with possibility for medication errors. Although barriers exist towards implementation of CPOE, there is a general willingness among the physicians to use such a system if it provides significant benefit.
Koppel (2005)345
Design: Mixed methods
N = 291 health care providers
Implementation: 00/1997
Study Start: 00/2002
Study End: 00/2003
PrescribingCPOE/POE system Integrated
nurses medication lists, Pharmacy
Acute care/tertiary, 750 Beds AcademicIdentified 22 previously unexplored medication-error sources that users report to be facilitated by CPOE. We group these as:
  1. information errors generated by fragmentation of data and failure to integrate the hospital’s several computer and information systems;
  2. human-machine interface flaws reflecting machine rules that do not correspond to work organization or usual behaviors.
A leading CPOE system often facilitated medication error risks, with many reported to occur frequently. As CPOE systems are implemented, clinicians and hospitals must attend to errors that these systems cause in addition to errors that they prevent.
Koppel (2008)346
Design: Mixed methods
N = 14,2203 medication administrations
Implementation: 12/2001
Study Start: 00/2003
Study End: 00/2006
AdministeringBCMA, e-Medication administration system (e-MAR, e-TAR)
Integrated Hospital information system
Acute care/tertiary, 1,399 Beds Academic15 workarounds falling into 3 categories were identified: omission of process steps (7 workarounds), steps performed out of sequence (1 workaround) and unauthorized process steps (7 workarounds). The probable causes and potential errors for each workaround were determined. Probable causes included technology, task, organizational, patient and environmental related causes.BCMA systems are intended to advance medication safety, our data reveal that integrating BCMAs within real-world clinical workflows requires critical attention to ensure that technology safety features are used as intended and that systems are designed to support this use. Compliance with patient safety protocols is best achieved by configuring BCMAs for efficient as well as safe patient care. Repeated examinations and corrections of BCMA actual uses are needed to optimize their role in preventing medication errors.
Krall (2002)347
Design: Qualitative
N = 16 clinicians (physicians, PAs and nurses) in 3 focus groups
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Monitoring including patient adherence and compliance, PrescribingOutpatient EMR
Integrated EHR/EMR system
Ambulatory care5 themes were identified from the focus group data.
  1. efficiency: alerts and reminders being efficient and not wasting time;
  2. usefulness: alerts being useful and appropriate;
  3. Information content: about timely, rich, and accessible information;
  4. user interface: important for smooth and efficient work and provision of valuable information quickly and accurately;
  5. workflow: issues related to the information being available when needed.
Note that considerable emotion was associated with alerts and reminders (criticism, embarrassment, guilt, frustration, annoyance, and anger).
the clinicians provided considerable feedback on the usefulness and usability of alerts and reminders in EMRs.
Lai (2007)348
Design: Mixed methods
N = 15 pharmacists
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Dispensing, Transmission, order communicationCPOE/POE systemUnspecified Hospital, Pharmacy, Inpatient hospital based
  1. patient safety: pharmacy leaders all believed CPOE would improve patient safety, allergy, dosing and interaction alerts. Some expressed concern that poor design/implementation could lead to increased errors;
  2. pharmacy practice: most believed the system would lead to improved efficiencies facilitating more time spent with patients;
  3. pharmacy profession: most felt CPOE would improve working relationships with physicians and nurses by facilitating new collaborations
The scaling analysis found that pharmacy leaders of community, academia, and hospitals had different experience and/or opinions regarding the impact of CPOE.
Most pharmacy leaders held positive opinions regarding the impact of CPOE on the pharmacy practice and the profession, with varying concerns regarding its impact on practice and safety.
Lapane (2008)90
Design: Mixed methods
N = 276 primary care prescribers and their staff
Implementation: 00/2003
Study Start: 04/2006
Study End: 08/2006
Prescribinge-RxAmbulatory careAn open-ended approach was used to elicit information about the benefits and drawbacks of e-Rx. 15 different parent nodes were defined. Attention focused on 2 parent nodes, impact on clinical practice and software features. Physicians found the drug allergy alerts useful. For drug-drug interactions, they found these beneficial to patient safety. Many of the interaction alerts were however ignored and many were viewed as too trivial or unnecessary. Physicians suggested that alerts be provided for current medication only and for them to be less sensitive, more sensible, possibly having a personal setting for severity levels.Prescribers believe that refinements to the drug alerting systems are necessary to reduce common overriding of alerts. In addition to honing the specificity of the alerts and permitting prescribers to set the severity threshold for alerts, prescribers recommend having the drug alert algorithms run against current medication regimens.
Li (2006)247
Design: Qualitative
N = 2 qualitative researchers (nurse and human factors psychology)
Implementation: 02/2004
Study Start: 00/0000
Study End: 00/0000
PrescribingCPOE/POE system
Integrated Hospital information system
Acute care/tertiaryThe 2 researchers used heuristic methods and identified 5 major problem areas with the CPOE system. These problems centered on text presentation, too much information/too many decisions at one time, color scheme (monochromatic blue/grey with red used as accent and not to note caution or problems). Problems were given to the developers who addressed them in the next redesign of the system.The 5 problem areas that were identified were given to the developers who addressed them in the next redesign of the system.
McAlearney (2006)349
Design: Qualitative
N = 71 Healthcare providers (Physicians)
Implementation: 00/0000
Study Start: 04/2002
Study End: 05/2005
PrescribingComputerized order sets & hand held computers, CPOE/POE system
Integrated EHR/EMR system
Pediatric stand alone hospital, Ambulatory care, Other, Academic2 major themes emerged:
  1. Can it work? Physicians expressed concerns about:
    1. appropriateness of physician-directed CIT as a solution for medical errors;
    2. current technical capabilities;
    3. level of technical support for CIT solutions;
    4. introduction of new errors.
  2. At what cost to the medical profession? Physicians were concerned about the time efficiency and workload redistribution associated with the introduction of CIT.
The study concluded that health care organization attempting to promote physician use of CIT should consider physician’s perspectives about technology adoption and use to address their concerns, reduce skepticism, and increase the likelihood of implementation success.
McCann (2008)251
Design: Mixed methods
N = 53 patients
Implementation: 0
Study Start: 03/2006
Study End: 09/2006
Monitoring including patient adherence and compliancesymptom management system
Integrated Handheld
Ambulatory care
  1. training and familiarization of the handset: patients felt that the training was adequate and the handset was straightforward and easy to use;
  2. length of data collection: patients felt that entering data twice a day for 14 days was acceptable;
  3. daily routine: the system did not appear to impact on patients’ daily routines as it was incorporated into their day in a variety of ways;
  4. symptoms: patients often felt that the six symptoms that were recorded on the handset were adequate, although some patients did indicate that they would have liked the opportunity to report other symptoms;
  5. the alerting facility: overall, patients were happy with the alerting facility of the system, and the real-time, quick response rate of the data collected.
The results of this study indicate that patients with breast, lung and colorectal cancer had positive perceptions and experiences of using ASyMS© to monitor and manage chemotherapy related toxicity.
Motulsky (2008)350
Design: Qualitative
N = 12 community pharmacists
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Prescribinge-Rx
Integrated Insurance, Personal health records systems
Pharmacy, OtherThe model of the effects of e-Rx on professionalization of community pharmacists had 7 themes:
  1. increased analytical capacity;
  2. greater dissemination of knowledge;
  3. better integration of process tasks;
  4. increased process automation;
  5. elimination of intermediaries;
  6. increased tracking capability;
  7. greater informational capability.
The main effects of the e-Rx were analytical capacity of the pharmacists and physician and dissemination of knowledge, integration of process tasks, process automation, facilitates interpretation of prescriptions, improves relevance and meaningfulness of interaction and improves quality of information transmitted.
e-Rx has tremendous capacity to change and improve pharmacists professional work and interactions.
Nanji, (2009)351
Design: Qualitative
N = 10 pharmacy staff
Implementation: 11/2003
Study Start: 00/0000
Study End: 00/0000
DispensingBarcoding-dispensing
Integrated Barcoding system, Pharmacy
Acute care/tertiary, 750 Beds Pharmacy, Inpatient hospital based, Academic3 barrier themes:
  1. process (training requirements and process flow issues);
  2. technology (hardware, software, and the role of vendors);
  3. resistance (communication issues, changing roles, and negative perceptions about technology).
Bar code scanning system implementation is a difficult process with several barriers involving processes, technology and organizational resistance. Adequate training, continuous improvement, and adaptation of workflow to address one’s own needs mitigated process barriers. Ongoing vendor involvement, acknowledgment of technology limitations, and attempts to address them were crucial in overcoming technology barriers. Staff resistance was addressed through clear communication, identifying champions, emphasizing new information provided by the system, and facilitating collaboration.
Novak (2008)352
Design: Qualitative
N = 50 hours of observations
Implementation: 00/2007
Study Start: 00/0000
Study End: 00/0000
AdministeringBCMA, e-Medication administration system (e-MAR, e-TAR)
Integrated CPOE/POE system
Acute care/tertiaryFor prior to BCMA implementation: Themes from the analytical coding were organized according to the nurses’ practice goals-the familiar “Five Rights” of medication: Right Patient, Right Drugs, Right Dose, Right Time, and Right Way. For after BCMA implementation: In addition to the “Five Rights” of medication another theme emerged, namely, “New Articulation Work” and describes support and problem resolution strategies employed as nurses developed new coordination mechanisms.The implementation of new information technology in the clinical setting can be disruptive to existing patterns of articulation work, or work that coordinates the activities of people across time and space. Implementation teams must familiarize themselves with articulation work and support users in developing new ways of coordinating with colleagues on other shifts and in remote physical spaces.
Novek (2000)353
Design: Mixed methods
N = 124 Health care providers (mostly nurses and pharmacists)
Implementation: 05/1997
Study Start: 02/1998
Study End: 00/0000
DispensingAMDs
Integrated Pharmacy
Long term care (nursing homes)Distrust, resistance, miscommunication, unrealistic expectations, speed and scale of implementation, concurrent changes, inadequate support, and social factors.Nurses were generally distrustful of the AMDs and skeptical that it reduced medication errors.
O’Grady (2006)354
Design: Qualitative
N = 20 Patients
Implementation: 06/2003
Study Start: 04/2003
Study End: 02/2004
Prescribinge-Medication administration system (e-MAR, e-TAR) e-Rx
Integrated Barcoding system, Ward-based automated dispensing system
Unspecified Hospital, 28 on the general surgery ward BedsThemes
  1. pre-EPA views: attitude about paper-based system was generally positive;
  2. anticipated advantages of EPA before its introduction (save time, improve accuracy, and decrease mistakes);
  3. the new system was expected to save time and be efficient (flexibility, comparisons with old system).
  4. Concerns were shown over time, loss of personal touch, and not understanding the system;
  5. advantage for staff when language is not English;
  6. error reduction;
  7. pre- EPA: inherent mistrust for computer systems;
  8. post EPA: perceived disadvantages of the paper- based systems;
  9. post EPA: perceived extra time needed if nursing staff had to check the drugs prescribed on the computer.
Patients generally had a good understanding of how paper-based system had worked and majority had safety concerns with it. Anticipated advantages were mostly about increased efficiency and reduced time. On balance, inpatients seemed neither for nor against EPA.
Patterson (2002)355
Design: Qualitative
N = 33 nurses--7 before BCMA and 26 after
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
AdministeringBCMA
Integrated EHR/EMR system
Acute care/tertiary, Other specialty hospital (rehab, oncology) 784 in the 4 settings Beds Long term care (nursing homes)6 unanticipated side effects were noted:
  1. confusion by automated removal of medications by BCMA;
  2. degraded coordination between nurses and physicians;
  3. dropping activities to reduce workload during busy periods;
  4. increased prioritization of monitored activities during goal conflicts;
  5. decreased ability to deviate from routine sequences;
  6. to reduce workload wristbands were not scanned and medication scanning was delayed.
Unanticipated adverse effects happen and nurses find solutions to cope with workloads.
Patterson (2004)356
Design: Qualitative
N = 46 clinicians working in 6 primary care clinics
Implementation: 00/0000
Study Start: 10/2001
Study End: 10/2002
Monitoring including patient adherence and compliance, PrescribingCDSS/CDS/CCDS/reminders CPOE/POE system
Integrated Hospital information system, Laboratory system, Pharmacy
Ambulatory care7 barriers were identified, some of which were not on the original list:
  1. workload;
  2. time to document;
  3. reminder did not apply;
  4. inapplicability to the situation;
  5. training lacks;
  6. quality of provider- patient interaction;
  7. use of paper forms.
Barriers exist. 17 recommendations were made to improve the situation: 9 related to design, 4 to the organization, and 1 each to team and role design, individual attitudes, patient and situation specific context, and interactions with other systems making issues redundant.
Pirnejad (2008)256
Pirnejad (2009)257
Design: Mixed methods
N = 149 nurses
Implementation: 09/2003
Study Start: 11/2003
Study End: 06/2007
PrescribingCDSS/CDS/CCDS/reminders CPOE/POE system
Integrated EHR/EMR system, Hospital information system
Acute care/tertiary, 1237 Beds Academic256 The coding scheme included differentiation between those features that were considered supportive from features that were considered non- supportive to nurses’ and physicians’ medication work. Many of the paper- based system’s non-supportive features were improved by the CPOE system. And, more useful features such as safety alerts and the possibility for physicians to prescribe electronically from everywhere in the hospital greatly benefited the prescription phase and improved the medication process. Nevertheless, nurses and physicians listed many non-supportive features of the CPOE system as well.257 Workflow impediments from the perspective of physicians and nurses are described. The care providers devised compensatory work-arounds due to interoperabilities in the CPOE system.256 It is clear that moving from the paper based to the CPOE system had positive and negative impacts on nurses’ and physicians’ medication work. In our study, many of the CPOE system’s non- supportive features were listed because the system damaged the synchronization and feedback mechanisms between nurses and physicians.257 The interviews revealed that both nurses and physicians considered the system to be an improvement in their medication work compared to the old paper-based system. They complained about problems in coordination and collaboration. Problems forced them to develop informal rules and work methods to adapt the system in a way that it met their work requirements.
Ruiz (2010)357
Design: Qualitative
N = 19 primary care practitioners
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
PrescribingCDSS/CDS/CCDS/reminders
Integrated EHR/EMR system
Ambulatory careOur data analysis elicited a number of themes, of which six are most relevant to the two areas of our inquiry.
  1. pain as part of growing old;
  2. concerns about using pain medications;
  3. waiting times for pain clinic;
  4. value of ancillary services;
  5. poor training in pain management;
  6. value of CPRS as a support tool.
The findings of this study clearly point to the need for a more systematic and solid understanding of the competencies of primary care practitioners in managing chronic nonmalignant pain in elderly veteran patients. While various types of support have been made available to primary care providers, competency-based training targeted on the elderly population must occur to facilitate the assessment and treatment of such pain. Particular attention must be given to the role of the EMR system as a source of clinical decision support complementary to and reinforcing competency-based training approaches.
Saleem (2005)358
Design: Qualitative
N = 90 Healthcare providers
Implementation: 00/0000
Study Start: 01/2004
Study End: 06/2004
Monitoring including patient adherence and complianceCDSS/CDS/CCDS/reminders
Integrated CPOE/POE system, EHR/EMR system, Laboratory system
Ambulatory careFive barriers, four of which have related subcategories, and four facilitators, organized by three themes:
  1. organizational;
  2. workflow;
  3. computer interface.
Barriers:
  1. Lack of coordination between nurses and providers;
  2. Using the reminders while not with the patient, impairing data acquisition and/or implementation of recommended actions;
  3. Workload;
  4. Lack of CR flexibility;
  5. Poor interface usability
. Facilitators:
  1. Limiting the number of reminders at a site;
  2. Strategic location of the computer workstations;
  3. Integration of reminders into workflow;
  4. Ability to document system problems and receive prompt administrator feedback.
Barriers might explain some of the variability in the use of CRs. These barriers may be difficult to overcome but some strategies may increase user acceptance and therefore the effectiveness of the CRs. These include explicitly assigning responsibility for each CR to nurses or providers, improving visibility of positive results from CRs in the electronic medical record, creating a feedback mechanism about CR use, and limiting the overall number of CRs.
Schoville (2009)359
Design: Qualitative
N = 58 nurses
Implementation: 09/2007
Study Start: 00/0000
Study End: 00/0000
PrescribingCDSS/CDS/CCDS/reminders CPOE/POE system
Integrated EHR/EMR system, Imaging systems, Laboratory system, Pharmacy
Acute care/tertiary, Pediatric stand alone hospital, Pedatric Hospital: 184 beds; Women’s Hospital: 40 Beds AcademicThere were 5 types of work-arounds and artifacts identified by both nursing leadership and staff nurses:
  1. workflow timing of events;
  2. communication changes;
  3. system problems;
  4. learning curve of the CPOE system;
Although CPOE is considered a technical solution to prevent or reduce errors and enhance communication among caregivers, errors could result because of the redundancy in documentation between the paper record and the EMR, systems not interfacing with one another, and multiple screens needing to be viewed to find information about the patient. It was verified that multiple variables affect a successful transition to an electronic order entry system and that workarounds and artifacts were used.
Topps, (2005)272
Design: Mixed methods
N = 313 Healthcare provider
Implementation: 11/2002
Study Start: 05/2002
Study End: 06/2003
AdministeringBCMA
Integrated Billing/administration system Hospital information system, Pharmacy
Pediatric stand alone hospitalQualitativeThemes derived from the pre- survey indicated that medications would be given in a timely manner with less error, but may result in an increase in time with increase in safety along with more reported errors, but fewer errors in administering actual medications (near misses). The surveys collected post-implementation indicated that the staff felt there were fewer medication errors with a smoother administration of medication; however, it was perceived that more time was spent administering medications taking time away from patient care.
Varonen (2008)360
Design: Qualitative
N = 39 physicians
Implementation: 00/0000
Study Start: 10/2005
Study End: 12/2005
PrescribingCDSS/CDS/CCDS/remindersAmbulatory care, AcademicFacilitating factors: Flexibility of the system; (tailoring the selection of topics or patients for reminders and possibility to switch off the system); Reliability; Reliable knowledge base and trust in the developers of the system; Simplicity and ease of use; Concise reminders that facilitate and help work processes; Adequate budgeting; Concise and tailored education for the use of CDSS barriers. In all groups, repeatedly: experience of imperfect health care information systems; Threats to doctor–patient relationship: the computer should not have the leading role in the encounter; Obscured responsibilities; loss of own reasoning and clinical autonomy; Knowledge management: too much information or erroneous information; Resistance towards change; Issues of compatibility and updating, problems with several poorly interacting computer programsFinnish physicians interviewed in this qualitative study had positive attitudes towards implementation of CDSS provided that they have some control over the system. They expected flexibility, individual tailoring and reliability of the CDSS. The high level of computerized practices and wide use of electronic guidelines have paved the way for the CDSS in Finland.
Vaziri (2009)361
Design: Qualitative
N = >30 informaticians, academic clinicians, pharmacists, clinicians with an IT (information technology) interest, human factor/user experience consultants and medical and non- medical commercial IT vendors, as well as members of the National Health Service (NHS) national programme for IT development team
Implementation: 00/000
Study Start: 00/2008
Study End: 00/2008
PrescribingCDSS/CDS/CCDS/reminders e-Rx
Integrated EHR/EMR system
Ambulatory careEnd-users (principally GPs) at the workshop reported that prescribing alerts were more often a source of frustration more than of help. Delegates reported concerns about the current prescribing support prompts, primarily the low specificity of the pop-ups, which were too numerous, often unhelpful and therefore ignored. Information overload may have a negative impact on cognitive performance.Prescribing errors remain a major source of unnecessary morbidity and mortality and current systems do not appear to have significantly reduced this problem; nor has the extensive literature about how to reduce unnecessary alerts been taken into account. We need a new and more rational basis for the selection and presentation of alerts that would help, not hinder, the clinician’s performance.
Vogelsmeier (2008)362
Design: Qualitative
N = 88 nursing home staff
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
Administeringe-Medication administration system (e-MAR, e-TAR)
Integrated EHR/EMR system
Long term care (nursing homes)Workarounds fell into 2 categories, relating to the technology itself and organizational processes. They occurred at new medication order entry, communication with the pharmacy and administration. The technology introduced intentional blocks (safety features such as excessive dose blocking, dual documentation and ADE monitoring) that lead to workarounds. Unintentional blocks leading to workarounds included wireless speed and printing each order on a separate page. Organization process blocks leading to workarounds included double checking of preparation and administration documents and limited resources such as fax machines.As new technologies are introduced, continued monitoring to identify work flow is needed so appropriate changes can be made to address the underlying problems that create work flow blocks ultimately leading to potential workarounds. Additionally, as technology is implemented, organizational processes that will interface with the technology must be carefully re-engineered to reduce the unintended consequences of change.
Weingart (2009)363
Design: Qualitative
N = 25 health care providers
Implementation: 00/0000
Study Start: 00/2007
Study End: 00/2007
Prescribinge-Rx Handheld
Integrated, CDSS/CDS/CCDS/reminders Formulary, Pharmacy
Ambulatory careproblematic features: list management for patients; creating medication lists; poor recording of allergy information; awkward prescription writing leading to work-arounds; problematic alerts leading to alert fatigueFront-line clinicians find many features of the e-Rx system burdensome. The value of e-Rx alerts is diminished by the quantity of irrelevant and inappropriate alerts. e-Rx triggers a variety of clinician behaviors (other than terminating or changing a prescription) that may improve patient safety.
Weir (1994)364
Design: Qualitative
N = 40 hospital staff (admin, physicians, support staff etc)
Implementation: 03/1993
Study Start: 00/0000
Study End: 00/0000
PrescribingCPOE/POE system
Integrated Pharmacy
Unspecified HospitalA survey requesting a list of 6–10 factors facilitating and 6–10 most significant barriers from staff at 3 hospitals with successful implementation and 3 with unsuccessful implementations of CPOE was analyzed using a modified Delphi technique. Fourteen facilitating factors and 14 barriers were identified. Several categories differentiated the two hospital groups. Significantly more people from the successful hospital group reported supportive administration and supportive heads of medical sections; direct involvement of physicians, mandatory implementation, adequate training, and sufficient hardware facilitated success. In terms of barriers, only inadequate hardware and lack of ability to easily do patient transfer and advance admission orders (medical records package) differentiated the two groups and in both cases the item was mentioned more frequently by the successful hospitals.These findings support the notion that the changes involved in instituting a physician order entry system are system wide and involve individual as well as organizational factors.
weir (2007)365
Design: Qualitative
N = 88 interviews
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
PrescribingCPOE/POE system
Integrated Hospital information system
Ambulatory careTasks were related to organization, assigning, determination, educating, scheduling, tracking, overview, correlating documenting, reminding, handing off, prioritizing, accepting, communicating, conforming, and informing. Task components were related to cueing, status, timing, communication, ownership, and linkage. Goals were associated with relevance screening, ensuring accuracy, minimizing memory load, and negotiating responsibility.User creates strategies to learn how to effectively deal with new systems and processes, information overload must be carefully managed, and communication is vital and is often affected by new systems.
Wentzer (2007)366
Design: Qualitative
N = 6 clinicians(2 physicians and 4 nurses)
Implementation: 00/0000
Study Start: 00/0000
Study End: 00/0000
PrescribingCPOE/POE system
Integrated EHR/EMR system
Acute care/tertiaryThe study started with 3 relations (physician and patient interacting, physician and nurse coordination work, and the patients further route and medication path). Themes centered on transformation of the prescription and drug order during physician rounds, transformation of the drug order and dispensing with the CPOE system (user rights, inflexibilities and displacements with the use of CPOE, going back to the paper system, unified and inflexible CPOE medication model), transformation of continuing medication with the system (discharge, withdrawal or discontinuous patient routes, and new tasks and demands on the clinicians with the CPOE system).CPOE system did not meet naive and early expectations. Some adverse effects of the CPOE system were noted.
Zhan (2006)181
Design: Mixed methods
N = 138,922 number of errors/100,000 doses
Implementation: 00/0000
Study Start: 01/2003
Study End: 12/2003
PrescribingCPOE/POE systemUnspecified HospitalSome of the themes taken from the CPOE-related error descriptions included: faulty computer interface, CPOE design failures, especially lack of connection with other parallel systems, inadequacy of decision support and human errors occurring in interactions with the computer.A national, voluntary medication error-reporting database cannot be used to determine the effectiveness of a CPOE system in reducing medication errors because of the variability in the level of underreporting from different institutions. However, it may provide valuable and useful information on the specific types of errors related to CPOE systems.

The HIT system studied is in bold, followed by the systems that it was integrated with.

*

indicates outcomes noted as being the primary outcome by the paper’s authors

Abbreviations: ADE = Adverse Drug Event; AMDs = Automated Medication Dispensing Systems; BCMA = Bar Code Medication Administration ; CCDS = Computerized Clinical Decision Support; CDS = Clinical/Computerized Decision Support ; CDSS = Clinical Decision Support System; CIT = Clinical Information Technology; CPOE = Computerized Provider Order Entry; CR = computer reminder; ED = Emergency Department; EDI = Electronic Data Interchange ; EHR = Electronic Health Record; e-MAR = Electronic Medication Administration Record; EMR = Electronic Medical Records; EPA = Electronic Prescribing and Administration System; e-RX = Electronic Prescribing; e-TAR = Electronic Treatment Authorization Request; GPs = General Practitioners; HIT = Health Information Technology; ICT = Information and Communication Technology; MICU = Medical Intensive Care Unit; MM = Medication Management; N = sample size; OTC = Over the counter; PA = Physician Assistants; PGY1 = First Year Postgraduate; POE = Provider Order Entry

From: Appendix C, Evidence Tables

Cover of Enabling Medication Management Through Health Information Technology
Enabling Medication Management Through Health Information Technology.
Evidence Reports/Technology Assessments, No. 201.
McKibbon KA, Lokker C, Handler SM, et al.

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.