• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of procamiaLink to Publisher's site
AMIA Annu Symp Proc. 2005; 2005: 946.
PMCID: PMC1560481

Using Technology to Teach Technology: Design and Evaluation of Bilingual Online Physician Education About Electronic Medical Records


The “EMR Tutorial” is designed to be a bilingual online physician education environment about electronic medical records. After iterative assessment and redesign, the tutorial was tested in two groups: U.S. physicians and Mexican medical students. Split-plot ANOVA revealed significantly different pre-test scores in the two groups, significant cognitive gains for the two groups overall, and no significant difference in the gains made by the two groups. Users rated the module positively on a satisfaction questionnaire.

Keywords: Computer assisted instruction, medical education, acculturation, language


Physicians need effective training in order to purchase, implement, and use electronic medical records (EMR) in their practices. Web-based learning environments are accessible and pedagogically sound whether or not local medical faculty has expertise in this topic (1). We present a description of the design and evaluation of an introductory tutorial about EMR.

Materials and Methods

Learning objectives were based on published literature about EMR; from these, tutorial content was developed. The resulting online module covers one hour of stand-alone educational material, available in English or in Spanish; translation accuracy was verified by native speakers.

Three reviewers performed heuristic evaluation (2) of the tutorial. We also solicited review from domain experts in EMR, adult education, and software usability. Pilot English and Spanish-speaking users offered feedback about content, navigation, and satisfaction. This feedback was incorporated into the final design.

Cognitive gain was measured with different, equivalent pre- and post-tests based on learning objectives. A user satisfaction survey was adapted from other published online learning evaluations (3).

Two groups of subjects completed the tutorial: family medicine faculty from a US medical school, and students from a Mexican medical school. Data were compared to assess significant differences in learning effectiveness and affective response to the material, using univariate or split-plot analysis of variance.


126 subjects – 100 Spanish, 26 English – completed the tutorial. The mean total score on the satisfaction survey was significantly (p=.000) higher than “the typical lecture”, where a higher score indicated more satisfaction. Analysis of variance of usability scores compared to the language version demonstrated that there was no significant difference in usability assessment between the Spanish and English version users (F = 3.627, p = 0.059).

Split-plot design analysis of variance to evaluate the impact of language version on cognitive outcomes demonstrated significant improvement from pretest to posttest for the two groups as a whole (F = 11.625, p = .001), significant differences in the overall performance of the two group (F = 52.965, p = .000), but no significant difference in the gains made by each group (F = .001, p = .980).


Evaluation of the online EMR tutorial reveals that the tutorial was very acceptable to a broad variety of students, and achieved significant learning gains,whose magnitude was not significantly affected by language.


1. Adler MD, Johnson KB. Quantifying the literature of computer-aided instruction in medical education. Acad Med. 2000;75(10):1025–8. [PubMed]
2. Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. J Biomed Inform. 2003;36(1-2):23–30. [PubMed]
3. Johnson CW, Grover PA. Hypertutor therapy for interactive instruction. Educational Technology. 1993;33(1):5–16.
4. Albanese MA. Challenges in using rater judgements in medical education. Journal of Evaluation in Clinical Practice. 2000;6(3):305–19. [PubMed]

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association
PubReader format: click here to try


Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...


  • PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...