• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of procamiaLink to Publisher's site
AMIA Annu Symp Proc. 2003; 2003: 328–330.
PMCID: PMC1480293

Randomized Comparisons among Health Informatics Students Identify Hypertutorial Features as Improving Web-Based Instruction

Craig W. Johnson, Ph.D., Yan Xing, M.D., and Lan Yang, B.S.

Abstract

Hypertutorials optimize five features – presentation, learner control, practice, feedback, and elaborative learning resources. Previous research showed graduate students significantly and overwhelmingly preferred Web-based hypertutorials to conventional “Book-on-the-Web” statistics or research design lessons. The current report shows that the source of hypertutorials’ superiority in student evaluations of instruction lies in their hypertutorial features. Randomized comparisons between the two methodologies were conducted in two successive iterations of a graduate level health informatics research design and evaluation course. The two versions contained the same text and graphics, but differed in the presence or absence of hypertutorial features: Elaborative learning resources, practice, feedback, and amount of learner control. Students gave high evaluations to both Web-based methodologies, but consistently rated the hypertutorial lessons as superior. Significant differences localized in the hypertutorial subscale that measured student responses to hypertutorial features.

INTRODUCTION

Learning environments that operationalize the Hypertutorial Model optimally implement five features – presentation, learner control, practice, feedback, and elaborative learning resources. Previous research1 throughout the last century consistently demonstrates that each feature, by itself, typically facilitates learning. In previous randomized studies hypertutorial statistics or research design lessons using sample sizes similar to those of the present study demonstrated significant superiority on a variety of measures including achievement test scores and student evaluations of instruction2,3,4,5. Previous research also showed that graduate students significantly and overwhelmingly preferred Web-based hypertutorials to conventional “Book-on-the-Web” statistics or research design lessons2,5. In the current report students identify hypertutorial features as the source of hypertutorial superiority in student evaluations of instruction.

METHODS

Subjects.

Twenty-four health informatics graduate students enrolled in two successive iterations of a graduate level health Web-based informatics research design and evaluation course at the University of Texas Health Science Center at Houston participated in the study. Students were given no training in computer or Web use. Students attended a brief introductory orientation class session where students were informed how to access the course materials including the course homepage and its associated hyperlinks, the syllabus and the Web-based tutorial lessons. All 24 students registered for the course volunteered to participate in the research. Three students were deleted from the data analysis because of missing data.

Procedure.

In each course, students came to the regularly scheduled classes and used either Netscape version 3.0 or above or Microsoft Internet Explorer version 4.0 or above to access, study and respond throughout the semesters’ class sessions to the Web-based tutorial lessons according to the schedule recommended in the course syllabus. All lessons were completed by all students by the end of the course. At the end of each lesson, students responded to a five-item quiz and a 22- item end-of-lesson student evaluation of instructional effectiveness.

Student responses to embedded practice exercises, quiz, and end-of-lesson student evaluation items were stored immediately as they were entered, using a small, client-side, JDBC applet, into a Microsoft SQL Server database and also logged redundantly via automated E-mail from within the browser. No Java servlets or CGI code were employed.

Instruments.

A five-item cognitive achievement lesson quiz and a 22- item end-of-lesson student evaluation of instructional effectiveness appeared at the end of each lesson. The lesson quizzes were individually designed to assess achievement of each lesson’s objectives. On the other hand, the same 22 items were used for all 18 lessons’ end-of-lesson student evaluation of instructional effectiveness.

Included among the 22 student evaluation of instruction items were 15, 5-point Likert type items distributed into 3 subscales. The 3 subscales assessed specific instructional, hypertutorial and outcome-oriented objectives, respectively. On most of the 15 5-point Likert type items the scale descriptors ranged from “Much better than the typical lecture”, scored as “5”, to “Much worse than the typical lecture” scored as “1”, with “Same as the typical lecture” as the middle rating, scored as “3”.

Items forming the 3 subscales were generated from a data source by objectives matrix using a standard top-down, goal/task analysis methodology for achieving content validity: First, the three subscale goals were identified. Second specific behavioral objectives were identified for each subscale goal. Third, items were written to measure degree of achievement of each objective for each subscale. The use of a data source by objectives matrix together with the top-down methodology assures that the instrument contains a representative sampling of the objectives, resulting in high content validity. Coefficient Alpha reliabilities of the subscales were 0.96, 0.81 and 0.94, respectively.

Lessons.

Each of the 18 tutorial lessons was written entirely in a combination of HTML and JavaScript and each referenced one and the same small JDBC applet via an “applet” tag. Each HTML lesson document delivered instruction, included embedded forms, and included JavaScript functions that implemented feedback, collected, scored, formatted and transmit via E-mail all student data (including timing information). All of these processes were implemented on the client end without CGI programs or servlets. Data was redundantly stored by the Java JDBC applet whose methods were called from JavaScript on the client to transmit the data to a Microsoft SQL Server database on the server.

Design.

Two different versions (Hypertutorial vs. Conventional) of each lesson were created that contained identical presentation text and graphics, but differed in the presence or absence of hypertutorial features. For the Hypertutorial versions of the lessons, hyperlinks operationalized the elaborative learning resources feature while increasing learner control, as well. Embedded practice exercises with immediate feedback (correct, incorrect) operationalized the practice and feedback features while also increasing learner control. Together, the hyperlinks and embedded practice exercises with immediate feedback operationalized the only differences between the Hypertutorial and Conventional lessons. Both versions contained identical presentation text and graphics, but differed only in the presence or absence of elaborative learning resources, practice, feedback, and amount of learner control. Hypertutorial lessons contained the hypertutorial features while conventional lessons did not. Conventional lessons could also be described as being like a “Book on the Web”.

Students taking each course were randomly assigned to two groups at the beginning of their participation in the study. At the beginning of each course, students in one group started the course using the Hypertutorial (H) version of Lesson One. Students in the other group began with the conventional (C) version of Lesson One. As each course advanced, and monitored by the instructor, lesson versions then alternated for each student on a lesson-by-lesson basis as students progressed through each of the 18 successive Web-based lessons during weekly formal class sessions.

Because no between courses significant differences existed in student responses to any of the 3 subscales, data from the 2 courses was pooled resulting in a research design that presents three factors (See Table 1): One between-subjects factor (Group - 2 levels; HCHC…HC vs. CHCH…CH) and two within-subjects factors (Odd vs. Even Numbered Lessons – 2 levels) and (Lesson Number – 18 levels). As can be seen from Table 1, the 18 levels of Lesson Number are nested within the two levels of the Odd vs. Even Numbered Lesson factor. The three scores for the three student evaluation of instruction subscales constituted the dependent variables tested for the study.

Table 1
Design diagram, Means & SD’s of hypertutorial subscale.

Data analysis.

Repeated measures data analysis employed the SPSS 11.01 MANOVA procedure for the three factor design having one between-subjects factor (Group) and two within-subjects factors, with the first within-subjects factor (Odd vs. Even Numbered Lessons) having the second within-subjects factor (Lesson Number) nested within its two levels. The three subscales (instructional, hypertutorial, outcome -oriented) constituted the dependent variables for the repeated measures analysis. As noted earlier, 3 students were deleted from the analysis because of missing data. Regression-based estimates replaced any remaining blanks or erroneous responses for each of the remaining 21 students. There were no significant differences between the two iterations of the course for any test. Therefore, data from both groups was pooled for the final analysis.

RESULTS

Student evaluations of the Web-based instruction were high overall, rating in the range of from “Better than the typical lecture” to the “Much better than the typical lecture” for both methodologies. But, students consistently rated hypertutorial lessons superior to conventional “Book on the Web” lessons.

A resulting significant Group by Oddeven interaction (F1,19 = 14.17, p = 0.001) was localized in the hypertutorial subscale that measured student responses to learner control, practice and feedback, and elaborative learning resources (See Table 1 and also Figure 1, below). There were no other significant effects.

Fig. 1
Group by Oddeven interaction on hypertutorial subscale

Hypertutorials significantly exceeded conventional lessons among the Group 2 students, who started with the conventional version of the lessons (F1,8 = 12.10, p = 0.008). And, hypertutorials exceeded conventional lessons among the Group 1 students, who started with the hypertutorial version of the lessons (F1,11 = 4.02, p = 0.070).

DISCUSSION

Health Informatics graduate students enrolled in two successive offerings of a Research and Evaluation for Health Informatics course rated Web-based hypertutorial lessons superior to highly-rated Web-based conventional “Book-on-the-Web” lessons in Web-based student evaluations of instruction. This superiority was localized to one of the three evaluation subscales (instructional, hypertutorial, outcome-oriented) of the Web-based student evaluation of instruction instrument – namely the hypertutorial subscale. Students gave high ratings to both Web-based hypertutorial lessons and Web-based conventional “Book-on-the-Web” lessons in Web-based student evaluations of the two instructional methodologies. Ratings averaged in the range of from “Better than the typical lecture” to the “Much better than the typical lecture” for both methodologies. While graduate students in the School of Health Information Sciences evaluated hypertutorial and conventional lessons similarly positively in terms of general instructional and outcome characteristics, they evaluated significantly more positively the learner control, practice exercises, feedback and elaborative learning characteristics of hypertutorial lessons. These results pinpoint and, taken together with results from previous studies, further substantiate the value of employing the Hypertutorial Model in the design, development and implementation of Web-based learning environments.

REFERENCES

1. Johnson CW, Grover PA. Hypertutor therapy for interactive instruction. Educational Technology. 1993;33(1):5–16.
2. Johnson, CW, Health Informatics Research and Evaluation Design Students Overwhelmingly Prefer Hypertutorial Web-Based Instruction in Randomized Comparison, Proceedings of Annual Conference of the American Medical Informatics Association, Los Angeles, 2000.
3. Johnson, CW. Web-browser implements unobtrusive randomized comparisons of instruction in Health Informatics classroom, Proceedings of the Annual Conference of the American Medical Informatics Association, Washington, D.C., 1999.
4. Johnson, CW, Oser, G. & Abedor, AJ. Web browser as medical educator/researcher using HTML & JavaScript [D004915]. Proceedings of the Annual Conference of the American Medical Informatics Association, Orlando, 1998.
5. Johnson, CW. Dental postgraduates prefer hypertutorial statistics lessons in randomized Web course comparisons. Proceedings of the Annual Conference of the American Medical Informatics Association, Washington, D.C., 2001.

Articles from AMIA Annual Symposium Proceedings are provided here courtesy of American Medical Informatics Association
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...