• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of amjpharmedLink to Publisher's site
Am J Pharm Educ. Apr 7, 2009; 73(2): 21.
PMCID: PMC2690899

An Audience Response System Strategy to Improve Student Motivation, Attention, and Feedback

Jeff Cain, EdD, MS,corresponding author Esther P. Black, PhD, and Jürgen Rohr, PhD

Abstract

Objective

To implement an audience response system (ARS) to improve student motivation and attention during lectures and provide immediate feedback to the instructor concerning student understanding of lecture content in a Physiological Chemistry/Molecular Biology course.

Design

Students used ARS devices to respond to strategically placed questions throughout physiological chemistry/molecular biology lectures. The instructor inserted 6 to 7 questions that promoted student/class interactivity into each of several 50-minute lectures to focus students' attention and provide feedback on students' comprehension of material.

Assessment

Ninety-eight percent of first-year pharmacy (P1) students (n = 109) reported that strategically placed ARS questions throughout lectures helped them maintain attention. Reports from an independent focus group indicated that students favored this strategy. Furthermore, ARS feedback helped the instructor gauge student comprehension and adjust lectures accordingly.

Conclusions

Focused, strategically placed ARS questions throughout lectures may help students maintain attention and stay motivated to learn. Feedback from these questions also allows instructors to adapt lectures to address areas of deficiency.

Keywords: audience response system, physiological chemistry course, technology, active learning

INTRODUCTION

The physiological chemistry/molecular biology sequence at the University of Kentucky College of Pharmacy is taught over 2 semesters to approximately 130 first-year (P1) pharmacy students. Physiological Chemistry and Molecular Biology I in the fall semester covers fundamentals such as acid/base theories, basic thermodynamics, and physiological macromolecules (DNA, RNA, proteins, membranes, lipids) and their processes (catalysis, enzyme kinetics, membrane transport, DNA metabolism, gene expression, translation of protein, and pharmacogenomics). Physiological Chemistry and Molecular Biology II content includes fuel metabolism; physiological energy provision; and biosynthesis of fuel metabolism relevant biomolecules such as sugars, fats, amino acids, and peptides; as well as diseases in context with fuel metabolism and the drugs used to treat such diseases. Although the second course is accompanied by and synchronized with a physiology course, students perceived it as difficult. Mean course evaluation ratings for the 2 years prior to the introduction of ARS were 2.4 on a 4-point scale, which were below the College mean. At one point, failure rates for this course were the highest in the College's curriculum. We hypothesized that this may have been due in part to (1) the complex and difficult material, (2) the lack of students' background knowledge in both organic chemistry and general biochemistry, and (3) a lack of motivation to learn, pay attention in class, and/or attend class among the majority of students.

Instructors observed that many students, after only a few lectures, were failing to comprehend much of the lecture material. These didactic lectures were delivered via PowerPoint and distributed as black-and-white handouts. Many students appeared to focus too much on details instead of trying to see the “big picture.” Noise in the classroom grew as students apparently became “lost” and increasingly inattentive during the lectures. These distractions negatively affected the course and class attendance declined by as much as 25%.

Changes in instructional strategies were needed to (1) capture and maintain student attention throughout the lecture, (2) monitor progress in student comprehension of material so that deficiencies could be addressed as they occurred, and (3) improve grades and student satisfaction. Our strategy change for Physiological Chemistry and Molecular Biology II was to use an ARS.

Audience response systems are receiving increased attention as a tool to aid faculty members in engaging and increasing interactivity with students.1 These technology-based systems—also known as “clickers,” student response systems, group response systems, and a bevy of other terms—allow instructors to pose questions to students and receive immediate feedback.2 An ARS consists of visual presentation software, remote input devices (handheld keypads, cell phones, or mobile computers equipped with special software) for students, and a wireless receiver connected to a computer. An instructor presents a question to students who respond by selecting the answer on their input device. The instructor immediately sees the results and can choose whether to reveal answers to students.2 Positive aspects of ARS use, as reported in published research, have included increased learning, interactivity, attendance, and enjoyment; potential drawbacks have included costs to students, technical issues, and decreased lecture coverage.1,3

Pharmacy educators have reported that students in ARS lectures scored significantly higher on examinations, performed better on analytical questions, and found greater satisfaction with their classroom experience than did students from traditional lecture sections.4 Instructors covered less material during ARS lectures, but perceived that the ARS enhanced learning.4 Most participants in an anticoagulation guide session agreed that ARS utilization increased their involvement in the presentation and understanding of its content. 5 Students and instructors in a dual-campus environment found that an ARS increased active learning, revealed student comprehension, and helped instructors identify course concepts that students misunderstood.6 Issues with ARS usage have included faculty members' concerns regarding academic integrity and students' concerns over technical issues that could negatively affect grades. An ARS also has been found useful for recording attendance, engaging students in lectures, and collecting anonymous feedback for presenters in a seminar course.7

Studies by educators in other health professions have shown similar results. ARS usage has been linked to higher learning gains than traditional lectures, perceived as a beneficial learning tool, and identified as a potential tool for improving classroom attention of continuing medical education (CME) participants.8-11 Other cited benefits of ARS utilization include perceived learning gains by students and higher CME lecture ratings.12,13

The success of an ARS, as with any other instructional technology, relies primarily on the instructional strategy used.14 How the technology is used is more important than the technology itself. Further research has been solicited for ARS strategies addressing specific outcomes such as increasing student and instructor interaction, student satisfaction with instruction, classroom participation, and learning.1,2,5,6 This study adds to that needed literature base.

This manuscript documents an instructional strategy using an ARS to maintain student attention during lectures and provide instructors with formative feedback on students' comprehension of lecture content. We describe the instructional strategy, the methods for assessing the strategy's success, and the results of those assessment methods, and provide a brief discussion on the widespread potential of this strategy.

DESIGN

A TurningPoint (Turning Technologies, Youngstown, OH) ARS was used during each of the first 28 lectures in Physiological Chemistry and Molecular Biology II. The instructor for the last 14 lectures of the semester elected not to use an ARS. As part of this second course in the biochemistry sequence, students were required to purchase handheld responders, available in the university bookstore for about $35 each. The instructional strategy in the lecture portion of the course was to use an ARS to pose questions to students on key concepts presented in the lectures.

The instructor inserted 6 to 7 ARS questions into the delivery of the lecture. As an incentive, students could earn bonus points based on the percentage of overall correct responses. Students could use their books and notes to answer questions, but were under a time limit varying between 1 and 2 minutes per question. The instructor adjusted the lecture according to how students responded to the questions, which were designed to test both fundamental knowledge and complicated concepts. Two types of questions were asked in equal proportions, silver and gold. The students had to answer “silver” questions on their own, with no help from classmates, but could collaborate on “gold” questions. To earn bonus points, 80% of the class had to answer a silver question correctly, 90% a gold question. All participating students received bonus points provided the class as a whole met or exceeded those thresholds. Thirteen bonus points were possible through participation in the ARS sessions. Available bonus points (approximately 5% of the grade) earned throughout the course equaled the amount available when the course was offered in previous semesters. Therefore, with regard to bonus points, students in this course did not have an advantage over students from previous semesters.

The instructor immediately discussed the results of each ARS question with the class. Topic discussion time was only marginally longer than in previous semesters. Immediate feedback from students via the ARS helped the instructor allocate more time to those topics students did not understand. Time spent on discussions varied according to topic complexity.

ARS was added based on studies showing that changing how a lecture is delivered can stimulate students' attention.15 Posing periodic questions via an ARS changes the typical lecture routine by shifting focus from the instructor and providing a brief break in content presentation, compelling students to interact with the material and provide formative feedback to both instructor and students.16 In this study, the instructor used results from ARS questions to gauge student comprehension, adjust the direction of the lecture accordingly, and spend more time on topics that students were having difficulty understanding. Figures Figures11 and and22 contain screen captures of typical ARS questions and corresponding results. This strategy served a dual purpose of maintaining student attention and providing the instructor with immediate feedback on student understanding.

Figure 1
Example screen shot of audience response system (ARS) “silver” question and pharmacy students' responses.
Figure 2
Example screen shot of audience response system (ARS) “gold” question and pharmacy students' responses.

Assessment Methods

This instructional strategy was assessed through a quadrangular approach consisting of an in-class student questionnaire, student-liaison focus group comments, course/instructor evaluations, and grade comparisons from previous semesters. Institutional Review Board exempt approval was obtained in order to publish results of this formative curricular assessment.

A 12-item questionnaire was delivered near the end of the course to students via a TurningPoint ARS. Eight of those questions related to ARS usage and pertained to students' preferences and perceptions of the instructional strategy.

Feedback from a P1 student-liaison focus group supplemented the questionnaire results. The liaison group consisted of 12 randomly chosen students who met twice per semester to review the effectiveness of their courses and provide formative feedback to instructors on “what is working well” and “what is hindering student learning.” An independent facilitator directed the meetings and provided written session summaries to course directors.

End-of-semester course and instructor evaluations were also analyzed to help determine whether the ARS strategy had any effect on student perceptions of the course and instructor. We reviewed 3 years of data from Physiological Chemistry and Molecular Biology II, and compared course evaluations with those from the previous semester's Physiological Chemistry and Molecular Biology I course. We also compared course grades from the semester with those from previous years, as well as with grades in the Physiological Chemistry and Molecular Biology I course taught the previous semester.

ASSESSMENT

This instructional strategy was assessed through (1) student questionnaires, (2) student-liaison focus group reports, (3) course/instructor evaluations, and (4) grade comparisons. Instructor self-reflections were also considered to provide further clarification of outcomes associated with this strategy.

Student Questionnaire Results

Students (n = 111) in Physiological Chemistry and Molecular Biology II completed an ARS-administered questionnaire related to aspects of ARS usage in the course. Sixteen students did not complete the questionnaire, resulting in an 87% completion rate. Table Table11 contains a detailed breakdown of the most pertinent items on the questionnaire. Overall, students commented favorably on ARS use in the course. All respondents (n = 111) preferred ARS-integrated lectures to those that did not use an ARS. Most students (93%, n = 103) preferred the course with an ARS component over the first course in the series, which had no ARS component. As a reason for this preference, 74% (n = 81) cited the ARS strategy as a major contributing factor, and an additional 9% (n = 10) named the ARS as the sole factor.

Table 1
Pharmacy Students' Responses to a Questionnaire Regarding Use of an Audience Response System in a Physiological Chemistry/Molecular Biology Course (N = 111)

Almost every student (n = 110) responded that ARS usage periodically throughout a class lecture helped them maintain attention, and 98% (n = 109) felt discussions stemming from the ARS question-and-answer method were beneficial. A majority of students (n = 85) also agreed that the ARS question-and-answer method should be incorporated into all course lectures. In terms of the strategy of extra credit and ARS usage, 42% (n = 47) preferred the ARS regardless of extra credit, while 58% (n = 64) preferred the ARS only when extra credit was offered. Eighty-two percent (n = 91) of students cited the cost (approximately $35 per student) of the handheld responders as the only significant negative aspect of using the ARS.

Student-Liaison Group Results

Student-liaison focus group reports also contained favorable remarks on the ARS strategy. A facilitator provided a summary of discussions regarding the different courses and noted that the ARS strategy used in the Physiological Chemistry and Molecular Biology II course was well received. Students in the liaison group commented on how the strategy benefitted them in learning the lecture content.

Course Evaluation Results

Following grade submission, students (n = 127) in the Physiological Chemistry and Molecular Biology II class evaluated the course and instructors through CourseEval (Academic Management Systems, Amherst, NY), an online survey/evaluation tool. The evaluation results were much more positive than in previous years before the ARS strategy was implemented. The overall course rating was 3.3 on a 4-point scale, compared to 2.4 in the 2 previous years. Of 96 positive free-text comments about the course, 48 mentioned the ARS.

On faculty evaluations, the instructor who taught most of the classes and used the ARS strategy received an overall rating of 3.5 on a 4-point scale, compared with ratings of 2.6 and 2.7 the previous years. Of 114 positive free-text comments on the instructor, 69 referred to the ARS strategy as a positive aspect of the teaching methods. Table Table22 contains a selection of course/instructor evaluation comments on the ARS strategy.

Table 2
Comments from 2008 Course/Instructor Evaluations for a Physiological Chemistry II Course in Which an Audience Response System Was Used

Evaluation comparisons with Physiological Chemistry and Molecular Biology I were favorable. Although the primary instructors in each of the courses received equivalent instructor ratings, Physiological Chemistry and Molecular Biology II (which used the ARS strategy) received the higher course rating (3.3 compared to 2.8). In previous years when an ARS was not used, the courses received comparable ratings between 2.4 and 2.8.

Grade Comparisons

An analysis of Physiological Chemistry and Molecular Biology II course grades over the past 3 years indicated that students enrolled in the 2008 course (which used the ARS strategy) had the highest final mean percentage grade. Table Table33 contains a detailed comparison of mean course grades and course letter grade allocations over the past 3 years. Results of a one-way ANOVA, F(2,381) =25.26; p < 0.001, revealed a significant difference in mean course grades over those 3 years. A Games-Howell post-hoc analysis showed that mean course grades in 2008 (which used an ARS) were significantly higher from 2007 (p < 0.05) and from 2006 (p < 0.001). Course examinations, grading structure, and availability of bonus points were equivalent across all years. The significant difference in course grades strengthens the case for the effectiveness of this strategy. Furthermore, each student's grade was compared with his or her grade from Physiological Chemistry and Molecular Biology I taught the previous semester. A paired samples t test, t(125) = 13.17, p < 0.001, revealed significantly higher final grades in the second course, Physiological Chemistry and Molecular Biology II (which used an ARS). Table Table44 contains a comparison of letter grade allocations for Physiological Chemistry and Molecular Biology I and II.

Table 3
Final Grades for Students Completing a Physiological Chemistry and Molecular Biology II
Table 4
Final Grades for Students Completing Physiological Chemistry and Molecular Biology I Without the Use of an ARS and Physiological Chemistry and Molecular Biology II Using an ARS

The limitation of the grades analysis was that this was not a controlled experimental study and no definite conclusion can be drawn on the ARS strategy effects. However, when combined with questionnaire and focus group results, the higher grades in this semester's course suggest that the instructional strategy contributed to a positive effect on student grades.

Instructor Self-Reflections

From the instructor viewpoint, the ARS helped to optimize the pace of lectures and clarify points that students misunderstood. The strategically placed ARS questions gave students a break from the lecture delivery and time to reflect on lecture content. If student answers revealed lack of comprehension on a certain topic or subtopic, the instructor could address the difficulties immediately through classroom discussion. Furthermore, class time was not wasted on topics or concepts already understood by more than 90% of the class. Classroom attendance near the end of the ARS modules ranged between 98% and 100%, marking an improvement from previous semesters when approximately 25% of the students were absent during lectures. Relationships between students and teachers seemed to improve because of the ARS strategy, as reflected by students' answers on the questionnaire, comments from the liaison groups, and end-of-semester course evaluations. The instructor was pleased with the ARS strategy and its resulting improvements to classroom attention, attendance, learning, course grades, and end-of-semester course and instructor evaluations.

Resources Utilized

The successful implementation of this teaching method required collaboration and coordination in the College. Technical staff members were needed to install the ARS software on classroom and instructor computers. A TurningPoint ARS receiver was purchased for $99 and installed on the classroom instructor computer. The College's Academic Affairs Office was responsible for working with the university bookstore to make handheld response units available to students. Minimal instructor time was needed to develop questions within TurningPoint and analyze results for grading purposes. During the lectures, approximately 2-3 minutes per question were allocated for students to answer ARS questions. Time for discussion varied based on feedback. An educational technology consultant provided training, support, and pedagogical advice to the instructor throughout implementation of the ARS.

DISCUSSION

This study documented the success of using a specific ARS strategy to increase student attention during lectures and provide the instructor with feedback. The study provides further information on specific outcomes and student perception of/satisfaction with using an ARS in the classroom, as proposed by several researchers.1,5,6,17 In particular, responses on the student questionnaire indicated that the ARS strategy helped maintain their attention and provided a perceived benefit to learning. Course/instructor evaluation scores and positive comments attributed to the ARS strategy show that this instructional strategy was well received by students. The positive reception is consistent with prior research on the use of an ARS with continuing medical education participants.13

A main reason for adopting this instructional strategy was to increase student motivation to attend class and actively engage in the lectures. Instructor behavior (ie, effective instructional strategies) has been cited as a determinant of student motivation to attend class.18 A cause-effect relationship cannot be implied from this study; however, questionnaire results, course/instructor ratings, student-liaison reports, and attendance records point to a positive acceptance of this strategy as a way to engage students and possibly motivate them to attend class.

Decreased lecture coverage due to time needed to collect answers is one potential negative aspect of ARS use.3,11 While this study did not directly measure classroom time spent with the ARS, the instructor noted that this strategy had only a nominal effect on lecture time. In previous semesters without the ARS, the instructor still allocated time for questions and discussions on difficult topics. The ARS provided feedback to both the instructor and students, allowing focused discussion only on those topics that students misunderstood the most. Although using the ARS required a few more minutes of lecture presentation, time spent discussing the results may have been more effective at addressing students' learning needs.

Additional costs to students for purchase of the handheld responders (clickers) has been cited as a drawback of adopting an ARS for the classroom.3 Students in this study reported that the cost of the responders was the most negative aspect of ARS use in the course. However, as the ARS use increases throughout the College and students are able to use the responders in subsequent courses, we believe students will recognize its value.

Austin and Gregory reported that basic science courses are receiving less emphasis in clinical programs, and as a result, pharmacy students may not appreciate their importance/relevance to all subsequent courses and ultimately to pharmacy practice.19 Clinical courses have introduced innovative pedagogies to motivate students and animate learning while most science courses remain strongly didactic, uninteresting, and of poor quality.19 Although these challenges may exist for basic science courses, we found that our ARS strategy improved student motivation to participate and excel. This further illustrates the usefulness of this strategy.

While using the ARS strategy in Physiological Chemistry and Molecular Biology II, classroom attendance improved, course grades increased, and course/instructor ratings improved. One unanswered question from this study is “Did the ARS strategy directly or indirectly affect the final grades and improve course/instructor ratings?” It is unclear whether the ARS strategy in itself improved student learning and hence improved grades and course/instructor ratings. Another possible explanation is that the strategy motivated students to attend class, which ultimately helped them perform better in the course and subsequently provide better course/instructor ratings. Further studies are needed to establish the direct and indirect effects of the ARS strategy on student grades and course/instructor evaluations.

Positive feedback about using an ARS in the Physiological Chemistry and Molecular Biology II course in spring 2008 encouraged the authors to integrate this strategy into all sections of the course, as well as into other pharmacy lectures. Several courses are team-taught, which will require further coordination (eg, general request to future incoming students to purchase ARS responders) and a more general acceptance by other teaching colleagues of the ARS. Future revisions to this strategy include slight modifications such as (1) allowing individual extra credit for a small percentage of the ARS questions, as preferred by many students, and (2) designing some of the ARS questions as individual response, followed by peer discussion and reassessment.20 Overall, this has been a valuable innovation for the course, and we will expand this strategy in the future.

SUMMARY

An audience response system was added to a Physiological Chemistry/Molecular Biology Course to improve student motivation and attention during lectures and provide immediate feedback to the instructor concerning student understanding of lecture content. According to student feedback, asking periodic questions throughout lectures via the ARS achieved its goal. Students also indicated that this strategy helped them learn lecture material more effectively. A secondary benefit of using the ARS was real-time, accurate feedback to the instructor with regard to student comprehension of lecture material. Student grades for the course were significantly higher than grades for previous courses in which the ARS was not used. Finally, course and instructor evaluations improved from previous semesters. We will continue to use, evaluate, and improve upon this instructional strategy.

REFERENCES

1. Fies C. Classroom response systems: A review of the literature. J Sci Educ Technol. 2006;15:101–9.
2. Cain J, Robinson E. A primer on audience response systems: Current applications and future considerations. Am J Pharm Educ. 2008;72 Article 77. [PMC free article] [PubMed]
3. Caldwell JE. Clickers in the large classroom: Current research and best-practice tips. CBE Life Sci Educ. 2007;6:9–20. [PMC free article] [PubMed]
4. Slain D, Abate M, Hodges BM, Stamatakis MK, Wolak S. An interactive response system to promote active learning in the doctor of pharmacy curriculum. Am J Pharm Educ. 2004;68 Article 117.
5. Trapskin PJ, Smith KM, Armitstead JA, Davis GA. Use of an audience response system to introduce an anticoagulation guide to physicians, pharmacists, and pharmacy students. Am J Pharm Educ. 2005;69 Article 28.
6. Medina MS, Median PJ, Wanzer DS, Wilson JE, Nelson E, Britton ML. Use of an audience response system (ARS) in a dual-campus classroom environment. Am J Pharm Educ. 2008;72 Article 38. [PMC free article] [PubMed]
7. Poirier TI. A seminar course in contemporary pharmacy issues. Am J Pharm Educ. 2008;72 Article 30. [PMC free article] [PubMed]
8. Schackow TE, Chavez M, Loya L, Friedman M. Audience response system: Effect on learning in family medicine residents. Fam Med. 2004;36:496–504. [PubMed]
9. Pradhan A, Sparano D, Ananth CV. The influence of an audience response system on knowledge retention: An application to resident education. Am J Obstet Gynecol. 2005;193:1827–30. [PubMed]
10. Johnson JT. Creating learner-centered classrooms: Use of an audience response system in pediatric dentistry education. J Dent Educ. 2005;69:378–81. [PubMed]
11. Miller RG, Ashar BH, Getz KJ. Evaluation of an audience response system for continuing education of health professionals. J Cont Educ Health Prof. 2003;23:109–15. [PubMed]
12. Uhari M, Renko M, Soini H. Experiences of using an interactive audience response system in lectures. BMC Medical Education. 2003;3:12. [PMC free article] [PubMed]
13. Copeland HL, Stoller JK, Hewson M. Making the continuing medical education lecture effective. J Cont Educ Health Prof. 1998;18:227–34.
14. Clark RE. Media will never influence learning. Educ Technol Res Dev. 1994;42:21–9.
15. Middendorf J, Kalish A. The “change-up” in lectures. Natl Teach Learn Forum. 1996;5:1–5.
16. Draper SW, Brown MI. Increasing interactivity in lectures using an electronic voting system. J Comput Assist Learn. 2004;20:81–94.
17. Kennedy GE, Cutts QI. The association between students' use of an electronic voting system and their learning outcomes. J Comput Assist Learn. 2005;21:260–8.
18. Fjortoft N. Students' motivations for class attendance. Am J Pharm Educ. 2005;69 Article 15.
19. Austin Z, Gregory PAM. Professional students' perception of the value, role and impact of science in clinical education. Clin Pharmacol Ther. 2007;82:615–20. [PubMed]
20. Crouch CH, Mazur E. Peer instruction: Ten years of experience and results. Am J Phys. 2001;69:970–7.

Articles from American Journal of Pharmaceutical Education are provided here courtesy of American Association of Colleges of Pharmacy
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...