![Page 1: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/1.jpg)
Audience Response Systems: Transforming your Lecture
Hall into a Platform for InteractivityAlice Chuang, MD
University of North Carolina – Chapel Hill
Francis S. Nuthalapaty, MDUniversity of South Carolina - Greenville
2009 APGO Faculty Development Seminar
![Page 2: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/2.jpg)
Objectives
• Demonstrate ARS in action*
• Discuss the rational for using Audience
Response Systems (ARS) in medical education
• Outline ARS question types and best practices
• Outline practical concerns regarding use of ARS
• Describe available ARS systems
![Page 3: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/3.jpg)
Audience Response Systems
• Definition: Electronic tool which allows lecturer to poll learners with immediate feedback.
• Synonyms:– Classroom response system– Student response system– Personal response system
![Page 4: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/4.jpg)
Instructions for Clickers
• To vote: Press 1/A, 2/B etc…• Vote Status Light:
– Green if your vote was recorded– Orange if not
• To change your vote – press another key• Clickers work like an RF remote• Receiver is plugged into USB port
![Page 5: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/5.jpg)
Are you here today?
Yes N
o
0%
100%1. Yes2. No
![Page 6: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/6.jpg)
I am a…
11%
22%
0%
11%
56% 1. Clerkship director
2. Program director
3. Chair
4. Clerkship coordinator
5. Other
![Page 7: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/7.jpg)
What is your role in education?
Tea
chin
g stu
dents
in th
e...
Tea
chin
g stu
dents
in th
e...
Adm
inis
tratio
n (Dea
n, C...
Oth
er
0%
44%
11%
44%1. Teaching students in the pre-clinical curriculum
2. Teaching students in the clerkship
3. Administration (Dean, Clerkship Director)
4. Other
![Page 8: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/8.jpg)
How often do you currently use ARS in your teaching?
70%
30%0%0%
Never used ARS before
Rarely (<25% of my lectures)
Often (>50% of my lectures)
Very Often (>75% of my lectures)
1. Never used ARS before
2. Rarely (<25% of my lectures)
3. Often (>50% of my lectures)
4. Very Often (>75% of my lectures)
![Page 9: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/9.jpg)
Roles of ARS
• Restarting the attention clock– Average human attention span: 20
minutes– Recall of information decreases after
15-20 minutes– At maximum 20-25% of material
presented in standard lecture is retained
![Page 10: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/10.jpg)
0
10
20
30
40
50
60
70
80
90
0 15 30 45 60
Time (min)
Att
enti
on p
aid
to lectu
rer
(%)
Adult Learner(i.e. tiredoverworkedmedicalstudent)
Has renewed sense of interest in topic at hand
Is finding lecture material somewhat
mundane, starts previewing following lecture for interesting
slides
Listens intently, hoping to catch key
summary points
Settling into lecture, previewing
lecture slides
![Page 11: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/11.jpg)
Roles for ARS
• To increase interactions– To assess learner’s attitudes– To start or focus discussions– To vote after a debate or panel discussion
• To assess preparation and ensure accountability– Questions about reading or homework
![Page 12: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/12.jpg)
Roles for ARS
• To engage learner throughout a lecture– To assess learner’s understanding and
misunderstanding of topics during a lecture– To find out more about student attitudes on a topic– To assess need for more explanation or detail
• For quizzes or tests– paying attention– taking good notes– preparing for class or labs– keeping up with homework– actively thinking
![Page 13: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/13.jpg)
Prevalence of ARS
• Used ubiquitously in many disciplines…mathematics, philosophy, psychology, economics
• Used in many age groups…K-12, elementary, undergraduate, graduate
![Page 14: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/14.jpg)
History
• Pradhan et al• RCT of ARS v. standard lecture format on
the topic of contraception• 17 ob/gyn residents at UMDNJ• Compared pretest/posttest (6 weeks
later) scores– ARS: 21% improvement– Standard lecture: 2% improvement– P = 0.18
![Page 15: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/15.jpg)
History
• Nayak and Erinjeri• 27 medical students who were instructed to
give a presentation at the end of course– From learners viewpoint
• Increased self-confidence with understanding material and verbally answering questions
• No differences in end of course exam scores
– From presenters viewpoint• Made them more comfortable that they were conveying
information effectively• Prepared them for future presentations
![Page 16: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/16.jpg)
Pros…based on the literature
• Instructor benefits– Immediate feedback on learners understanding,
identify gaps– Improved retention of knowledge
• Technical issues– Decreased error in collecting data because of
electronic transmission– Anonymous
• Can increase attendance– in one study only if participation was worth at least
10% of grade– Can reduce attrition through the duration of a
course (4% vs 8-12%)– Particularly if class was midday
![Page 17: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/17.jpg)
Pros…based on the literature
• Increased learner satisfaction by improving interactivity and by engaging learner– Student gave instructors and “learning
environment” more positive ratings• Allows learner to compare own understanding
with others• Increases active participation
– Learner must commit to answer, thus increasing attention
– Affects active participation in other ways (asking more questions)
– Less sleeping during class– Superior to show of hands technique with increased
participation
![Page 18: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/18.jpg)
Cons
• Need initial investment in software and hardware– Program to create questions– “clickers” or other data entry modality
• Training and buy-in from faculty• Willingness to participate by students
– Some negative responses
• Using ARS decreases time for actual delivery of material during the course of lecture
• Can be distracting if overused in a single lecture (2-5/50 minute lecture)
![Page 19: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/19.jpg)
Question Types
• Factual Questions• Conceptual Questions• One-best-answer Questions• Opinion Questions• Data Gathering Questions• Questions Asking for Predictions• Feedback on Teaching
![Page 20: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/20.jpg)
Question Types
Factual Questions
These questions might be used to see if students did the reading, remember
important points from prior classes, or have memorized key facts.
Vanderbilt Center for Teaching
![Page 21: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/21.jpg)
Question Types
Conceptual Questions
Multiple-choice questions that demonstrate whether or not students understand important concepts and principles.
Vanderbilt Center for Teaching
![Page 22: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/22.jpg)
Question Types
One-best-answer Questions
Questions which include multiple answer choices, more than one of which could be argued as correct. Students are asked to
select the one best answer from these choices. Also referred to as "partially
correct multiple-choice questions" (Burnstein & Lederman, 2006)
Vanderbilt Center for Teaching
![Page 23: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/23.jpg)
Question Types
Opinion Questions
Evaluative and opinion questions which may not have correct answers. Asking
these questions can engage students and provoke rich discussions, particularly in
response to ethical, legal, or moral issues.
Vanderbilt Center for Teaching
![Page 24: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/24.jpg)
Question Types
Data Gathering Questions
Used to quickly gather demographic, opinion, or other data from a class full of
students. This is often useful in social science classes that utilize social or
economic experiments in the classroom.
Vanderbilt Center for Teaching
![Page 25: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/25.jpg)
Question Types
Questions Asking for Predictions
Students can be asked to predict the outcome of an experiment prior to being
shown the experiment. This gives the teacher a sense of the students’
preconceptions and increases the surprise value of the experiment when students see
just how many of their classmates expected different outcomes.
Vanderbilt Center for Teaching
![Page 26: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/26.jpg)
Question Types
Feedback on Teaching
Ask students to provide direct feedback on how well a particular class is going. For example, one might ask students to rate
their level of understanding of the lecture thus far on a scale of 1 to 4.
Vanderbilt Center for Teaching
![Page 27: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/27.jpg)
Question Best Practices
• Apply Bloom’s Taxonomy to test concepts rather than recall
• Test understanding of jargon separately from understanding of concepts to be sure source of confusion can be clearly identified
• Include “I don’t know” to ensure data truly reflects understanding rather than some sample of student guessing
![Page 28: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/28.jpg)
Bloom’s Taxonomy
![Page 29: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/29.jpg)
Bloom’s Taxonomy6. Evaluation: think critically about and defend a position
– Judge, Justify, Defend, Criticize, Evaluate7. Synthesis: transform, combine ideas to create
something new– Develop, Create, Propose, Design, Invent
8. Analysis: break down concepts into parts– Compare, Contrast, Distinguish
9. Application: apply comprehension to unfamiliar situations– Apply, Use, Compute, Solve, Predict
10.Comprehension: demonstrate understanding of ideas, concepts– Restate, Explain, Summarize, Interpret, Describe
11.Factual Knowledge: remember and recall factual information– Define, List, State, Name, Cite
![Page 30: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/30.jpg)
Question Best Practices
• Use variety of types of questions• Ask about linking a general concept to a specific
example• Ask about linking a familiar idea to a new concept• Ask about a concept yet to be introduced to
increase student curiosity• Can ask question twice
– First for individuals– Second after student discussion to promote exchange
of ideas
![Page 31: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/31.jpg)
How would you rate this workshop so far?
Fas
cinat
ing
Som
ewhat
inte
rest
ing
Borin
g
A w
aste
of t
ime
25%
0%0%
75%1. Fascinating2. Somewhat
interesting3. Boring4. A waste of time
![Page 32: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/32.jpg)
Question Writing Exercise
![Page 33: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/33.jpg)
35 yo G0 BMI 40 recently married with long h/o oligoovulation. EMB shows CAH. Propose next step:
Obse
rvat
ion, r
epea
t bx
in..
Rep
eat e
mb o
r D a
nd C n
ow
Ultr
asoun
d
Med
ical
mgm
t: p
roges
ti...
Gyn
Onc
consu
lt/pa
th re
...
hys
tere
ctom
y
0%
14%
0%
14%
71%
0%
1. Observation, repeat bx in 3 mos
2. Repeat emb or D and C now
3. Ultrasound4. Medical mgmt:
progestin Rx and eval for PCOS
5. Gyn Onc consult/path review
6. hysterectomy
![Page 34: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/34.jpg)
To use for future conference planning, Palm Beach is a great location for a conference because:
0%
31%
54%
15% 1. Accessible location
2. Pleasant weather
3. Luxurious accomodations
4. Various recreational activities
![Page 35: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/35.jpg)
UNC Experience
• Questions should be similar difficulty as those on examinations
• Students liked ARS especially if questions provided hints to examination materials
• Participation was about 60-70%, not the 100% we anticipated
![Page 36: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/36.jpg)
Best Practices
• Planning– Be sure to have tech support– Decide how ARS will or will not affect student
grades– Define clear objectives for use of ARS in your
classroom and write questions with these objectives in mind
– First year will take a little more time and effort– Practice when first starting to use ARS
![Page 37: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/37.jpg)
Best Practices
• Student communication– If used for grading, be sure to have rules
about use (i.e. no use of other’s clickers)• Consider throwing out some grades for
inadvertent clicking• Consider making it worth 5-10% of total grade (i.e.
low stakes, but not too low)
– Plan for time to debrief after each question
![Page 38: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/38.jpg)
ARS Options
• Different technologies– IR transmitters– RF transmitters– Web-based– Smartphone
• Presentation software integration• LMS integration
![Page 39: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/39.jpg)
ARS Options
Turning Point
iClickerInterwrite
![Page 40: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/40.jpg)
Acknowledgements
• Special thanks to:
Jake AcheyInstructional Technology TechnicianUNC School of MedicineMedical Students Teaching Lab
![Page 41: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/41.jpg)
References• 1. BARBER M, NJUS D. Clicker evolution: seeking intelligent
design. CBE Life Sci Educ 2007;6:1-8.• 2. CALDWELL JE. Clickers in the large classroom: current
research and best-practice tips. CBE Life Sci Educ 2007;6:9-20.• 3. COLLINS J. Audience response systems: technology to
engage learners. J Am Coll Radiol 2008;5:993-1000.• 4. COLLINS LJ. Livening up the classroom: using audience
response systems to promote active learning. Med Ref Serv Q 2007;26:81-8.
• 5. NAYAK L, ERINJERI JP. Audience response systems in medical student education benefit learners and presenters. Acad Radiol 2008;15:383-9.
• 6. PRADHAN A, SPARANO D, ANANTH CV. The influence of an audience response system on knowledge retention: an application to resident education. Am J Obstet Gynecol 2005;193:1827-30.
• 7. TURPIN DL. Enhance learning with an audience response system. Am J Orthod Dentofacial Orthop 2003;124:607
![Page 42: Audience Response Systems: Transforming your Lecture Hall into a Platform for Interactivity Alice Chuang, MD University of North Carolina – Chapel Hill](https://reader036.vdocuments.us/reader036/viewer/2022062716/56649ddc5503460f94ad4050/html5/thumbnails/42.jpg)
Session content, including narrated MS Powerpoint slides
available at:
http://www.obgynknowledgebank.net