M&M - M&M - Final Mobile Learning ProjectFinal Mobile Learning Project

The Use of Mobile The Use of Mobile Audience Response Audience Response Apps in Post-Graduate Apps in Post-Graduate Medical EducationMedical Education

ETEC 647 - Fall12ETEC 647 - Fall12Tod AebyTod AebyJim PetersenJim PetersenSean MoroneySean Moroney

Please Join Us

Overview

• Review the purpose of the project

• The M&M Concerns

• What happened on November 16, 2012

• What we learned

Were you paying attention?

• Please enter what you you remember about what we were trying to improve with mobile learning

Were you paying attention?

• Please enter what you you remember about what we were trying to improve with mobile learning

Learner Engagement!!

Mobile Audience Response: enhanced engagement for learningThere is substantial evidence in the educational literature There is substantial evidence in the educational literature

indicating that increased engagement, participation, and indicating that increased engagement, participation, and interaction in classroom and other educational settings can interaction in classroom and other educational settings can positively impact the learning of students positively impact the learning of students

The purpose of this project is to introduce the use of a free mobile The purpose of this project is to introduce the use of a free mobile audience response application with the goal of increasing audience response application with the goal of increasing engagement and participation of the residents who do not usually engagement and participation of the residents who do not usually respond or participate in the conference. The assumption is that respond or participate in the conference. The assumption is that increased engagement and participation will enhance the value of increased engagement and participation will enhance the value of the M&M Conference as an important learning tool in the residentsthe M&M Conference as an important learning tool in the residents ’’ medical education.medical education.

Recall:Recall:

Review of Literature•Note: the literature concerning Audience Response Systems refers almost entirely to the more traditional “Clicker” systems

•Audience response systems (ARS) were first introduced at Stanford and Cornell Universities in the mid 1960s and became commercially available in 1992. (Kay & LeSage, 2009)

•Since the early 1990s, the use of audience response systems in higher education, professional education, and business has grown substantially.

•In the past two decades, there have been numerous studies concerning the use of audience response systems that have indicated that this strategy has benefits in the teaching and learning process.

Some of these are:

•“students are more engaged in class”, students participate with peers more...to solve problems”, and “students are more focused in class”.

•There is also evidence that the use of these devices improves formative assessment and contingent teaching. (Kay & LeSage, 2009)

•There is evidence that the use of ARS provides a more positive impact on student participation than do more traditional audience response strategies such as surveys and note cards. (Stowell & Nelson, 2007)

Review of LiteratureInvestigators have reported that the use ARS can be efficacious in medical education; specifically in the process of

delivering medical information to postgraduate medical residents.

In one study of Family Medicine Residents, the investigators conducted a controlled crossover study and found a significant difference in the retention of information between groups who received traditional didactic lectures, lectures with an interactive component, and interactive lectures incorporating ARS. The investigators concluded that “improved retention occurs with active participation in the lecture process.” (Schackow, Chavez, Loya, & Friedman, 2004)

Another study compared the efficacy of traditional lecture style delivery of information to Obstetrics and Gynecology Residents with interactive lecture style delivery supplemented with an audience response system. The results of this randomized controlled trial indicated that Residents who received the ARS interactive lecture treatment showed a pre/post test improvement approximately ten times that of the group receiving the traditional lecture treatment. (Pradhan, Sparano, & Ananth, 2005)

A review of the literature concerning the efficacy of ARS predominantly indicates the value of interactive devices in promoting learning, retention, and engagement. (Banks, 2006) (Schmidt, 2011) (Guse & Zobitz, 2011)

There is also evidence that the use of ARS to replace traditional didactic lecture delivery enhances motivation and positive affective outcomes. (Cain, Black, & Rohr, 2009)

*! - important*! - important

Clients costcost(50 seats)(50 seats)

ease of deploymentease of deployment1-5 stars1-5 stars

ease of useease of use1-5 stars1-5 stars

types of responses available

other costsadd-ons

developer link Platforms Comments

eClicker Audience Free to student *** **** yes/noagee/dis

muliple choiceNo short answer

Presenter App is $15.00 http://www.bignerdranch.com/apps/eclicker-audienceiOS, html Well established and stable company.

MeridaWebKeyPad

Get quote *** **** yes/nomultiple choice

NO short answer

Hard to know! http://www.meridiaars.com/audiencepollingsoftware/Android, iOS, BlackBerry or Windows

Phone, html

Lots of polling options.  Have to email for a price quote (sounds

expensive)

Socrative free (50 seats) ***** ***** yes/nomultiple choiceshort answer

gamesexit ticket

None http://www.socrative.com Android, iOS, BlackBerry or

Windows Phone, html

Great Blogeasy to use

Solero Respond $5.00 Client **** **** yes/nomuliple choiceshort answer

$10-30 per month subscription

http://solaro.com/otherproducts IOS

SRN Client free to student ** ** yes/nomuliple choiceshort answer

$125 (50 clients) http://studentresponsenetwork.comIOS, html personal server-based

Turning TechnologiesResponseWare

$19/yrper student ** **** yes/no

multiple choiceshort answer

Presenter costs http://www.turningtechnologies.com/response-solutionsProprietary clickers, Android, iOS,

BlackBerry or Windows Phone

Can be used in mixed environment (phones, various clickers, etc)

Via Response free (10 seat trial) ** ** extensive (if you have to ask, you can’t afford it - fee not

openly posted)

http://www.viaresponse.com/Whatisit.htmlIOS, html (low ratings on App Store)

Poll Everywhere(SMS based)

available free on smart device *** *** yes/no

multiple choice$15-1400 per month

subscription http://www.polleverywhere.complatform independent

(smartphone)

Selection of Mobile Audience Selection of Mobile Audience Response AppResponse App

Three strategies

• Test general knowledge on a subject• What do they know and are they paying attention?

• Force commitment to test clinical judgement and reasoning

• What would they do and are they paying attention?

• Assess the current mental model

• Retest to confirm mastery of the concepts• Did they follow the conversation and construct the appropriate mental model

The Conference The project team arrived early to set up the equipment

The Conference

The audience customarily seats itself in a “culturally” determined manner, by role group

The Conference The Moderator was positioned so that he could observe the presenter, audience, and both screens

The Conference

Audience questions were clearly delineated by a special slide style

The Conference

Using Socrative on the iPad to present responses from the audience

The Conference

Audience responses are then projected to the smaller screen

The Conference Unexpected Consequence -

Formative Assessment - immediate feedback to moderator allowing instructional adjustment

The Residents present the The Residents present the cases...cases...

Other information of importance to medical practice is also presented

73% Laparoscopy 65% Laparotomy

73% Laparoscopy 65% Laparotomy

Evidence-based Treatment of URI

• Zinc lozenges

• High dose vitamin C

• Symptom relief

Vote or New List

Text

Text

Text

Text

Text

Text

Average 6.4 (2-10)

• 30% “Totally Got It”, 68% “Pretty Well”

• “Need bigger screen”, “Slow internet”, “Awesome, dude”,”Kinda fun”

Staff ObserversStaff Observers

Three staff members served as volunteer observers

They were provided a link to a Google Form to record their observations

Observers

• Audience size estimate at start, 39-50

• Mobile devices out/running @ 5 minutes 40 - 95%

• “Socrative taking a while”, “May need printed directions for late comers”, “Can you mask the entering answers so you don’t influence the later ones”, Faculty and private docs had mobile devices but not using Socrative”, “Most residence [sic] seemed engaged”

Observers

• The observers, some known to the audience (hospital staff) and some unknown (UH ETEC students), unobtrusively made periodic counts of the audience to assess involvement.

• At any time throughout the session, about two-thirds of the audience members had a mobile device on.

• Perhaps, not all of these were linked to Socrative.

Audience Engagement

Observer

7:45 8:00 8:15 8:30 8:45

1 9 9 9 8 9

2 10 10 9 9 9

3 4 7 9 8 8

Important Logistical Points• Late arrivals may have had difficulty logging in.

Some form of written instructions on this would probably be handy.

• Not revealing the answers given to a question until the question time is closed can give a clearer picture of the thinking of the audience members.

• Network speed can be an issue in the briskness of the interactions.

JimJim’’s s ReflectionReflection

• Process

• This was a fascinating project on a number of different levels

• The process was challenging since all of the team members are over-employed adult professionals residing in geographically disparate locations and with vastly differing schedules.

• Thus, online collaboration was absolutely necessary. Extensive use was made of Google Docs, Dropbox, and BlackBoard.

• It became apparent during the course of the project that organization and communication of information needs to be very different in an online collaboration setting.

JimJim’’s s Reflection 2Reflection 2

• Content

• One of the big realizations for me was how the app enabled the moderator to make on-the-fly formative assessments and adjust instruction

• This project has, for me, wide extensibility in a variety of educational settings. Several teachers in my school are now using the tool in classroom settings

• It was mentioned in the research literature and observed in the trial that there is a strong affective component when mobile devices are used in teaching and learning.

• I learned valuable lessons about the physical layout necessary to integrate the mobile element and I have some suggestions I will forward to the app developer to make it a more effective tool. (e.g. ability to hide responses until all recorded)

Sean’s Reflection

Tod’s Reflection

How did all the old white-hairs all end up in the same group?

A dry run really helps

Improved participation,will it last?

Too many devices in the kitchen spoils the soup

-Have to have broad band -What in the world am I going to do on Maui???

This has been a wonderful, eye-opening educational experience

Thanks Dr. Grace!