tablets for medical education (t4me) in rwanda: usage patterns & outcomes over three years...

Tablets for Medical Education (T4ME) in Rwanda: Usage Patterns & Outcomes Over Three Years

Distance Teaching & Learning Conference, Madison, WIAug 13, 2015

Adam Papendieck MPHProgram Director, Tulane University School of Law, Payson Graduate Program in Global

DevelopmentDoctoral Student, University of Texas at Austin, C&I - Learning Technologies

Nancy Mock DrPHAssistant Professor, Tulane University, School of Public Health and Tropical Medicine

Patrick Kyamanywa MDAssociate Professor, University of Rwanda, School of Medicine

Tablets for Medical Education (T4ME)

• In fall 2012, 91 third year medical students were given 7-inch Samsung Galaxy tabs with a case, keyboard, storage, sim card (2 months data)

• Preloaded with a variety of apps, e.g.:• Epocrates• Medscape• DropBox• Kindle• Annotation• Skype

• Support• Training• Facebook User Groups• Email support

Goals1. Increase access to internet

and (digital) resources

2. Network a community of practice: improve connectivity, communication and collaboration

3. Enhance new literacies and digital participation (Jenkins et al., 2006)

4. Improve learning outcomes

Photo source: The New Times / JP Bucyensenge

Outcomes: Internet and Device Usage

Baseline 3 Months 12 Month 33 Month0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

Use computer daily for studyUse tablet daily for studyUse of internet daily for study

N=91 n=86 n=82 n=21

Outcomes: Centrality of Web

Baseline 3 Months 12 Month 33 Month0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

Online resources are important for studyOnline resources are easy to access

N=91 n=86 n=82 n=21

Outcomes: Resource Access

Baseline 3 Months 12 Month 33 Month0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

Easy to access assigned readingsEasy to access high quality resources

N=91 n=86 n=82 n=21

2012 Test Cohort vs. 2011 (Control)

HAEMIMMUNO

CVS

RESPIRATORY

GENETICS

MUSCULOPER

INFECTIOUS

GIT

OVERALL

-20.0 -15.0 -10.0 -5.0 0.0 5.0 10.0 15.0

Performance difference (%) on summative exams*

Performance difference (%)

*all differences were significant at p<.000 (86 out of 91 responding)

The Angry Birds Hypothesis

Type of usage and exam scores

• No significant association between overall scores and usage patterns

• A few significant negative associations with individual exam scores, for example:– daily internet access (MUSCULO-PER, p<.004)– daily use of the computer (MUSCULO-PER, p<.010)– daily use of tablet for social networking (MUSCULO-PER, p<.025)– daily use of tablet for web conferencing (MUSCULO-PER, p<.028) – gaming more than once per day (INFECTIO, p<.04)

• One significant positive association– daily usage of the tablet’s phone (HAEMIMMU, p<.032)

Students reported tablets as important academically, professionally and personally

% n

Tablets were important for academic work 95.2 21

Tablets were important for life outside of the university 100 21

Tablet ownership influenced peers to procure a computing device

95.0 21

*33 months post intervention

*

Outcomes and Impact

Portable access to resources“It helped me during my ward rounds and bedside learning and in preparation of my presentations using android apps for medicine like medscape, skyscape and others”

“I have some medicals apps that I do use which increase my performance [because I check in the] field what I don’t know.”

“I have been able to know…different medical issues and discoveries in medicine.”

“Academically I am able to access online materials”

Outcomes and Impact

Key literacy development

“I [now] read often per day.”

“As far as it is easier to carry it [everywhere] it indirectly increase[s] my study reading.”

“The tablet has improved my skills on electronic devices”

“…improving skills [on] other computing devices and in communication.”

Outcomes and Impact

Community networking and communication

“I have been able to access my emails without going to internet cafes.”

“The tablet helps me in communication either by sms, phone or by social networks.”

It has also impacted the way I use social networks.

…communication with my colleagues has also become easy.

…helps me to socialize with the world, interact with my friends.

Main Problems Reported

• Issues of “isolation” and “dependence” • Limited internet access• Device malfunction (crashes)

Support wishes

• Free, high-bandwidth connectivity• Access to protected sites and resources (e.g. Hinari)• App and online resource orientation• Training on very specific apps (e.g. UpToDate) or

tasks (presentation creation)

Lessons

• Digital literacy and self-efficacy may be positively impact with tablets in Rwanda, while decreasing overall usage of computers

• Learner agency, resource access and sense of community may be positively impacted with tablets in Rwanda, but internet access is limiting

Lessons• In context, summative semester exam scores are probably not a

satisfactory measure of impact (on their own)– Exams may change semester to semester– Exams may not measure what we actually want to know

• In general, it is helpful to measure usage and impact broadly in technologically marginalized contexts– We may miss key issues by focusing on narrow measures, or by not

measuring.– There is evidence that extremely constructionist interventions which use

technology may sometimes disrupt technologically marginalized learners• See Warschauer and Ames (2010) look at the impact of OLPC• See Reardon (2011) and Baily & Dynarsky (2011) about the widening education

and achievement gaps in the US

LessonsLightweight, mixed methods monitoring and evaluation strategies which are adaptable and active over longer periods of are helpful and feasible:

• Monitor small indicator sets over time for implementation guidance– SMS: Textit (textit.in) RapidPro (rapidpro.io)– Email/web: surveymonkey, polleverywhere

• Qualitative open-ended questions and focus groups – Barriers/solutions/support gaps, – Relevant resources– Innovative user strategies in context– Development of CoP and participation– Wikisurveys: allourideas.org

• Mobile device management (MDM), Web and Learning Analytics

Thanks!

Adam Papendieck

apapendieck@gmail.com

@apapendieck

Health Systems Transformation

Faculty development

Educational Programs

Applied Research: policy, operations and

evaluation

Administrative Systems

Faculty and student extension/outreach

Higher Ed. Partnerships Health systems change

Improved program policies, program design and implementation

Sustainable production of health workforce needs in country

Systems Change

Student graduation growth in Rwanda

Source: Rwanda Higher Education Council (2011)

Research Context

• Extremely constructionist tech interventions might further disrupt technologically marginalized– See Warschauer and Ames (2010) look at the

impact of OLPC– See Reardon (2011) and Baily & Dynarsky (2011)

about the widening education and achievement gaps in the US