DEVELOPMENT OF AN EDUCATION DASHBOARD TO SUPPORT MEDICAL STUDENT LEARNING ROBERT KAMEI, SANDY COOK, SCOTT COMPTON, SANDRA BOESCH

TEACHING ENHANCEMENT GRANT, NUS

MEDICAL SCHOOL REQUIREMENTS• Knowledge

• Pre-Clinical Sciences • Anatomy, Biochemistry, Physiology, etc

• Clinical Specialities• Pediatrics, Internal Medicine, Surgery, Psychiatry, etc

• Skills• Clinical Reasoning• Psychomotor

• Suturing, IV placement• Interpersonal Communication

• Attitudes• Professionalism• Empathy

AssessmentPerformance

This image was released to the Public Domain by the National Cancer Institute,

AssessmentPerformance

For the Student and Faculty Mentor to make sense of this:

BACKGROUND

• Medical students are typically assessed in numerous ways and at multiple points in time.

• Students and their advisors are generally unable to combine this assessment data in ways that could provide powerful summary evidence of the students’ overall strengths and weaknesses.

• As a result, they often lack a cohesive strategy to assist them in interpreting this data in ways that can direct their future learning.

STUDENT DASHBOARD

Mission Motorcycle digital dashboard Gramm Bjoern https://www.youtube.com/watch?v=WDCUakUkVvY

OBJECTIVE

Develop an interactive, continuous feedback, technological resource for managing and interpreting assessment data in order to facilitate individualized educational planning.

METHODS: TEG GRANTA secure website (“Dashboard”) for medical students with three functional purposes: 1. To serve as a repository for assessment

reports from various testing services, such as the National Board of Medical Examiner subject-specific exam reports,

2. To serve as a launching pad to third-party test preparation services, and

3. To receive performance reports in real time for continuous feedback and future self-directed learning.

GOOGLE APPS

NUS VERIFICATION ‐ LOGIN

Upload New Students

Rosh Review Invitation Email

Rosh ReviewAll student feedback on one place accessible via single sign-on

Image from ad: https://itunes.apple.com/us/app/rosh-review-mobile/id991352355?mt=8

STUDENT REPOSITORY

RESULTS1. Successfully created a cloud-based website using our design goals.

2. Built Application Programming Interfaces (APIs) with two third-party vendors:

1. USMLE-Rx (www.usmle-rx.com), 2. Rosh Review (www.roshreview.com),

3. The APIs allowed students to launch out to the third-party sites, respond to question banks, and the results were then pushed back to the Dashboard for visualization.

4. Sixteen students piloted the Dashboard.

1. Students rated the access to and the performance output reports from the USMLE-Rx question bank the most valuable elements of the Dashboard.

SINGAPORE PERSONAL DATA PROTECTION ACT• Enacted in July 2014 during the time of this study project.

• Required to shut down Dashboard to comply with Act requirements

• Additional safeguards with the dashboard are required and will incorporate our learning in new web-based learning management system dashboard

CONCLUSIONEarly positive feedback regarding the value of developing individualized education plans.

On a separate but related project, our Predictive Analytics allowed us to achieve a 98% first time pass rate for USMLE Step 1 Examination

Next steps will be to:

1. Ensure that the Dashboard is compliant with the Personal Data Protection Act (PDPA)

2. Investigate additional third-party sites for potential partnerships.

3. Improve the visualization of performance data

4. Evaluate the overall impact on learning provided by this resource.

CONCLUSIONOur education work at DukeNUS Medical School has previously focused on two key areas:

• developing technology to efficiently deliver contentto our students and

• creating a complementary in-class, team based experiences

This project begins to improve the learning that is achieved after the in-class activities. Medical learning can’t stop after the classroom end, instead, students must learn medicine over a lifetime by continually assessing areas of weakness.

As Toffler stated, “unlearning and

relearning” information while acquiring new

knowledge and skills will be necessary.

Acquiring new knowledge and skills is

only possible with accurate and

continuous feedback.

REFERENCESAcknowledgement: The authors would like to thank the support from the National University of Singapore, Teaching Enhancement Grant.

1) Slotnick HB (1999) How doctors learn: physicians' self-directed learning episodes. Acad Med. Oct;74(10):1106-17. 2) Eva KW, Cunnington JPW, Reiter HJ, Keane DR, Norman GR (2004). How Can I Know What I Don’t Know? Poor Self Assessment in a Well-Defined Domain, Advances in Health Sciences Education, 9: 211–224 3) Brown WE, Lovett M, Bajzek, D & Burnette JM (2006). Improving the Feedback Cycle to Improve Learning in Introductory Biology Using the Digital Dashboard. In T. Reeves & S. Yamashita (Eds.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2006 (pp. 1030-1035). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). 4) Bajzek D, Brown W, Lovett M & Rule G (2007). Inventing the Digital Dashboard for Learning. In C. Montgomerie & J. Seale (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 (pp. 1084-1092). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE). 5) Corrin L & de Barba P (2014). Exploring students’ interpretation of feedback delivered through learning analytics dashboards. In B. Hegarty, J. McDonald, &S.K. Loke (Eds.), Rhetoric and Reality: 7 Critical perspectives on educational technology. Proceedings ascilite Dunedin.NZ 2014 (pp. 629-633).

www.duke-nus.edu.sg