usability evaluation of an online resource designed to help teachers to recognize and respond...
TRANSCRIPT
Usability Evaluation of an Online Resource Designed
to Help Teachers to Recognize and Respond
Appropriately to a Concussion
Darryl FunaiUniversity of Hawaii at Manoa
4/22/14
Poll Question
Have you or someone you know
ever been concussed?
Background
1.7 million people sustain a concussion every year (CDC, 2006)
Background
1.7 million people sustain a concussion every year (CDC, 2006)
60% of all teenage athletes will sustain a concussion before they graduate from high school. (Collins & Hawn, 2002)
What is a concussion?
A traumatic brain injury that can significantly influence the brain's ability to function at its normal capacity.
Rationale for study?
It is especially important for educators to know how to recognize a concussion
Rationale for study?
It is especially important for educators to know how to recognize a concussion
What to do if their student sustains a concussion
Rationale for study?
It is especially important for educators to know how to recognize a concussion
What to do if their student sustains a concussion
The importance of physical and mental rest, followed by a gradual progression back to activity
Rationale for study?
Information for specifically for educators is limited.
Rationale for study?
Information for specifically for educators is limited.
It is time consuming and ineffective for educators to curate information themselves.
Purpose
To evaluate the content and organization of a website designed to provide instruction and resources for educators to help them recognize and respond appropriately to a concussion.
Why a Usability Study?
Direct feedback from the target audience
Improve the users' experience
Quality content is only as good as the navigation of the website
Methods
Based on Krug's, Rocket Surgery Made Easy (2009)
Methods
Week Action
1 Identified potential participants and collected their demographic information via a pretest questionnaire on Google Forms.
2 Performed the usability test and aggregated the collected data.
3 Data from the first round of testing was analyzed and the website was redesigned accordingly.
4 The second iteration of the usability test was conducted.
5 Data from the second iteration was analyzed and the website was redesigned accordingly.
Timeline
Methods
Participant pool15 educators were contacted via email6 participants selected
Based on level of technical expertise and prior use of site
2 iterations of 3 participants each
Methods
General comments
4 tasks
Given in random order
40 minutes to complete
Tasks
1. Please explain to me the schools Return to Play Protocol for Concussions.
Tasks
1. Please explain to me the schools Return to Play Protocol for Concussions.
2. Please explain to me the signs and symptoms of a concussion an educator might observe in their student.
Tasks
1. Please explain to me the schools Return to Play Protocol for Concussions.
2. Please explain to me the signs and symptoms of a concussion an educator might observe in their student.
3. Please open up the resource: "Heads up: Concussion in Youth Sports".
Tasks
1. Please explain to me the schools Return to Play Protocol for Concussions.
2. Please explain to me the signs and symptoms of a concussion an educator might observe in their student.
3. Please open up the resource: "Heads up: Concussion in Youth Sports".
4. Please explain to me how you may accommodate a student who has the following symptoms in your class:a. Difficulty learning new conceptsb. Difficulty concentratingc. Feeling slowed down
Results
Demographic Survey4 males and 2 femalesAverage 13 years of teaching experienceAll participants have had a concussed
student in their class17% reported having visited the site prior to
test67% self rated at least a 4 on 5 pt. scale on
comfort level navigating websites.
Results
Eight issues
Focused on top three "Had to scroll down to bottom of page to find
important information" "Drop down boxes are confusing" "Resources section doesn't make sense"
Discussion
• Bagley, A. F., Daneshvar, D. H., Schanker, B. D., Zurakowski, D., dʼHemecourt, C. a, Nowinski, C. J., Goulet, K. (2012). Effectiveness of the SLICE program for youth concussion education. Clinical journal of sport medicine : official journal of the Canadian Academy of Sport Medicine, 22(5), 385–9. doi:10.1097/JSM.0b013e3182639bb4
• Glang, A., Koester, M., & Beaver, S. (2010) Online training in sports concussion for youth sports coaches. International Journal of Sports Science Coach, 5(1), 1–12. doi:10.1260/1747-9541.5.1.1.
• Covassin, T., Elbin, R. J., & Sarmiento, K. (2012). Educating Coaches About Concussion in Sports: Evaluation of the CDC’s “Heads Up: Concussion in Youth Sports” Initiative. Journal of School Health, 82(5), 233–238. Retrieved from 10.1111/j.1746-1561.2012.00692.x
• David, A., & Glore, P. (2010). The Impact of Design and Aesthetics on Usability, Credibility, and Learning in an Online Environment. Online Journal Of Distance Learning Administration, 13(4).
ResultsGeneral Comments
Official school website
Too much Information
"clean" and "professional"
Liked video's
DiscussionToo much information
Before
Can only see 1/3 of webpage when landing on webpage
2 Video's
DiscussionToo much information
Before After
Deleted Video
DiscussionDrop Down Boxes
BeforeEasy to miss
Confusing
Discussion
Drop Down Boxes
Before After
Changed to make it consistent
DiscussionResources
Before
Confusing
DiscussionResources
Before After
Videos
PDF's
Conclusion
Content and navigation well received
Informational and professional looking
Benefit from layout modification and additional usability testing
References Centers for Disease Control and Prevention (CDC). (2006) Traumatic Brain Injury in the
United States: Emergency Department Visits, Hospitalizations and Deaths 2002 – 2006.
Collins, M.W., & Hawn, K.L. (2002, February). The clinical management of sports concussion . Current Sports Medicine Reports 1(1), 12-22.
David, A., & Glore, P. (2010). The Impact of Design and Aesthetics on Usability, Credibility, and Learning in an Online Environment. Online Journal Of Distance Learning Administration, 13(4).
Guskiewicz, K.M., Weaver, K.M., Padua, D.A., & Garrett, W.E. (2000). Epidimiology of concussion in collegiate and high school football players. American Journal of Sports Medicine, 28(5), 643-650.
Krug, S. (2009) Rocket surgery made easy: The do-it-yourself guide to finding and fixing usability problems. Berkeley, CA: New Riders Press.
McCrory, P., Meeuwisse, W. H., Aubry, M., Cantu, B., Dvorák, J., Echemendia, R. J., … Turner, M. (2013). Consensus statement on concussion in sport: the 4th International Conference on Concussion in Sport held in Zurich, November 2012. British journal of sports medicine, 47(5), 250–8. doi:10.1136/bjsports-2013-092313
McCrea, M., Hammeke, T., Olsen, G., Leo, P., & Guskiewics, K. (2004). Unreported concussion in high school football players: Implications for prevention. Clinical Journal of Sports Medicine, 14(1), 13-17.