assessment of knowledge and skills in nursing and allied health student information literacy...
TRANSCRIPT
ASSESSMENT OF KNOWLEDGE AND SKILLS IN NURSING AND ALLIED HEALTH STUDENT INFORMATION LITERACY INSTRUCTION
A systematic review of the literature
Presented by Pamela Harrison on behalf of Jill Boruff & Pamela HarrisonJune 22, 2015
Background
•Primary researcher: Jill Boruff•Offering course-integrated instruction in PT/OT since 2008• Boruff & Thomas, 2011
•Currently, assessment in PT/OT takes the following forms:• Clicker-based assessment • Self-assessment (confidence scale, short answer)• Assignment: include clinical question, search strategy and selected resources
@jillyb3
#ILassess
What others are saying: Assessment•Oakleaf, 2008: overview of IL assessment methods•Oakleaf, 2009: Information Literacy Instruction Assessment Cycle (English 101)•Schilling & Applegate, 2012: assessment methods used by librarians (multidisciplinary)•Brettle, 2007: systematic review of measures for information skills training (health settings)#ILassess
What others are saying: Instruction
•Maggio & Kung, 2014: information retrieval instruction (UGME); includes information on librarian involvement in teaching/assessment cycle• Just, 2012: information retrieval instruction and assessment (medical students and residents); data on intervention, assessment tool & target•Koufogiannakis & Wiebe, 2006: systematic review of IL instruction (undergraduates)#ILassess
What we hope to do
•Provide a better understanding of how information literacy instruction is being assessed in nursing and allied health•Provide librarians and other educators with suggestions on how to improve assessment within IL instruction
#ILassess
Research question
How is assessment used in IL instruction in Nursing and Allied
Health?
Forms of assessme
nt
Target of assessme
nt
Instructional
context
#ILassess
• Tests of knowledge or skill• Examples: Multiple choice tests, short-answer tests
• Self-assessment or attitudinal surveys• Examples: Quizzes, focus groups
• Course assignments or products• Examples: citation analyses, bibliographies
• Is the assessment formative or summative?
Forms of assessment
#ILassess
Target of assessment 4 assessment areas identified (Khan & Coomarasamy, 2006; Schilling & Applegate, 2012):
Knowledge
• Recall and understand concepts taught
Skills• Ability to apply knowledge in context
Attitudes• Recognize the need to use acquired knowledge and skills
in scenarios, and intend to use those skills in practice
Behaviour• Use acquired knowledge and skills in practice
#ILassess
Instructional context
•Course-integrated, librarian-led workshops•One-shot librarian-led workshops• Instructor-led instruction•Other forms of instruction
(Sources: Khan & Coomarasamy, 2006; Maggio & Kung, 2014)#ILassess
Methodology: Search details
• Search: 1990 to January 16, 2015• Ovid Medline• EMBASE• CINAHL (EBSCO)• ERIC (EBSCO)• LISA• LISTA• ProQuest Theses & Dissertations
•Documentation: PRISMA flow chart (Source: Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & PRISMA Group. (2009))#ILassess
Search concepts
Nursing or Allied Health
Educational
program
IL Instructio
n or Assessme
nt
#ILassess
Methodology: Inclusion criteria
✔Nursing or Allied Health program✔Degree-seeking students✔Assessment of information literacy skills✔Question formulation✔Searching/resource selection✔Critical appraisal (if a librarian is involved)
#ILassess
Exclusion criteria✗Exclusively about medical students, medical residents, or non-degree programs
✗Continuing education programs or workplace/practitioner training programs
✗Exclusively about accreditation standards or competencies
✗Exclusively about faculty attitude toward, or use of, resources
✗Commentaries: position or policy statements; editorials; product reviews
✗Lists#ILassess
PRISMARecords identified through database
searching(n = 4366 )
Records after duplicates removed
(n = 2965)
Records published from 1990 onwards, and screened (n =
2724)
Additional records identified through other
sources (n = TBD)
Records excluded(n = 2143)
#ILassess
PRISMA continued…Records screened
(n = 2724)Records excluded
(n = 2143) (from previous slide)
Full-text articles assessed for eligibility (n = 605)
Studies included in qualitative synthesis (n=
130)
Full-text articles excluded (n = 475)• No librarian and no IL skills (n =
166)• No assessment described (n =
134)• No program described (n = 84)• Reasons from abstract level (n
= 34)• Review articles (n = 4)• Article not available (n = 11)• Language (n = 33)• Duplicates (n = 6)• Other (n = 3)
#ILassess
Preliminary results: (n = 130) # of included papers, by discipline
PT / OT SLP Nursing Mix Nurs/AH0
10
20
30
40
50
60
70
80
90
100
#ILassess
Preliminary results: (n = 22)PT/OT/SLP IL assessment snapshot
Assessment # Validated ( ✔ )
Assignment(s) 10
Adapted Fresno Test 3 ✔
Dutch Modified Fresno Test 1 ✔
Evidence-Based Practice Profile Questionnaire
1 ✔
Research Readiness Assessment 1 ✔
Vignette Skill Exercise 1
Survey/test: Self-efficacy 2
Survey: Satisfaction 3
Pre-/Post test: Knowledge 2
Survey/test: Behaviour and Attitudes 4
#ILassess
Next steps•Additional material:• For screening and data extraction: studies cited in review articles• For data extraction only: assessment tools or additional data (will contact authors)
•Data extraction for accepted items•Analysis: • What are the most common forms of assessment? Are any of them validated? • Which assessments are being used for which EBP skills (i.e. question formulation or searching)?
#ILassess
Observations and lessons learned•Full text screening for qualitative data takes a long time•Multi-purpose terms result in many false drops•Learning assessment vs. usability testing •Variation in depth of descriptions suggests that the literature would benefit from guidelines on reporting assessment methods#ILassess
Questions?Jill Boruff, Primary Investigator
McGill University [email protected]
Pamela Harrison, Co-Investigator McGill University
@jillyb3
@tweet_nothings_
References• Boruff, J. T., & Thomas, A. (2011). Integrating evidence-based
practice and information literacy skills in teaching physical and occupational therapy students. Health information and libraries journal, 28(4), 264.
• Brettle, A. (2007). Evaluating information skills training in health libraries: a systematic review. Health Information & Libraries Journal, 24(s1), 18.
• Just, M. L. (2012). Is literature search training for medical students and residents effective? a literature review. Journal of the Medical Library Association: JMLA, 100(4), 270.
• Khan, K. S., & Coomarasamy, A. (2006). A hierarchy of effective teaching and learning to acquire competence in evidenced-based medicine. BMC Medical Education, 6(1), 59. doi:10.1186/1472-6920-6-59
• Koufogiannakis, D,. & Wiebe, N. (2006). Effective methods for teaching information literacy skills to undergraduate students: A systematic review and meta-analysis. Evidence Based Library and Information Practice, (1)3: 3.
• Continued…
References (continued)• Maggio, L. A., & Kung, J. Y. (2014). How are medical students trained
to locate biomedical information to practice evidence-based medicine? a review of the 2007–2012 literature. Journal of the Medical Library Association: JMLA, 102(3), 184.
• Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA Group. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine, 6(7), e1000097.
• Oakleaf, M. (2008). Dangers and opportunities: a conceptual map of information literacy assessment approaches. portal: Libraries and the Academy, 8(3), 233.
• Oakleaf, M. (2009). The information literacy instruction assessment cycle: A guide for increasing student learning and improving librarian instructional skills. Journal of Documentation, (65)4, 539.
• Schilling, K., & Applegate, R. (2012). Best methods for evaluating educational impact: A comparison of the efficacy of commonly used measures of library instruction. Journal of the Medical Library Association: JMLA, 100(4), 258.