my learning: do online learning modules have an effect?
TRANSCRIPT
CareSearch is funded by the Australian Government Department of Health.
My Learning: Do online learning modules have an effect?Deb Rawlings, Jennifer Tieman
PCNA 2016
What will be covered today:
• Knowledge into practice• CareSearch ‘My Learning’ Modules• Evaluation• Results• Implications
We all know that research evidence can be important in highlighting how effective a treatment or therapy is or in making us aware of the differences between groups and care settings.But it can be more difficult to understand how to use this evidence in your clinical practice.
Knowledge TranslationOne important concept that has driven the development of the CareSearch project is the role of evidence from creation to application, 'the knowledge translation cycle'
Knowledge translation looks at the processes that affect how evidence is generated, communicated and utilised as well as barriers to the transfer and uptake of this knowledge
• implementation science, research utilization, knowledge transfer and uptake, knowledge translation, knowledge-to-action
Knowledge Translation• CareSearch Hubs
• These knowledge hubs consolidate the relevant evidence and facilitate access to practical resources and relevant literature for specific professional groups
• My Learning• These modules demonstrate how to find relevant evidence,
and how to use the evidence resources in CareSearch to make a difference in clinical care
Why are we doing ‘My Learning’?
• Other avenues for education on pain etc..• We are building understanding of evidence into
practice which we have seen as a major gap• Specifically using CareSearch evidence in palliative care practice
• My Learning fits in with professional responsibilities re: education
Online Learning• Active education can increase the chance of
practice change• Look at things, do things, engage with content• This partly prompted the redesign of the modules
• Characterised in narrative terms• Personal and relevant to practice
• Learners can study when time and capacity allow
My learning?• Quick and easy to access• Cost effective (ie, free)• Voluntary• Case based• After completing the learning packages and
answering the review quiz, a certificate can be printed for professional records (such as Continuing Professional Development)
CareSearch online learning modules
• The first CareSearch ‘My Learning’ modules were introduced in January 2012
• Passive approach to evaluation• We knew they were being done• We knew that they were seen to be useful
• We thought that we could do better
Informal evaluation via email
“Accessed from ipad. Completed the 6 modules: very interesting and relevant information for my work
practice. Enjoyable reading and learning. Learnt a lot.”
Evaluation• Evaluation cycle (CareSearch Evaluation)
• Evaluation or Research? Ethics discussion
Evaluation is a term that generally encompasses the systematic collection and analysis of information to make judgements, usually about the effectiveness, efficiency and/or appropriateness of an activity. In many situations, oversight of the activity is required, but ethical review is not necessary.
Ethical considerations in quality assurance and evaluation activities (National Health and Medical Research Council, March 2014)
Evaluation• March 2016 – Refresh and Review process
• Education principles - how to improve what we were doing• Worked with an educationalist to update and develop• Formative evaluation – user feedback eg, visuals, language• Questions developed, including intent to change practice• Capacity to get data to see if we are changing outcomes
Evaluation• CareSearch is Government funded• Included as part of our evaluation schedule• We need to know if we are having an effect• Planned as ongoing evaluation, not a one-off• Sign off from our external evaluators
Data? • Data from April to July 2016• Pre (demographics) and post (usefulness)
• Is voluntary – so not everyone will complete• Some modules are more interesting than others• Some modules were developed for specific audiences
My Learning Modules• My Learning 1: Finding Evidence (Published
Literature)• Eileen has non-small cell lung carcinoma. This module
shows you how the Clinical Evidence pages, Review Collection and PubMed Topic Searches helped Eileen's breathlessness.
• Evaluation pre (n=115) and post (n=65)
My Learning Modules• My Learning 2: Finding Evidence (Searching for
Evidence) • Clare is caring for her husband John. She is concerned about
the vivid and realistic dreams he is having about his dead brother. You use some PubMed resources to answer her questions.
• Evaluation pre (n=46)and post (n=31)
My Learning Modules• My Learning 3: Searching the Web
• Ruth was diagnosed with stage II melanoma five years ago and has now had maximal treatment for liver and brain metastases. She lives with her husband Bruce and their two teenage children.
• Evaluation pre (n=44) and post (n=34)
My Learning Modules
• My Learning 4: Residential Aged Care • Ken is a resident in the Kangaroo Valley Aged Care
Facility. The staff are not sure if Ken is approaching the end of his life. This module profiles resources in the Residential Aged Care Hub.
• Evaluation pre (n=52) and post (n=35)
My Learning Modules
• My Learning 5: Dementia • Anna's father Lou has dementia. As his condition
worsens, the family disagrees about his care. This module highlights dementia resources in CareSearch.
• Evaluation pre (n=54) and post (n=37)
My Learning Modules
• My Learning 6: Carers • Lana is a 35 year old woman who is sole parent to
Rhiannon and John, 10 year old twins. Her mother June became very ill and moved in with Lana. Lana struggles to cope.
• Evaluation pre (n=39) and post (n=30)
ResultsPre- modules 1-6 (n=349)• Age, gender• Profession - representative of workforce• Undertaken a previous module
• My Learning one – 92% had not done one before• My Learning 2-6 more equal numbers
Respondents Modules 1-6 PREDEMOGRAPHICS
Age(n=349)
18-3031-4041-5051-6061+
23389514350
7%11%27%41%14%
Gender(n=349)
FemaleMale
31929
92%8%
Profession(n=341)
Aged Care WorkerCarerAllied Health ProfessionalDoctorNurseOther
26320623650
8%1%6%1%69%15%
Previous Module(n=347)
YesNo
164183
47%53%
Post module• All found easy to use
• Navigation, design• 92% intend to use information found in their
practice• Influence, quality product
• 93% will visit CS in next 3 months• 96% will undertake another module
• Intent to learn• 93% will recommend to a colleague
My Learning ModulesEvaluated separately• My Learning 7: Knowledge Translation • This module looks at how a clinical team uses the
Knowledge to Action cycle to improve outcomes for their patients. This module was developed for the Centre for Research Excellence in end of Life Care .
• Evaluation pre (n=90) and post (n=35)
Respondents Modules 1-6 POSTUSEFULNESS OF MODULES
I found this module easy to use (n=230) Yes 230 100%
I am likely to use information from this module in my practice (n=227)
YesMaybe
20819
92%8%
I will visit CareSearch in the next 3 months (n=227)
YesMaybe
21215
93%7%
I would recommend this My Learning module to a colleague (n=229)
YesMaybe
21415
93%7%
I will undertake another My Learning module (n=225)
YesMaybe
2178
96%4%
Intent to change• Theory that says to be specific - time-bonded - is
an indication of potential action (Webb and Sheeran, 2006)
• It suggests that learning has occurred and that change is likely to follow
Webb T and Sheeran P (2006) Does Changing Behavioral Intentions Engender Behavior Change? A Meta-Analysis of the Experimental Evidence Psychological Bulletin 132 (2): 249–268
ResultsModule 7 (n=90)• Different topic, larger numbers despite being
available for a shorter time• Age and gender profiles similar to 1-6• Slight increase in Allied Health professionals (3%
increase) and Doctors (8% increase) undertaking the modules (although numbers are generally low)
Respondents Module 7 (KNOWLEDGE TRANSLATION) PREDEMOGRAPHICS
Age(n=90)
18-3031-4041-5051-6061+
311234013
3%12%26%44%14%
Gender(n=90)
FemaleMaleOther
8271
91%8%1%
Profession(n=90)
Aged Care WorkerAllied Health ProfessionalDoctorNurseOther
6885216
7%9%9%58%18%
Results
Post module 7• Questions concentrating more on evidence use,
knowledge to practice and change management• People identify this as an issue• A more popular module as learning on this is not
provided routinely elsewhere
Respondents Module 7 (KNOWLEDGE TRANSLATION) POST
I found this module easy to use (n=35) YesNo
332
94%6%
After completing this module, I feel more confident to participate in a knowledge translation project at my workplace (n=35)
YesNoUnsure
2636
74%9%17%
I believe evidence should be an important part of change management (n=35)
Agree 35 100%
If you were undertaking a knowledge translation project, which part would you expect to find most difficult? (n=32)
Would you recommend this My Learning module to a colleague? (n=35)
YesNoUnsure
2915
83%3%14%
• 17% (n=6) respondents were ‘unsure’ of an increase in their confidence allowing them to participate in an KT project
• 14% (n=5) respondents were ‘unsure’ about recommending it to a colleague• Could be a reflection of the level it was pitched at, the concept or its’
relevance
• KT is hard but it’s what everyone wants everyone to be doing
If you were undertaking a knowledge translation project, which part would you expect to find most difficult? (n=32)
• Assessing Barriers locally (8) (25%)• Evaluating outcomes (2) (6%)• Finding evidence (2) (6%)• Identifying a knowledge-practice gap (2) (6%)• Monitoring knowledge use (1) (3%)• Sustaining knowledge use over time (7) (21%)• Tailoring and implementing interventions for change (10) (31%)
Do online learning modules have an effect? Translating knowledge into practice adds complexity
• The modules are easy to use • The intent is to use them in practice• Module 7 post evaluation highlights the complexities
involved• 100% believing that evidence should be an important part of change
management• Only75% confident in participating in a knowledge translation project• 25% would expect to find Assessing barriers locally as most difficult
What we have learnt• Need to contemporise learning
• Investigate educational theory to see if that would effect how content is organised and delivered
• Need to look to design and user experience
• My Learning does appear to be having an effect
Future Plans• In what order are they doing them?• When and where doing?• How did you find out about My Learning?• How do we promote?• Who do we promote to?
• How do YOU think we should be engaging with the nursing community to build knowledge?
Conclusion
• Structured approaches to knowledge translation• Ability to capture data that can help in reporting and
in decision making• Ability to give users a voice• Better quality products as a result of this approach
CareSearch is funded by the Australian Government Department of Health.
CareSearch would like to thank the many people who contribute their time and expertise to the project, including members of the National Advisory Group and the Knowledge Network Management Group.www.caresearch.com.au