download related file
TRANSCRIPT
The NCSBN Na*onal Simula*on Study
Jennifer Hayden, MSN, RN
2
Review of Literature
• Small sample size • No control group • No randomiza*on • Variability in study design • No longitudinal studies
3
Research Ques*ons-‐Part I
1. Does subs*tu*ng clinical hours with 25% and 50% simula*on impact educa*onal outcomes (knowledge, clinical competency, cri*cal thinking and readiness for prac*ce) assessed at the end of the undergraduate nursing program?
4
Research Ques*ons-‐Part I
2. Are there course by course differences in nursing knowledge, clinical competency, and percep*on of learning needs being met among undergraduate students when tradi*onal clinical hours are subs*tuted with 25% and 50% simula*on?
5
Research Ques*ons-‐Part I
3. Are there differences in first-‐*me NCLEX pass rates between students that were randomized into a control group, 25% and 50% of tradi*onal clinical subs*tuted with simula*on?
6
Research Ques*ons-‐Part II
1. Are there differences in clinical competency, cri*cal thinking and readiness for prac*ce among the new graduate nurses from the three study groups?
2. Are there differences among new graduates from the three study groups in acclima*on to the role of the professional nurse?
7
Research Design
• Randomized • Controlled • Large-‐scale • Mul*-‐site • Longitudinal • Follow-‐up component
8
Study Groups
9
• Tradi*onal clinical experiences as usual (up to 10% simula*on) Control Group
• 25% of clinical *me spent in simula*on • 75% spent in tradi*onal clinical 25% Group
• 50% of clinical *me spent in simula*on • 50% of *me in tradi*onal clinical 50% Group
10
Courses • Fundamentals of Nursing • Medical-‐Surgical Nursing • Advanced Medical-‐Surgical Nursing • Maternal-‐newborn Nursing • Pediatrics • Mental Health Nursing • Community Health Nursing
11
INSTRUMENTS
12
Instruments • Nursing knowledge
ATI Content Mastery Series® • Clinical competency
Creighton Competency Evalua*on Instrument (CCEI) New Graduate Nurse Performance Survey Na*onal Council Licensure Exam® (NCLEX)
• Global assessment of clinical competency & readiness for prac*ce
13
Instruments • Learning needs met
Clinical Learning Environment Comparison Survey (CLECS)
• Cri*cal thinking Cri*cal Thinking Diagnos*c
• Acclima*on to the role of the RN Lee first nursing posi*on, pa*ent loads, charge nurse responsibili*es, workplace stress
14
15
Interven*on: Simula*on as used in the study
For the purpose of this study, simula*on includes • scenario based simula*on using high and medium
fidelity manikins • standardized pa*ents • role playing • computer based cri*cal thinking simula*ons • Skills sta*ons
16
Simula*on day
• En*re clinical group reported to the sim lab with clinical instructor
• Students were assigned to two nursing roles • Clinical instructor completed CCEI ra*ngs
Simula*on curriculum • Needed a standardized simula*on curriculum so that all 10 schools were doing the same thing for the study
Determining the curriculum • Select group of experts • Itera*ve process • Consensus of opinion
3 round Delphi study used
• Round 1: Survey • Round 2: Refine the list • Round 3: Topics ranked
Round 1: Survey • Link to electronic survey distributed to 2 nursing simula*on communi*es
• Link ac*ve for 4 weeks October-‐November 2010
Round 2: Refine the list of topics
Round 3: Item ranking
• Lists distributed back to the 10 par*cipa*ng schools • Course faculty were asked to rate each of the items as: – most important – important – good content to cover if there’s *me
23
24
THE SAMPLE
25
Study Par*cipants (Fall 2011) • Consented: 847
Started study Completed study Control 268 218 25% group 293 236 50% group 286 212
• Female: 86% • White: 84% • Age: 26.3 years (SD 8.0, range 18-‐60) • Experience as nursing assistant: 16%
26
Ajri*on Overall Control group 25% group 50% group
Number of students randomized 847 268 293 286
Number of students comple*ng the study 666 218 236 212
Rate of Comple,on 78.6% 81.3% 80.5% 74.1%
Number of students who failed a course during the study 66 25 22 19
Rate of Failure 7.8% 9.3% 7.5% 6.6%
Number of students who withdrew or were withdrawn from the study for any reason
115 25 35 55
Rate of Withdrawal 13.6% 9.3% 11.9% 19.2%**
27
FINDINGS
28
Research Ques*on 1
Does subs*tu*ng clinical hours with 25% and 50% simula*on impact educa*onal outcomes (knowledge, clinical competency, cri*cal thinking and readiness for prac*ce) assessed at the end of the undergraduate nursing program?
29
Knowledge-‐End of Program ATI Comprehensive Predictor
30
69.1% 69.5% 70.1%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Total Score
p=0.478
Clinical Competency-‐End of Program Preceptor Ra*ngs New Graduate Nurse Performance Survey (1-‐6 scale)
Control group (n=155)
25% group (n=171)
50% group (n=136)
Effect size
P value
Mean SD Mean SD Mean SD
Clinical Knowledge 5.12 0.73 5.18 0.60 5.09 0.72 0.14 0.481
Technical Skills 5.06 0.76 5.09 0.64 5.01 0.86 0.11 0.659
Cri*cal Thinking 5.11 0.72 5.06 0.71 5.03 0.88 0.10 0.668
Communica*on 5.30 0.65 5.34 0.65 5.24 0.87 0.13 0.478
Professionalism 5.38 0.69 5.47 0.61 5.39 0.85 0.14 0.432
Management of Responsibili*es
5.22 0.71 5.20 0.70 5.17 0.85 0.06 0.849
31
1=lowest ra*ng, 6=highest ra*ng
Cri*cal Thinking-‐End of Program Preceptor Ra*ngs
CriTcal Thinking DiagnosTc (1-‐6 scale)
Control group (n=155)
25% group (n=171)
50% group (n=136)
Effect size
P value
Mean SD Mean SD Mean SD
Problem Recogni*on 4.97 0.70 5.07 0.65 5.02 0.75 0.15 0.494
Clinical Decision Making 5.09 0.60 5.18 0.61 5.12 0.81 0.15 0.469
Priori*za*on 5.14 0.66 5.08 0.63 5.03 0.77 0.15 0.418
Clinical Implementa*on 5.10 0.61 5.19 0.60 5.10 0.76 0.15 0.361
Reflec*on 5.13 0.64 5.23 0.59 5.15 0.78 0.16 0.318
32
1=lowest ra*ng, 6=highest ra*ng
Overall Clinical Competency-‐EOP Preceptor Ra*ngs
Global assessment of clinical competency & readiness for pracTce (1-‐10 scale)
Control group (n=156)
25% group (n=168)
50% group (n=135)
Effect size P value
Mean SD Mean SD Mean SD
Overall ra*ng 8.20 1.34 8.29 1.48 8.34 1.44 0.10 0.688
33
1=lowest ra*ng, 10=highest ra*ng
Research Ques*on 2
Are there course by course differences in nursing knowledge, clinical competency, and percep*on of learning needs being met among undergraduate students when tradi*onal clinical hours are subs*tuted with 25% and 50% simula*on?
34
Fundamentals of Nursing Knowledge
35
68.1% 67.3% 68.6%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Total Score
ATI Assessment Total Score (n=800)
p=0.155
Fundamentals of Nursing Clinical Competency
36
0.0
20.0
40.0
60.0
80.0
100.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14
% of items s
cored as Com
petent
# of Weeks
CCEI Scores: Clinical Se\ng
Control group
25% group
50% group
Medical-‐Surgical Nursing Clinical Competency
37
0.0
20.0
40.0
60.0
80.0
100.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
% of items s
cored as Com
petent
# of Weeks
CCEI Scores: Clinical Environment
Control Group
25% Group
50% Group
Advanced Medical-‐Surgical Nursing Knowledge
38
62.7% 64.1% 65.5%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Total Score
ATI Assessment Total Score (n=683)
p=0.005
Advanced Medical-‐Surgical Nursing Clinical Competency
39
0.0
20.0
40.0
60.0
80.0
100.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14
% of items S
cored as Com
petent
# of Weeks
CCEI Scores: Clinical Se\ng
Control Group
25% Group
50% Group
Maternal-‐Newborn Nursing Knowledge
40
68.4% 69.2% 71.1%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Total Score
ATI Assessment Total Score (n=680)
p=0.011
Maternal-‐Newborn Nursing Clinical Competency
41
0.0
20.0
40.0
60.0
80.0
100.0
1 2 3 4 5 6 7 8 9 10 11 12 13
% of items s
cored as Com
petent
# of Weeks
CCEI Scores: Clinical Se\ng
Control Group
25% Group
50% Group
Pediatric Nursing Knowledge
42
63.7% 65.0% 67.1%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Total Score
p=0.002
ATI Assessment Total Score (n=620)
Pediatric Nursing Clinical Competency
43
0.0
20.0
40.0
60.0
80.0
100.0
1 2 3 4 5 6 7 8 9 10 11
% of items s
cored as Com
petent
# of Weeks
CCEI Scores: Clinical Se\ng
Control Group
25% Group
50% Group
Mental Health Nursing Knowledge
44
63.4% 65.2% 66.3%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Total Score
ATI Assessment Total Score (n=633)
p=0.011
Mental Health Nursing Clinical Competency
45
0.0
20.0
40.0
60.0
80.0
100.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
% of items s
cored as Com
petent
# of Weeks
CCEI Scores: Clinical Se\ng
Control Group
25% Group
50% Group
Community Health Nursing Knowledge
46
65.5% 65.5% 67.1%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Total Score
ATI Assessment Total Score (n=252)
p=0.387
Community Health Nursing Clinical Competency
47
0.0
20.0
40.0
60.0
80.0
100.0
1 2 3 4 5 6 7 8 9
% of items S
cored as Com
petent
# of Weeks
CCEI Scores: Clinical Se\ng
Control Group
25% Group
50% Group
Learning Needs Met-‐End of Program Clinical Environment Comparison Survey (CLECS)
Control group 25% group 50% group
n Mean SD Effect Size n Mean SD Effect
Size n Mean SD Effect Size
Tradi*onal 197 3.50 0.42 1.23
202 3.41 0.41 0.28
187 3.26 0.53 0.57
Simula*on 174 2.82 0.67 202 3.28 0.51 187 3.54 0.45
48
Scale 1-‐4 1=Learning needs not met 4=Learning needs well met
Research Ques*on 3
Are there differences in first-‐*me NCLEX pass rates between students that were randomized into a control group, 25% and 50% of tradi*onal clinical subs*tuted with simula*on?
49
NCLEX May-‐December 2013
50
88.4% 85.5% 87.1%
0%
20%
40%
60%
80%
100%
Control group 25% group 50% group
Pass Rate
NCLEX First Time Pass Rates
p=0.737
PART II: FOLLOW UP STUDY
51
Follow-‐up Surveys • Completed aeer start date: 6 weeks 3 months 6 months
52
Follow-‐up Study Response Rate
• 575 students agreed to be contacted for follow-‐up surveys (86.3% of study completers)
• 379 new graduates provided their start date informa*on (65.9% of those who agreed to follow-‐up study)
• 355 completed at least one follow-‐up survey
53
Study Part II Research Ques*on 1 Are there differences in clinical competency, cri*cal thinking and readiness for prac*ce among the new graduate nurses from the three study groups?
54
Clinical Competency 6 Month Manager Ra*ngs New Graduate Nurse Performance Survey (1-‐6 scale)
Control group (n=72)
25% group (n=86)
50% group (n=84)
Effect size
P value
Mean SD Mean SD Mean SD
Clinical Knowledge 5.21 0.63 5.07 0.92 5.21 0.66 0.17 0.376
Technical Skills 5.28 0.65 5.09 0.95 5.19 0.65 0.23 0.325
Cri*cal Thinking 5.11 0.78 5.06 0.92 5.15 0.72 0.11 0.741
Communica*on 5.40 0.66 5.22 0.95 5.42 0.70 0.24 0.203
Professionalism 5.54 0.58 5.30 0.95 5.50 0.65 0.30 0.096 Management of
Responsibili*es 5.26 0.73 5.13 0.94 5.32 0.73 0.23 0.284
55
1=lowest ra*ng, 6=highest ra*ng
Cri*cal Thinking 6 Month Manager Ra*ngs
CriTcal Thinking DiagnosTc (1-‐6 scale)
Control group (n=72)
25% group (n=85) 50% group (n=84) Effect size
P value
Mean SD Mean SD Mean SD
Problem Recogni*on 5.36 0.60 5.17 0.88 5.28 0.57 0.25 0.251
Clinical Decision Making 5.40 0.53 5.28 0.88 5.36 0.53 0.16 0.491
Priori*za*on 5.36 0.68 5.25 0.90 5.35 0.57 0.14 0.559
Clinical Implementa*on 5.40 0.49 5.21 0.92 5.38 0.55 0.25 0.116
Reflec*on 5.42 0.53 5.26 0.92 5.39 0.58 0.21 0.325
56
1=lowest ra*ng, 6=highest ra*ng
Overall Clinical Competency 6 Month Manager Ra*ngs
Global assessment of clinical competency & readiness for pracTce (1-‐10 scale)
Control group (n=72)
25% group (n=86)
50% group (n=84)
Effect size P value
Mean SD Mean SD Mean SD
Overall ra*ng 8.60 1.37 8.36 1.46 8.55 1.16 0.16 0.527
57
1=lowest ra*ng, 10=highest ra*ng
Study Part II Research Ques*on 2
Are there differences among new graduates from the three study groups in acclima*on to the role of the professional nurse?
58
Prepara*on for Prac*ce 6 Week Survey
Control group (n=68) 25% group (n=99) 50% group
(n=99) Effect size P value
freq % freq % freq %
0.16 0.030**
Very well prepared 15 22% 20 20% 36 36%
Quite a bit prepared 25 37% 42 42% 36 36%
Somewhat prepared 26 38% 37 37% 27 27%
Not at all prepared 2 3% 0 0 0 0
6 Month Survey Control group
(n=98) 25% group (n=108) 50% group (n=109)
0.11 0.261 Very well prepared 14 14% 21 19% 29 27%
Quite a bit prepared 44 45% 40 37% 46 42%
Somewhat prepared 38 39% 46 43% 33 30%
Not at all prepared 2 2% 1 1% 1 1%
59
Lee First Nursing Posi*on 6 Month Survey
Control group 25% group 50% group
freq % freq % freq % Effect size
P value
Lee first posi*on 5 7.2% 9 7.3% 11 9.1% 0.07 0.462
60
Pa*ent Care Assignments 6 Month Survey
Control group (n=88)
25% group (n=106)
50% group (n=105)
freq % freq % freq % Effect size P value
Not challenging enough 4 5% 5 5% 5 5%
0.01 0.999 Just right 73 83% 89 84% 87 83%
Too challenging 11 13% 12 11% 13 12%
61
Workplace Stress
62
3.7% 4.4% 4.0% 3.0% 2.6% 3.8% 3.6% 3.0% 3.4% 3.8% 1.9%
32.3% 26.5% 36.6% 32.0%
25.2% 30.4% 26.4% 19.8% 19.7% 11.4%
25.5% 21.0%
50.9% 52.9%
47.5% 53.0% 54.7% 44.6%
54.7% 63.1% 63.5% 67.1%
57.6% 66.7%
13.0% 16.2% 11.9% 12.0% 17.5% 25.0%
15.1% 13.5% 13.7% 18.2% 13.2% 10.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Total Contol Group
25% Group
50% Group
Total Contol Group
25% Group
50% Group
Total Contol Group
25% Group
50% Group
Strongly disagree Disagree Agree Strongly agree
I am experiencing stress at work 6 week 3 month 6 month
Workplace Stress
63
71.0% 72.1% 72.3% 69.0% 63.1% 57.6% 63.2% 67.6% 56.2%
47.7% 55.7%
63.8%
19.7% 20.6% 16.8% 22.0% 24.3%
25.0% 23.6%
24.3% 32.4%
39.8% 30.2% 28.6%
7.4% 5.9% 6.9% 9.0% 9.1% 12.0% 11.3% 4.5% 8.4% 8.0% 11.3%
5.7% 1.9% 1.5% 4.0% 3.6% 5.4% 1.9% 3.6% 3.0% 4.6% 2.8% 1.9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Total Contol Group
25% Group
50% Group
Total Contol Group
25% Group
50% Group
Total Contol Group
25% Group
50% Group
Almost never Some*mes Oeen Almost always
In the last week, I felt expectaTons of me were unrealisTc 6 week 3 month 6 month
Limita*ons
• Par*cipa*ng schools were not randomly selected • Only used one method of conduc*ng simula*ons • Quality of the clinical experiences was not assessed
64
Limita*ons
• End of program preceptors and nurse managers not blinded as to study group
• Distribu*on of end of program and nurse manager surveys relied on the study par*cipants
65
Conclusions
1. Up to 50% simula*on can be effec*vely subs*tuted for tradi*onal clinical experience in all core courses across the prelicensure nursing curriculum.
2. 50% simula*on can be effec*vely used in various program types, in different geographic areas in urban and rural seqngs with good educa*onal outcomes.
66
Conclusions 3. NCLEX pass rates were unaffected by the
subs*tu*on of simula*on throughout the curriculum.
4. All three groups were equally prepared for entry into prac*ce as a new graduate RN.
5. Policy decisions regarding the use and amount of simula*on in nursing needs to be dependent upon the u*liza*on of best prac*ces in simula*on.
67
Qualifiers
These results were achieved using: • INACSL Standards of Best Prac*ce • High quality simula*ons • Debriefing method grounded in educa*onal theory • Trained and dedicated simula*on faculty
68
Recommenda*ons for BONs
• Formally trained faculty in simula*on pedagogy • Use of theory-‐based debriefing methods using subject majer experts
• Adequate numbers of simula*on faculty to support the learners
• Equipment and supplies to create a realis*c environment
69
Summary
70