![Page 1: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/1.jpg)
Systems Thinking to Improve Student Outcomes: Introduction to Improvement Science
NYCOSS Thought Leadership SummitFairport, New YorkMarch 22‐23, 2018
v6
![Page 2: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/2.jpg)
2
Agenda Outline
• Introductions (10 minutes)• Improvement Science Overview (20 minutes)• Small Group Discussions (25 minutes)• Large Group Share‐out (15 minutes)• Take‐aways & Resources (10 minutes)• Q & A (10 minutes)
![Page 3: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/3.jpg)
3
WestEd overview
• Making a quality difference in the lives of children, youth, and adults.
• WestEd is a research, development, and service agency that works with education and other communities to promote excellence, achieve equity, and improve learning for children, youth, and adults.
![Page 4: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/4.jpg)
4
Introductions
Terry Hofer, Ed.D.Director, New York School & District Services
Tran Keys, Ph.D.Senior Researcher
Joe SassoneDevelopment Director, School & District Services
Facilitator Presenters
![Page 5: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/5.jpg)
5
Show of fingers – How familiar are you with Improvement Science?
1‐ Brand new2‐ Familiarwith key concepts and tools3‐ Experience applying concepts and tools4‐ Experience leading and/or coaching others5‐ “I am an Improvement Science expert!”
![Page 6: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/6.jpg)
6
Learning Objectives
Participants will understand…1. Three key ideas behind an organizational learning
approach to improvement2. Components of a learning system3. Improvement Science = Continuous Improvement
![Page 7: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/7.jpg)
7
Flying
On average, how many FAA monitored flights happened each day in 2016?1) 0 – 10,0002) 10,000 – 20,0003) 20,001 – 30,0004) 30,001 – 40,0005) More than 40,000
![Page 8: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/8.jpg)
8
Flying
On average, how many FAA monitored flights happened each day in 2016?1) 0 – 10,0002) 10,000 – 20,0003) 20,001 – 30,0004) 30,001 – 40,0005) More than 40,000
42,700 average daily flights handled by the FAA in 2016.
![Page 9: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/9.jpg)
9
Flying, cont’d
In 2016, how many people flew on FAA monitored flights (Millions)1) 0 – 250 M2) 250 – 500 M3) 500 – 750 M4) 750 – 1,000 M5) More than a billion passengers
![Page 10: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/10.jpg)
10
Flying, cont’d
In 2017, how many people flew on FAA monitored flights (Millions)1) 0 – 250 M2) 250 – 500 M3) 500 – 750 M4) 750 – 1,000 M5) More than a billion passengers
In 2017, about 841 million people flew on commercial airlines in the United States.
![Page 11: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/11.jpg)
11
Flying, cont’d
How many people died in commercial plane crashes last year in the United States?1) 02) 1‐103) 11‐1004) More than 100
![Page 12: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/12.jpg)
12
Flying, cont’d
How many people died in commercial plane crashes last year in the United States?1) 02) 1‐103) 11‐1004) More than 100
![Page 13: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/13.jpg)
13
Why is flying so safe?
Thoughts?
![Page 14: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/14.jpg)
14
Why is flying so safe?
RESEARCH ACCOUNTABILITY LEARNING
![Page 15: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/15.jpg)
15
Why is flying so safe?
RESEARCH ACCOUNTABILITY LEARNING
![Page 16: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/16.jpg)
16
Why is flying so safe?
RESEARCH ACCOUNTABILITY LEARNING
![Page 17: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/17.jpg)
17
Why is flying so safe?
RESEARCH ACCOUNTABILITY LEARNING
![Page 18: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/18.jpg)
18
Reading
Approximately how many fourth graders are there in the United States?1) 0 – 1,500,0002) 1,500,001 – 3,000,0003) 3,000,001 – 4,500,0004) 4,500,001 – 6,000,0005) More than 6,000,000
![Page 19: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/19.jpg)
19
Reading
Approximately how many fourth graders are there in the United States?1) 0 – 1,500,0002) 1,500,001 – 3,000,0003) 3,000,001 – 4,500,0004) 4,500,001 – 6,000,0005) More than 6,000,000
~3,840,000fourthgraders
![Page 20: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/20.jpg)
20
Reading, cont’d
In 2017, according to standardized assessments, how many can read at or above grade level?1) 0 – 20 percent2) 21 – 40 percent3) 41 – 60 percent4) 61 – 80 percent5) 81 – 100 percent
![Page 21: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/21.jpg)
21
Reading, cont’d
In 2017, according to standardized assessments, how many can read at or above grade level?1) 0 – 20 percent2) 21 – 40 percent3) 41 – 60 percent4) 61 – 80 percent5) 81 – 100 percent
~35%
![Page 22: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/22.jpg)
22
Reading –Why only 35%?
RESEARCH ACCOUNTABILITY LEARNING
![Page 23: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/23.jpg)
23
Reading –Why only 35%?
RESEARCH ACCOUNTABILITY LEARNING
![Page 24: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/24.jpg)
24
Three Ways Data Support ImprovementPrimary Audience Purpose
MeasurementCriteria
Research
• Scientific community
• Policymakers• Decision makers
• New knowledge, irrespective of applicability
• Many• Complex collection
Accountability• Parents• Students• Taxpayers
• Basis for choice• Reassurance• Spur for change
• Very few• Complex collection
Organizational Learning
• Teachers• Principals • District Leaders• Managers
• Understanding of process or student learning
• Motivation and focus• Baseline• Evaluation of changes
• Few• Easy to collect• Frequent
![Page 25: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/25.jpg)
25
Three Ways Data Support ImprovementPrimary Audience Purpose
MeasurementCriteria
Research
• Scientific community
• Policymakers• Decision makers
• New knowledge, irrespective of applicability
• Many• Complex collection
Accountability• Parents• Students• Taxpayers
• Basis for choice• Reassurance• Spur for change
• Very few• Complex collection
Organizational Learning
• Teachers• Principals • District Leaders• Managers
• Understanding of process or student learning
• Motivation and focus• Baseline• Evaluation of changes
• Few• Easy to collect• Frequent
![Page 26: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/26.jpg)
26
How does organizationallearning lead to improvement?
Three key ideas…
(Adapted from Berwick, 1996)
![Page 27: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/27.jpg)
27
All improvement begins with dissatisfaction with the status quo.
![Page 28: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/28.jpg)
28
Every system is perfectly designed to get the results it gets.
![Page 29: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/29.jpg)
29
All improvement requires change, but not every change is an improvement.
![Page 30: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/30.jpg)
30
How does organizational learning lead to improvement?
1. All improvement begins with dissatisfaction with the status quo.
2. Every system is perfectly designed to get the results it gets.
3. All improvement requires change, but not every change is an improvement.
![Page 31: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/31.jpg)
31
Example: Reading Recovery
Evidence‐based intervention
Huge experiment (141 schools)
Results after year one of a randomized field trial (i3)
Example adapted from the Carnegie Foundation for the Advancement of Teaching
![Page 32: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/32.jpg)
32
![Page 33: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/33.jpg)
33
![Page 34: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/34.jpg)
34
![Page 35: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/35.jpg)
35
How does organizational learning lead to improvement?
To achieve an ambitious improvement goal, you need robust mechanisms for learning
that influence the processes, structures, and norms of the organization.
![Page 36: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/36.jpg)
36
Goal of Improvement: Replicate Positive Results over Diverse Contexts
![Page 37: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/37.jpg)
37
The Model for Improvement
![Page 38: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/38.jpg)
38
Working Theory
Disciplined Testing Data
Components of a Learning System
![Page 39: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/39.jpg)
39
Small group discussions (25 minutes)
Outline a problem of practice in your district that could benefit from applying an Improvement Science methodology?
Review: How does learning lead to improvement?1. All improvement begins with dissatisfaction with the status quo.2. Every system is perfectly designed to get the results it gets.3. All improvement requires change, but not every change is an
improvement.
![Page 40: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/40.jpg)
40
Large Group Share‐out (15 minutes)
![Page 41: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/41.jpg)
41
Take‐aways
![Page 42: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/42.jpg)
42
![Page 43: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/43.jpg)
43
![Page 44: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/44.jpg)
44
![Page 45: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/45.jpg)
45
Improvement Science
![Page 46: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/46.jpg)
46
Improvement Science =Continuous Improvement
• Improvement Science supports the education field to “get better at getting better”
• Continuous improvement entails ongoing cycles of analysis and adjustment of practice involving all stakeholders
• Data‐based analysis and decision making is the foundation of continuous improvement
![Page 47: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/47.jpg)
47
Resources
![Page 48: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/48.jpg)
48
WestEdmain page
![Page 49: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/49.jpg)
49
Resources, cont’d
• Carnegie Foundation for the Advancement of Teaching https://www.carnegiefoundation.org/
![Page 50: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/50.jpg)
50
Thank you!
• Carnegie Foundation for the Advancement of Teaching• Jonathan Dolle, WestEd• Sola Takahashi, WestEd
![Page 51: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/51.jpg)
51
Thank you!
Questions?
![Page 52: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/52.jpg)
52
Contact Information
Terry Hofer, Ed.D.: [email protected], New York School and District Services
Tran Keys, Ph.D.: [email protected] Researcher
Joe Sassone: [email protected] Director, School and District Services
52
![Page 53: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/53.jpg)
53
Backup Slides
![Page 54: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/54.jpg)
54 54
![Page 55: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/55.jpg)
55
Engine for Learning: The PDSA Cycle
PLAN• What’s your change?• What’s your prediction?
• Plan to conduct test
DO• Execute test• Collect data, documentobservations
STUDY• Compare results to prediction
• What did you learn?
ACT• Next steps: Adapt, adopt, abandon
![Page 56: Systems Thinking to Improve Introduction to Science](https://reader031.vdocuments.us/reader031/viewer/2022013018/61d14ca58a92277f844a25fa/html5/thumbnails/56.jpg)
56
Practical Measurement
Practical measures are measures that act as sensing mechanisms at the level at which work is carried out. They are “practical” in that they can be collected, analyzed, and used within the daily work lives of practitioners. They are also “practical” in that they reflect practice. Practical measures are used to identify improvement targets and to learn continuously whether the changes we introduce are improvements.