![Page 1: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/1.jpg)
Comparing the Effects of Two Versions of Professional
Development on Science Curriculum
Implementation and Scaling-UpSession 43.030
April 13, 2005American Educational Research Association Annual Meeting
Montreal, Canada
![Page 2: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/2.jpg)
Introduction and Overview
Paul R. Brandon
Curriculum Research & Development GroupUniversity of Hawai‘i at Mānoa
![Page 3: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/3.jpg)
3
Project Purpose• A randomized study of the effects of
variations in professional development (PD) on – program implementation– student achievement– scale-up
• First phase (ending 2/28/06) to prepare for five-year second phase (if funded)
![Page 4: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/4.jpg)
4
FAST
• The project compares two versions of teachers’ PD for Foundational Approaches in Science Teaching (FAST).– An award-winning middle-school inquiry-
based science program– Shown to positively affect achievement and
other student outcomes
![Page 5: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/5.jpg)
5
Phase-I Project Tasks
• Prepare the 5-day PD institute with electronic resource and on-line course
• Prepare instruments for the second phase:– Teacher questionnaire– Teacher log– Observation procedures– Student assessments
![Page 6: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/6.jpg)
6
Focus on Span of PD
• Traditionally, FAST PD is delivered in a 10-day institute.
• We will compare it with an alternative 5-day institute followed by an online course, with an electronic multimedia resource.
![Page 7: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/7.jpg)
7
Phase-II Hypothesis
The 5-day inquiry-based science training institute, followed by an on-line university-credit
course with computer-based multimedia, will have more favorable outcomes than a 10-day
institute without the follow-up university course.
![Page 8: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/8.jpg)
8
Examining the effects of PD on:
• levels of classroom implementation
• intensiveness and extensiveness of long-term use of inquiry-based science (i.e., scale-up)
• levels of student achievement
![Page 9: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/9.jpg)
9
Experimental Design
• Randomized cluster sample, with schools (N = 80) as clusters
• One teacher per school
• Even if attrition is 25%, statistical power will be .80.
![Page 10: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/10.jpg)
10
Merit of the Study
• Addresses at least five deficits in the literature:– Few studies on effects of PD time span– Few studies about using technology in PD– Few randomized studies– Few studies of the effects of PD on student
learning– Few, if any studies, of inquiry science PD
![Page 11: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/11.jpg)
11
Order of Presentations
• Gray, Nguyen, & Speitel: Description of PD • Brandon: Log and questionnaire development
and questionnaire validation• Taum: Observation guide development• Ayala: Student assessment development • Lawton: Comparison of early effects of the two
versions of PD
![Page 12: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/12.jpg)
Introduction and Overview
Paul R. Brandon
Curriculum Research & Development GroupUniversity of Hawai‘i at Mānoa
![Page 13: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/13.jpg)
Developing and Implementing an Alternative Version of FAST
Professional Development
Mary E. GrayThanhTruc T. Nguyen
Thomas W. SpeitelUniversity of Hawai‛i at Mānoa
![Page 14: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/14.jpg)
14
FAST
• Foundational Approaches in Science Teaching
• Inquiry-based, middle school science
• Physical, biological, and earth sciences
• Exemplary program (USDOE, 2001)
![Page 15: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/15.jpg)
15
FAST Professional Development
• All FAST teachers must participate in PD– Certification required to
purchase science curricula
• Content• Inquiry• Standards• Assessment• Classroom organization
and management• Safety
![Page 16: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/16.jpg)
16
The ChallengeTo develop and implement an alternative version of FAST I professional development
Year Round Schedules
![Page 17: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/17.jpg)
17
Considerations
• Current needs of science teachers
• Future possibilities (emerging technologies)
• Unique challenges (the dynamic nature of inquiry)
![Page 18: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/18.jpg)
18
Review of LiteratureProfessional Development
On-Line LearningInquiry Science
![Page 19: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/19.jpg)
19
Review of Literature Professional Development
• Strong subject area content (Kennedy, 1999)
• Focus on higher-order teaching strategies (Porter, Garet, Desimone, Yoon, & Birman, 2000)
– Collaborative, Same subject-grade-school, Active learning, Consistent with teacher goals
![Page 20: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/20.jpg)
20
Review of Literature Professional Development
• Quantity linked to improvements (Radford, 1998; Supovitz, Mayer, & Kahle, 2000; Fishman, Marx, Best, & Revital, 2003)
– science content knowledge, process skills, and attitudes
• Quantity linked to standards based teaching (Supovitz & Turner,2000)
• Problems with researching inquiry-based teaching and student learning (Fishman et al., 2003)
![Page 21: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/21.jpg)
21
Review of Literature On-line Learning
• 3 million adult learners on-line (Waitts & Lewis, 2003)
• Convenience and increased flexibility (Hülsmann,1999)
• Evidence of implementation of various pedagogical approaches (Schlager & Schank, 1997)
![Page 22: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/22.jpg)
22
Review of Literature On-line Learning
• Quantifiable learning outcomes were not significantly linked to technology adoption (Jones and Paolucci,1998).
• Focus on motivation, skills and knowledge, self-directed learning, interactive competence, and technology skills (Kabilan, 2004)
• Increased efficacy and self-perception in teachers (Huai, Braden, White, & Elliot, 2003)
![Page 23: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/23.jpg)
23
FASTPro
![Page 24: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/24.jpg)
24
How are the traditional PD and FASTPro different?
• Two-weeks in duration, face-to-face
• One-week in duration, face-to-face
• Conduct 43 investigations
• Extensive modeling and practicing time
• Conduct 19 investigations
• Electronic resource• WebCT-based
learning community
![Page 25: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/25.jpg)
InquiryInquiry
Teaching and learning science through inquiry is a new experience for many teachers and requires a significant change in attitude and behavior.
![Page 26: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/26.jpg)
26
FASTPro – Addressing Inquiry• Strategies (Loucks-Horsley, Hewson,
Love, & Stiles,1998)
– Examples• Immersion in inquiry
into science• Coaching and
mentoring • Technology for
professional learning
![Page 27: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/27.jpg)
27
FASTPro – Addressing Inquiry• Change (Hord, Rutherford, Huling-Austin, & Hall,1987; Guskey,
2000)
– takes time and persistence• awkwardness and frustration expected
– as teacher’s progress through a change process, their needs for support and assistance change
• Optimal Mix (Guskey, 2000) – A combination of teacher, program and change
agent that will help create a positive relationship for PD to be effective
![Page 28: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/28.jpg)
28
FASTPro
![Page 29: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/29.jpg)
29
FASTStart• One week face-to-face
institute
• Teachers conduct nearly half of actual student investigations
• Use inquiry methodology including modeling, discussion, assessment, and practice
![Page 30: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/30.jpg)
30
![Page 31: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/31.jpg)
31
FASTeR
• Observe interactions of students and teachers
• View and reflect upon investigations not experienced
• See detailed step-by-step procedure suggestions
• Movies and animations
• Web interface– Quicktime and Flash
plugins
• DVD-ROM medium
![Page 32: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/32.jpg)
32
Examples from FASTeR
![Page 33: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/33.jpg)
33
Examples from FASTeR• Slideshow 9, Density
and the Cartesian Diver
• Slideshow 22, Identifying Unknown Substances
![Page 34: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/34.jpg)
34
![Page 35: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/35.jpg)
35
FASTForward
![Page 36: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/36.jpg)
36
Formative Findings
• Suggest that teachers were able to implement successfully and share teaching practice
• Added utility, focused and meaningful to teachers’ own environment
![Page 37: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/37.jpg)
37
Future considerations• Infuse technological
aspects into the FASTStart face-to-face experience
• Address credit equivalencies to enable more robust activity requirements
• Expand to include FAQ’s, indexing, and rich descriptions
• Expand to include a Website as well as DVD-ROM
• Automate some computer assisted instruction and feedback mechanisms
![Page 38: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/38.jpg)
38
Reflections…
![Page 39: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/39.jpg)
39
Reflections…• Teamwork and
flexibility
• Coordination of equipment and resources
![Page 40: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/40.jpg)
40
Future Considerations• Building capacity
• Application to other CRDG quality educational programs
![Page 41: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/41.jpg)
41
Mahalo.
![Page 42: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/42.jpg)
Developing and Implementing an Alternative Version of FAST
Professional Development
Mary E. GrayThanhTruc T. Nguyen
Thomas W. SpeitelUniversity of Hawai‛i at Mānoa
![Page 43: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/43.jpg)
Instrument Development for a Study Comparing Two
Versions of Inquiry Science Professional Development
Paul R. Brandon Alice K. H. Taum
University of Hawai‘i at Mānoa
![Page 44: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/44.jpg)
44
Identifying Constructs
• on teaching science with inquiry methods– Reviewed FAST and other inquiry science documents
and worked closely with inquiry science experts.
• on the context within which inquiry-based science is taught.– Reviewed 55 books and key articles on curriculum
indicators and school effectiveness.
![Page 45: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/45.jpg)
45
Developing the Teacher Log
• Purpose: To identify the extent to which teachers implement the key features of inquiry-based science.
• Reviewed the recent literature on logs.
• Kept the instrument short to avoid overburdening teachers (21 items).
• Is completed immediately after finishing each student science investigation.
![Page 46: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/46.jpg)
46
Focus of the Teacher Log
• Instrument addresses topics such as:– students’ questioning behaviors.– the teacher’s use of questioning strategies.– the teacher’s circulation about the classroom.– teacher-led discussions about variations in
the data.
![Page 47: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/47.jpg)
47
Developing the Teacher Questionnaire
• Purposes: – To obtain information about implementation
that is most appropriately collected once a year.
– To collect information on implementation of investigations not covered on logs.
![Page 48: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/48.jpg)
48
Focus of the Questionnaire
• 150 items address topics such as:– implementation of key features.– the investigations taught during the entire
year.– the adequacy of materials, equipment, etc.– teacher demographics.– teacher attitudes toward science.– teacher participation in science activities
outside the classroom
![Page 49: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/49.jpg)
49
Types of Validation Studies to Date
• Two types of questionnaire-data validation studies conducted to date:– analyses addressing the relationship between
the responses on the log and the questionnaire (“concurrent” validity)
– an analysis examining the theoretical model underlying our study (construct validity)
![Page 50: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/50.jpg)
50
Relationship Between Log and Questionnaire Responses
• Compared results on total scores for five items that the two instruments had in common
• Correlation = .54; effect size = .50
![Page 51: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/51.jpg)
51
Examining the Theoretical Model• Second set of analyses examined whether the
expected relationships were found among some questionnaire variables.
• We regressed a variable measuring program implementation on five independent variables measuring:– teachers’ ongoing learning about science.– the resources available for teaching science.– the number of science courses taken.– the number of years have taught K–12 science– teachers’ attitudes toward teaching science
![Page 52: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/52.jpg)
52
Validity Argument• Teachers’ ongoing learning about science
(including learning that occurred in recent PD) should predict implementation more than– science education background.– years of experience teaching science.– attitudes toward teaching science.– classroom resources available.
![Page 53: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/53.jpg)
53
Results of Second Analysis• Teacher learning about science outside the classroom: • beta = .19 (significant at the .05 level)• Teacher attitudes toward teaching science: • beta = .17 (significant)
• Classroom and school resources for teaching science: • beta = .04 (not significant) • Number of university science courses taken: • beta = -.02 (not significant) • Number of years the teacher has taught K–12 science: beta =
-.02 (not significant)
![Page 54: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/54.jpg)
Instrument Development for a Study Comparing Two
Versions of Inquiry Science Professional Development
Paul R. Brandon Alice K. H. Taum
University of Hawai‘i at Mānoa
![Page 55: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/55.jpg)
Coding Teachers in Science Classrooms using the Inquiry Science Observation
Guide
Alice K. H. TaumCurriculum Research & Development Group
University of Hawai‘i at Mānoa
![Page 56: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/56.jpg)
56
Inquiry Science Observation Guide• Introduction and Overview
– FAST, SCUP Project, Purpose and Development of ISOG
• General coding guidelines• Descriptions for each of the six
Activities• Definitions of Activity details
• Code Sheet • Recording Sheet • Reconciling Recording Sheet
![Page 57: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/57.jpg)
57
Development of the Code Sheet
• 1.5 year process
• Collaborative efforts between researchers at the University of Hawai‘i at Mānoa, Stanford University and Sonoma State University; FAST teachers; curriculum developers, and coders
• 36+ revisions
![Page 58: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/58.jpg)
58
Goal of the Code Sheet • Identify strings of activities/teacher
behaviors using the details in columns A, B, C and D. – questioning strategies used – level of engagement between teacher and
students– creating a profile of teachers implementation
of FAST in the classroom through science inquiry
![Page 59: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/59.jpg)
59
Design of the Code Sheet
• 6 activities.
• 66 possible activity details to plug into the 6 activities.
• Multiple variations between an activity and the activity details which describe an activity string of teachers pedagogical practices.
![Page 60: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/60.jpg)
60
Inquiry Science Observation Code Sheet (ISOCS)
Activity Activity details Activity
descriptors
1 - 6 A B C
D
1. Teacher directs student (s) ______A______ to ______B______ _______C______.
1A1. individually 1A2. in a small group
1B1. record 1B2. discuss 1B3. define 1B4. read materials to whole class
1C1. observations 1C2. predictions/ hypotheses 1C3. procedures 1C4. data 1C5. science (concept, vocab words, mechanics of science, etc…)
1D1. with evidence or examples 1D2. does not apply
2. Through ______A______, the teacher ______B_______ science ______C______.
2A1. direct instruction questioning: 2A2a. rhetorical 2A2b. interactive
2B1. introduces or provides an overview of 2B2. reviews/summarizes 2B3. demonstrates 2B4. collects 2B5. compares/contrasts 2B6. clarifies
2C1. concepts(idea) 2C2. procedures(activity) 2C3. tools/equipment 2C4. investigation (science experiment) 2C5. problem 2C6. goal 2C7. -related safety issues 2C8 data (unknown): 2C8a. differences in 2C8b. relationships between 2C8c. quality of 2C8d. analysis of 2C8e. synthesis of 2C8f. evaluation of 2C9. vocabulary words
2D1. new information 2D2. previously learned information 2D3. investigation 2D4. unit 2D5. does not apply
1. Teacher directs student(s)
_____A_______ to
_____B_____ _____C_____.
![Page 61: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/61.jpg)
61
1. Teacher directs student(s) individually (1A1) to discuss (1B2) procedures (1C3).
1. Teacher directs student(s) _____A_____ to _____B______ _____C_____.
![Page 62: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/62.jpg)
62
Inquiry Science Observation Recording Sheet
• Notes the start time of when the “broader” and “other activities” a teacher is engaged in begins
• Allows for comments to facilitate the reconciliation process
• Designed to record multiple activities occurring at the same time
![Page 63: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/63.jpg)
63
Broader and Other Activities– The broader activity captures the larger activity
occurring:Through interactive questioning (2A2b), the teacher introduces or provides an overview of (2B1) science investigation (2C4).
– The other activities capture the details occurring within the broader activity:Teacher questions students through clarifying (3A1a) questioning and responds to student comment (3B1) by repeating (3C2) the comment and probing further (3C6).
![Page 64: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/64.jpg)
64
![Page 65: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/65.jpg)
65
Reconciling Codes• Two coders are paired to compare their
independent codings, identifying any differences between them.
• Differences are then discussed and each coder provides a rationale for his or her selected code.
• When necessary, a review of the DVD is conducted by the paired coders.
• Coding differences are discussed until consensus is reached between coders.
![Page 66: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/66.jpg)
66
Inquiry Science Observation Reconciling Code Sheet
![Page 67: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/67.jpg)
67
Challenges• greatest challenges:
– establishing universal definitions for terms and phrases
• What constitutes a “procedure”? • What is the difference between a science “term” and a
science “concept”?
– standardizing the recording of activity strings using the Recording Sheet
• broader verses other activities
– focusing on the teacher, rather than on the students
![Page 68: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/68.jpg)
68
What Next?• To date we have:
– videotaped 15 teachers;– 140 videotape cassettes have been digitized
to DVDs;– 116 have been quality-checked for audio
clarity and teacher visibility; and – 8 coders are trained and ready to begin
coding!
![Page 69: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/69.jpg)
Coding Teachers in Science Classrooms using the Inquiry Science Observation
Guide
Alice K. H. TaumCurriculum Research & Development Group
University of Hawai‘i at Mānoa
![Page 70: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/70.jpg)
Developing Student Outcome Measures Frameworks
Carlos C. Ayala Sonoma State University
![Page 71: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/71.jpg)
Scaling Up: Student Measures
The Buck Stops HereThe Buck Stops Here
Carlos Ayala
![Page 72: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/72.jpg)
Let’s Talk Charge
Develop student assessment instruments in order to tease out differences between professional development models.
2 Session Pre Test3 Session Post Test Suite
![Page 73: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/73.jpg)
73
Where we expect differences
• Student’s content knowledge• Student’s science inquiry skills• Student’s views of the nature of science• Student’s efficacy toward science• Student’s motivation to learn science
![Page 74: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/74.jpg)
74
Final Assessment Suite
• Pre-test – 30 item multiple-choice/ short answer test (α= .86)– Attitudinal Survey
• Post-test – 30 item multiple-choice/ short answer test– Attitudinal survey– Mānoa River performance assessment
![Page 75: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/75.jpg)
75
![Page 76: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/76.jpg)
76
Knowledge Type Framework
Type of Knowledge Definition Examples Prompt
Declarative Knowing that Concepts & facts “What is
percolation?”
Procedural Knowing how Actions, steps, “How do
measure how much & procedures
water soil will hold?”
Schematic Knowing why Principles & “How does the water
cycle mental models work?”
![Page 77: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/77.jpg)
77
![Page 78: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/78.jpg)
78
Content Validity
• Reviewed curriculum and created content matrices• Curriculum developers parsed content down• Linked matrices to materials• Created test linking items to matrices• Piloted assessments• Talk alouds
![Page 79: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/79.jpg)
79
• As students become more engaged in the FAST curriculum and the teacher more fully implements the curriculum, students will be more proficient at Science Inquiry.– Design and conduct a scientific investigation– Use appropriate tools to gather, analyze and interpret data– Develop descriptions, explanations predictions and models
using evidence
• Targets based on – Duschl’s Transformations in Three Domains– Pottenger’s Deductive Explanatory Inquiry
Student Science Inquiry
NRC. (1996). National Science Education Standards . Washington D.C.: National Academy of the Sciences.
![Page 80: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/80.jpg)
80
Transformation 1Data to Evidence: Deciding if the data are evidence, irrelevant and/or problematic.
Transformation 2: Evidence to patterns or models decisions about selecting tools for identifying patterns or models
Transformation 3:Patterns and models to explanations. Deciding how the patterns or models lead to explanations.
Reformulation: From explanations to new questions: Deciding what next questions to ask and what new data are needed.
Student Science Inquiry
![Page 81: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/81.jpg)
81
![Page 82: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/82.jpg)
82
• As students become more engaged in the FAST curriculum and the teacher more fully implements the curriculum, students will understand that– anyone can be a scientist.– science knowledge is useful.– science knowledge builds over time.– science is creative.
Nature of Science
Lederman, Abd-El-Khalick, Bell and Schwartz (2002) Views of Nature of Science Questionnaire; Toward Valid and Meaningful Assessments of Learner’ Conceptions of the Nature of Science.
![Page 83: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/83.jpg)
83
• As students become more engaged in the FAST curriculum, the greater control they will feel towards science, science investigations and science knowledge.
– I can make accurate measurements during a science investigation.– I can make appropriate predictions about what will happen during a science investigation.
Self Efficacy
Britner, S. and Pajares F., (2001) Self-Efficacy Beliefs, Motivation, Race and Gender in Middle School Science.
![Page 84: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/84.jpg)
84
• As students become more engaged in the FAST curriculum, patterns of motivation may change.
• Dweck Groups (goals and epistemic beliefs)– Mastery Orientation– Ego Orientation– Helpless Orientation
Motivation
Haydel, A., & Roser, R. (2001). On the links between students' motivation patterns and their perceptions of, beliefs about and performance on different types of science achievement, Multidimensional Approach to Achievement
![Page 85: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/85.jpg)
85
Next Steps
• 760 Tests and Surveys completed• Collect post test data• Run analyses• Provide results to group
![Page 86: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/86.jpg)
Developing Student Outcome Measures Frameworks
Carlos C. Ayala Sonoma State University
![Page 87: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/87.jpg)
The Differential Effects of Two Versions of Professional Development on Teachers’ Self-Efficacy to Implement
Inquiry-Based ScienceBrian Lawton
University of Hawai‘i at Mānoa
![Page 88: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/88.jpg)
88
Purposes
• Effects of both versions of PD institutes on teacher self-efficacy to implement program
• Differences between the two versions
• Factors associated with self-efficacy change
![Page 89: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/89.jpg)
89
Data Collection• Qualitative and quantitative methods used• Self-efficacy scale
– Developed to facilitate focus group discussion• Focus groups
– Conducted immediately following the institutes
– Provide in-depth information about changes in self-efficacy• Five stages of inquiry science
1. Introducing new science investigations2. Facilitating valid experimental design3. Facilitating the investigation process4. Constructing meaning5. Linking knowledge to new situations
• Seven teachers from the 10-day and seven teachers from the 5-day participated in the study
![Page 90: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/90.jpg)
90
Data Collection (cont.)
Self-Efficacy Scale
1. STAGE: Introducing new science investigations - my ability to introduce students to new science investigations by reviewing and tying in previous work.
Now Low High ability ability
Before Low Highability ability
1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
![Page 91: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/91.jpg)
91
Data Analysis
• Only presenting focus group results• Analyzed for changes in self-efficacy
– Two categories: • Statements implying an increase in self-efficacy• Statements implying no change in self-efficacy
• Categories analyzed to identify the influencing factors
• The statements and factors were quantified
![Page 92: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/92.jpg)
92
Results, Influencing Factors
• Overall seven factors were identified as influencing self-efficacy.
1 Factors identified as most influential in increasing self-efficacy
2 Factor identified as most associated with no change in self-efficacy
Factors influencing increased self-efficacy
•Content/pedagogical knowledge1
•Program Structure1
•Professional Collaboration
•Experience
Factors influencing no change in self-efficacy
•Practice/feedback2
•Student characteristics
•Experience
•Time issues
![Page 93: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/93.jpg)
93
Results: Changes in Self-Efficacy
Changes in self-efficacy10-day version
5-day version
Statements identified as increases in self-efficacy
11 8
Statements identified as no change in self-efficacy
6 9
![Page 94: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/94.jpg)
94
Discussion
• Group increases in self-efficacy
• Changes in self-efficacy between the groups
• Groups perceived gains in content and pedagogical knowledge
• Importance of 5-day follow-on support
![Page 95: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/95.jpg)
95
Further Study
• A study comparing the groups after the 5-day group has received all their training.
![Page 96: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/96.jpg)
96
Thanks!
![Page 97: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/97.jpg)
The Differential Effects of Two Versions of Professional Development on Teachers’ Self-Efficacy to Implement
Inquiry-Based ScienceBrian Lawton
University of Hawai‘i at Mānoa
![Page 98: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/98.jpg)
Comparing the Effects of Two Versions of Professional
Development on Science Curriculum
Implementation and Scaling-UpSession 43.030
April 13, 2005American Educational Research Association Annual Meeting
Montreal, Canada
![Page 99: Session 43.030 April 13, 2005 American Educational Research Association Annual Meeting](https://reader035.vdocuments.us/reader035/viewer/2022070410/568146af550346895db3cb2e/html5/thumbnails/99.jpg)
99
Questions and Comments• Paul Brandon - Overview• Mary Gray and Truc Nguyen - Description of PD• Paul Brandon - Log and questionnaire development and
questionnaire validation.• Alice Taum –Observation guide development• Carlos Ayala - Student assessment development • Brian Lawton - Comparison of early effects of the two versions
of PD
Paper, Presentation and Contact Info available at:
http://hisii.hawaii.edu/SCUP/research/