wayne state university college of education teac: inquiry brief proposal presentation
DESCRIPTION
Wayne State University College of Education TEAC: Inquiry Brief Proposal Presentation. Presented by the TEAC Advisory Committee April 24 , 2012. Overview. Program Overview Claims & Rationale Methods of Assessment Results, Discussion, Plan Claim 1 Claim 2 Claim 3 Auditor’s Comments. - PowerPoint PPT PresentationTRANSCRIPT
Wayne State University
College of Education
TEAC: Inquiry Brief Proposal Presentation
Presented by the TEAC Advisory Committee
April 24, 2012
Overview Program Overview
Claims & Rationale
Methods of Assessment
Results, Discussion, Plan Claim 1
Claim 2
Claim 3
Auditor’s Comments
Program Overview Background and History
Distinguishing Features of the Program
Requirements for Admissions & Graduation
Demographics of Initial Certification Candidates
Description of Faculty
Logistics of Program
Claims & Rationale Claim 1: Initial Certification
Completers understand subject matter knowledge they will teach (Learning to Learn, Multicultural, Technology).
Claim 2: Initial Certification Completers have pedagogical knowledge (Learning to Learn, Multicultural, Technology).
Claim 3: Initial Certification Completers know how to teach in a caring and effective matter (Learning to Learn, Multicultural, Technology).
Methods of Assessment
Sources of Measure
Surveys (performance assessments, WSU surveys, MDE surveys, observations)
Course Grades (methods & total GPA)
Ratings of Capstone Conversation
Scores on MTTC
Research Design
Reliability and Validity Factors
Claim 1:Subject Matter
Knowledge
Subcommittee: Janet Andrews, Mariane Fahlman, Chavon Jackson, Jo-Ann Snyder, Cheryl Somers,
Initial Certification Completers understand
subject matter knowledge they will teach.
Learn and understand the subject matter they hope to teach.
Competencies & Data Sources
Competencies: 1, 2,
Data Sources:•Unit Plan
•University Supervisor Final Eval. Observation
•Cooperating Teachers Final Eval. Observation
•WSU Student Exit
Survey•MDE Supervisor Survey •GPA
•MTTC
Performance Assessments and Observation Data
Claim 1: Findings
Finding 1 – based on unit plan assessment, 95% of students demonstrated knowledge of subject matter at the proficient or exemplary level
Finding 2 – both cooperating teachers and college supervisors rates 93% proficient or exemplary in knowledge of subject matter
Claim 1: Findings
Finding 3 – all teacher candidates receive a grade of C or better in all subject matter courses.
Finding 4 – all teacher candidates successfully complete the MTTC.
Findings of CCT: LEARNING TO LEARN
Receiving passing grades in 100 % of content area classes is evidence that strategies for learning were at their command
Successful completion of the MTTC test demonstrated their mastery of content material again evidencing their ability to learn
Discussion of Content Knowledge (claim 1)
The data we examined suggests that our candidates have learned and do understand the subject matter when they leave us.
Our data sources for cross-cutting themes appeared to be too narrow.
Additionally we have data we have not examined for cross-cutting themes (e.g. mc course, tech course).
Implications for Content Knowledge (claim 1)
Need better documentation of how students use technology to develop concepts in their discipline.
We need to include other sources that demonstrate evidence of this such as:DIVERSITY:University
Foreign Culture requirement,
COE Multicultural requirement,
COE inclusion requirement,
TECHNOLOGYComputer
literacy requirement,
Technology course in every curriculum
E-PortfoliosBlackboard
LEARNING TO LEARN:
What else can we do?
Claim 2:Knowledge of
Pedagogy
Subcommittee: Oscar Abbott, Abby Butler, Bev Schneider, David Whitin, Phyllis Whitin,
They organize for effective instruction by establishing a safe and orderly environment that is conducive to learning and creative and critical thinking.
They utilize school, community, and technological resources and integrate content across the curriculum.
They use a range of assessment strategies to inform ongoing instruction.
Initial Certification Completers have
pedagogical knowledge:
Competencies & Data Sources
Community Observation
District School Curriculum Analysis
Classroom Management Plan
Lesson Plan Unit Plan Case Study Cooperating
Teacher’s Final Eval Observ.
Univ. Supervisor’s Final Final Eval Observ.
WSU Student Entry Survey
WSU Student Exit Survey
MDE Supervisor Exit Survey
Competencies: 2, 3, 4, 8, 9Data Sources:
Performance Assessments and Observation Data
Percentages are proportions of students rated proficient or higher on the respective competency for each data source. *Indicates assessments given early in the program where a score of basic is considered acceptable.
Competency Performance Assessments Observations
CO* DSC* CMP* LP UP CS CTF USF
2 37% 49% 89% 95% 93% 96%
3 48% 91% 97% 87% 95% 94%
4 42% 90% 95% 95% 96%
8 44% 85% 90% 76% 97% 97%
9 40% 49% 89% 95% 97%
Findings for Pedagogy (Claim 2)
• Higher percentages of proficient or exemplary ratings are found in assessments that take place later in the program.
• Highest ratings (Lesson/Unit Plan) and Cooperating Teacher and Supervisor assessments indicate that candidates exit the program pedagogically well prepared.
• There are lower ratings for CO, DSC, CMP, and CS. CO, DSC, and CMP reflect introductory experiences. CS is assessed later in the program.
Discussion for Pedagogy (Claim 2)
CO, DSC, and CMP, with only 37 – 48% showing proficient and above performance, are indicative of a beginning level understanding when this content is first introduced. A rating of basic is considered satisfactory for these 3 assignments.
Data from 2010-2011 reflect the work of many candidates who were not exposed to the service learning component that is designed to give candidates experience working with children and community organizations.
The low Case Study scores (87% and 76%) are of more concern since it is assessed later in the program.
Discussion for Pedagogy: Cross-Cutting Themes (Claim
2) Learning to Learn is reflected in five performance
assessments and two observations: CO, DSC, CS, LP, UP, and CTF, USF.
Multicultural is reflected in three performance assessments and two observations: DSC, LP, UP and CTF, USF.
Evidence for these two themes is integrated throughout students’ experiences.
Technology is only reflected in two assessments: DSC, LP.
Evidence for Technology theme is thin.
Implications for Pedagogy (claim 2)
Closely monitor assessments with lower ratings, and implement changes to strengthen the program.
These results, particularly for CS, suggest modifications to our program, e.g. to incorporate into methods classes additional experiences involving formulating instructional decisions based on ongoing observational assessments.
Modules developed by SED may be useful to all initial certification candidates in their work on CMP.
Revise assessment rubrics that reflect technology in candidates’ teaching.
Initiate conversations to examine evidence of technology-integrated pedagogy at the program level for each content area.
Claim 3:Caring and Effective Teaching
Subcommittee: Tom Edwards, Sharon Elliott, Sharon Sellers-Clark, Bo Shen, Marshall Zumberg,
Initial Certification Candidates are Caring and
Effective Caring Teaching: Reflects interest
in, and concern for, students.
Effective Teaching: Results in student learning
Competencies & Data Sources
Competencies: 3, 4, 5, 6, 7, 10
Data Sources:
Community Observation (CO), Classroom Management Plan (CMP), Reflective Journal (RJ),Teaching and
Learning Statement (TLS), Lesson Plan (LP),Unit Plan (UP),Case Study (CS).
Percentages are proportions of students rated proficient or higher on the respective competency for each data source.
*Indicates assessments given early in the program where a score of basic is considered acceptable. Note:Community Observation (CO), Classroom Management Plan (CMP), Reflective Journal (RJ), Teaching and Learning Statement (TLS), Lesson Plan (LP), Unit Plan (UP), Case Study (CS)
CompetencyPerformance Assessment
CO* CMP* RJ TLS LP UP CS
3 48% 73% 91% 97% 87%
4 73% 84% 90% 95%
5 47% 87% 88% 95% 85%
6 42% 44% 77% 81% 86% 91% 82%
7 35% 44% 74% 86%
10 39% 82%
Competencies & Data Sources Competencies: 3, 4, 5, 6, 7, 10
Data Sources: (Con’t.)
Capstone Conversation (CC), WSU Student Entry Survey (WSEn), WSU Student Exit Survey (WSEx) MDE Student Exit Survey-MDE (MSEx), MDE Supervisor Survey (MSS), University Supervisor Final Observation
(USF), Cooperating Teacher Final Observation
(CTF).
Survey and Observation Data
Percentages are proportions of students rated proficient or higher on the respective competency for each data source.
*Indicates assessments given early in the program where a score of basic is considered acceptable.
Note: CC=Capstone Conversation; WSEn=Student Entry Survey; WSEx=Student Exit Survey; MSS=MDE Supervisor Survey; USF=University Supervisor Final Evaluation Observation; CTF=Cooperating Teacher Final Evaluation Observation.
Findings for Caring & Effective (Claim 3)
Percentage of proficient and exemplary ratings tends to be higher on assessments that occur later in the program.
Highest percentages of students rated proficient and exemplary on items related to caring and effective teaching occurred in the MSS, USF, and CTF.
There are only small differences in the percentages of students rated proficient and exemplary on the WSEx and MSEx compared to the USF and CTF.
Lowest percentages of proficient and exemplary ratings occurred on the CO and CMP, both of which occur early in the program. (It is also noteworthy that a rating of “basic” is considered satisfactory on those two assessments.)
Findings for Cross-Cutting Themes
in Caring & Effective Teaching Caring & Effective Teaching: Data drawn from 7
performance assessments, 4 surveys, and 2 observations across 6 competencies.
Learning to Learn data was drawn from
Four of the performance assessments (CO, RJ, TLS, LP) across Competencies 7 & 10.
Three (CC, WSEn, WSEx) of the surveys in Competency 6.
Both of the observations in Competency 6 (CTF, USF)
Findings for Cross-Cutting Themes
in Caring & Effective Teaching Caring & Effective Teaching: Data drawn from 7
performance assessments, 4 surveys, and 2 observations across 6 competencies.
Multicultural Perspective data was drawn from
Six of the assessments (CO, RJ, TLS, LP, UP, CS) across Competencies 5, 6, 7, & 10.
All four of the of the surveys across Competencies 5 and 7.
Both of the observations across Competencies 5 and 7
Findings for Cross-Cutting Themes
in Caring & Effective Teaching Caring & Effective Teaching: Data drawn from 7
performance assessments, 4 surveys, and 2 observations across 6 competencies.
Technology data was drawn from
Three of the assessments (TLS, LP, UP) in Competency 4.
Three of the surveys (WSEn, MSS, WSEx) across Competencies 3, 4, and 6.
There was no data addressing issues of caring & effective teaching from either either observation that provided direct evidence of technology.
Findings for Cross-Cutting Themes (CCTs) in Caring & Effective
Teaching
There was data for each of the CCTs embedded in the data for Caring & Effective.
The data that was there supports each theme.
Multi-Cultural Perspective had the most supporting data.
Technology had the least supporting data.
Technology is not addressed directly in either the CTF or USF.
Discussion: Caring & Effective Teaching
Our analysis of the data revealed a clear trend in the growth of the proportion of candidates rated proficient or higher on the performance assessments in areas related to caring & effective teaching.
This analysis suggests that our program completers grow over time in their ability to provide caring and effective teaching.
The vast majority reach proficient or exemplary ratings.
The evidence we have supports the claim that we prepare teachers who are caring & effective.
Discussion: CCTs in Caring & Effective
Regarding the cross-cutting themes, the evidence of a multi-cultural perspective is strongest, because it
comes from more data sources and addresses more of the related competencies.
of learning to learn is adequate ,
of technology is weakest, coming from only a few data sources, one of which is self-reported.
Implications for Caring and Effective Teaching
Revisit the rubrics of those performance assessments with the lowest percentages of proficient and exemplary ratings to ensure that the rubrics are providing the information we seek.
Consider revising the rubrics for all of the performance assessments so they provide more information about our candidates technological proficiency in the service of pedagogy.
Revisit the CMP rubric to learn why no data related to caring & effective teaching from that source supports any cross-cutting theme.
Revise the observation reporting forms to provide information on technology.
Small Group Discussion 1. Do we have the evidence to support these claims?
Why/Why not?
2. What are the strengths of the evidence?
3. What might be missing?
4. Other assessments/tools we should consider to assure: Subject Matter Knowledge Pedagogical Knowledge Caring & Effective Teaching
With the 3 cross cutting themes: Learning to Learn Multicultural Perspectives Technology
Auidtor’s Comments