caep accreditation reporting measures€¦ · edtpa rubric summary task rubric task 1: planning for...
TRANSCRIPT
1
CAEP Accreditation Reporting Measures
The Master of Science in Education degree that leads to dual certifications in Childhood 1-6 and Special Education attained National Recognition
from the Association of Childhood Education (ACEI) in 2015 and National Recognition from the Council for Exceptional Children in 2016.
In 2015, the Selected Improvement Commission of the Council for the Accreditation of Educator Preparation (CAEP) continued MCNY’s NCATE
accreditation, citing no areas for improvement relative to any of the NCATE Standards.
1. Impact on P-12 learning and development: The edTPA performance assessment has been required for certification in New
York State since May 2014. For the past four years, the NYSED has provided a “safety-net” for teacher candidate examinees. If a teacher
candidate submits a technically scored edTPA assessment and fails, for a pathway to approved certification a teacher candidate can
either re-take the edTPA and pass or take and pass the older ATS-W written exam. The edTPA safety net ended on June 30, 2018. With
this choice for the pathway for teacher certification, some MCNY teacher candidates utilized the safety net rather than re-taking and
passing the edTPA.
The following report is a review and analysis of aggregated and disaggregated data from the edTPA performance of a recent cohort of MSED
candidates:
2
edTPA Elementary Education Assessment
Task Rubric
Task 1: Planning for Literacy Instruction and Assessment
Rubric 1: Planning for Literacy Learning
Rubric 2: Planning to Support Varied Student Learning Needs
Rubric 3: Using Knowledge of Students to Inform Teaching and Learning
Rubric 4: Identifying and Supporting Language Demands
Rubric 5: Planning Assessments to Monitor and Support Student Learning
Task 2: Instructing and Engaging Students in Literacy Learning
Rubric 6: Learning Environment
Rubric 7: Engaging Students in Learning
Rubric 8: Deepening Student Learning
Rubric 9: Subject-Specific Pedagogy
Rubric 10: Analyzing Teaching Effectiveness
Task 3: Assessing Students' Literacy Learning
Rubric 11: Analysis of Student Learning
Rubric 12: Providing Feedback to Guide Further Learning
Rubric 13: Student Understanding and Use of Feedback
Rubric 14: Analyzing Students' Language Use and Literacy Learning
Rubric 15: Using Assessment to Inform Instruction
Task 4: Assessing Students' Mathematics Learning
Rubric 19: Analyzing Whole Class Understandings
Rubric 20: Analyzing Individual Student Work Samples
Rubric 21: Using Evidence to Reflect on Teaching
3
Elementary Education edTPA
Table 1: 2019 vs. 2018 results for the Elementary Education test with average and standard error by rubric and task. Student-specific
breakdown is for 2019 only.
4
Special Education edTPA
Table 2: 2019 vs. 2018 results for the Special Education test with average for every rubric and task. Student-specific breakdown is for
2019 only.
5
Results by Rubric and Task for the 2019 Elementary Education Exam
Chart 1: Average and standard error for each rubric based on all attempts at the Elementary Education exam in 2019 (based on Table
1).
1.7
1.9
2.1
2.3
2.5
2.7
2.9
3.1
3.3
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Ave
rage
Sco
re
Rubric #
Average Rubric Scores - Elementary Education
6
Chart 2: Average and standard error for each task based on all attempts at the Elementary Education exam in 2019 (based on Table
1).
1.80
2.00
2.20
2.40
2.60
2.80
3.00
Task 1: Planning Task 2: Instructing Task 3: Assessing Task 4: Math
Ave
rage
Sco
re
Task
Average Task Scores - Elementary Education
7
Year-over-year Comparison of Results by Rubric and Task for the Elementary Education Exam
Chart 3: Average and standard error for all tests vs. passed tests across every rubric in 2019 vs. 2018 (based on Table 1).
1.5
1.7
1.9
2.1
2.3
2.5
2.7
2.9
3.1
3.3
3.5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Ave
rage
Sco
re
Rubric #
Average Rubric Scores - Elementary Education
2019 Results 2018 Results
8
Chart 4: Average and standard error for all tests vs. passed tests across every task in 2019 vs. 2018 (based on Table 1).
1.50
1.70
1.90
2.10
2.30
2.50
2.70
2.90
3.10
Task 1: Planning Task 2: Instructing Task 3: Assessing Task 4: Math
Ave
rage
Sco
re
Task
Average Task Scores - Elementary Education
2019 Results 2018 Results
9
edTPA Rubric Summary
Task Rubric
Task 1: Planning for Literacy Instruction and Assessment
Rubric 1: Planning for Literacy Learning
Rubric 2: Planning to Support Varied Student Learning Needs
Rubric 3: Using Knowledge of Students to Inform Teaching and Learning
Rubric 4: Identifying and Supporting Language Demands
Rubric 5: Planning Assessments to Monitor and Support Student Learning
Task 2: Instructing and Engaging Students in Literacy Learning
Rubric 6: Learning Environment
Rubric 7: Engaging Students in Learning
Rubric 8: Deepening Student Learning
Rubric 9: Subject-Specific Pedagogy
Rubric 10: Analyzing Teaching Effectiveness
Task 3: Assessing Students' Literacy Learning
Rubric 11: Analysis of Student Learning
Rubric 12: Providing Feedback to Guide Further Learning
Rubric 13: Student Understanding and Use of Feedback
Rubric 14: Analyzing Students' Language Use and Literacy Learning
Rubric 15: Using Assessment to Inform Instruction
Task 4: Assessing Students' Mathematics Learning
Rubric 19: Analyzing Whole Class Understandings
Rubric 20: Analyzing Individual Student Work Samples
Rubric 21: Using Evidence to Reflect on Teaching
Table 3: Task and Rubric names for the Elementary Education and Special Education exams. Task 4: Assessing Students’ Mathematics
Learning is excluded from the Special Education exam. Highlighted Rubrics indicate best performance across all students. Best
performance in 2018 was in Rubrics 2, 6, 11 and 19.
10
Analysis
For a student to pass their exam, they must answer every question with minimum sufficiency, gaining at least a score of 1.0 per
rubric. Failure to complete any question at this minimum level results in overall failure of the test. One student had to retake the
Elementary Education once this year given incomplete performance on Task 2 (Instructing and Engaging Students in Literacy
Learning). In 2018, four students had to retake the exam, primarily due to poor performance on Task 4 (Assessing Students’
Mathematics Learning).
Elementary Education Exam
Of the 2019 cohort, students who took the Elementary Education exam performed best in Task 3: Assessing Students’ Literacy
Learning with an average score of 2.73. The one student who took the Special Education exam also performed best on this task,
bringing average performance across all students to 2.80. By contrast, performance was worst in Task 3 in 2018 (average score of
2.18) and best performance was in Task 1: Planning for Literacy Instruction and Assessment with an average score of 2.80.
As in 2018, students performed most consistently in Task 2, where standard error of 0.10 was the lowest across all tasks.
In both years, students performed worst in Task 4: Assessing Students' Mathematics Learning. Students in 2019 showed a marked
improvement compared to the previous year with an average score of 2.17 compared to 1.73 in 2018. However, when only
considering the passing scores in 2018 (as mentioned, four students failed on the first attempt last year), then the average score on
this task was 2.49.
This year, performance was highest in Rubric 12: Providing Feedback to Guide Further Learning, which increased its average score
from 2.3 in 2018 to 3.1 this year. Poorest performance resulted in an average rubric-specific score of 2.0, which occurred in Rubric 5:
Planning Assessments to Monitor and Support Student Learning (compared to 2.6 last year) and Rubric 19: Analyzing Whole Class
Understandings (compared to 1.8 last year).
On Task 4: Assessing Students' Mathematics Learning, students performed best on Rubric 20: Analyzing Individual Student Work
Samples and Rubric 21: Using Evidence to Reflect on Teaching this year. Both these rubrics demonstrated improved performance
11
compared to 2018, when students did worse on these questions than on Rubric 19: Analyzing Whole Class Understandings.
Performance increased on all three rubrics comprising this task.
Special Education Exam
Only one student chose to take the Special Education exam; therefore, there were too few results to conduct any statistical analysis.
Standard Error
Standard error was used in place of standard deviation to determine the distribution of test results across students within each Task.
The use of standard error is recommended in place of standard deviation when conducting such analysis on small samples.
𝑆𝐸 =𝜎
√𝑛
As demonstrated in the calculation above, standard error (SE) is determined as the standard deviation (σ) of the population divided
by the square root of the number of samples (n).
As shown on pages 2 and 3, Charts 1 – 4 display �̅� ± 𝑆𝐸 for every Task or Rubric, which is the average [plus or minus] the standard
error.
The greater the standard error, the larger the variation of performance between grades. A small standard error shows that the
majority of grades are concentrated around the average.
As shown on pages 2 and 3, Charts 1 – 4 display �̅� ± 𝑆𝐸 for every Task or Rubric, which is the average [plus or minus] the standard
error.
The greater the standard error, the larger the variation of performance between grades. A small standard error shows that the
majority of grades are concentrated around the average.
12
2. Indicators of teaching effectiveness
MSED Student Teaching Proficiencies
The Student Teacher Proficiencies Form is completed by the cooperating teacher and by the college supervisor in each placement of student teaching. The means reported in the data table represent the scores from both cooperating teachers and the college supervisors. The rubric indicators are designed to assess candidates in three categories: Knowledge, Skills, and Dispositions/Values.
Proficiencies
Mean Number % Meet or
Above Std.
Knowledge
A. Subject Matter
Candidate knows, understands, and can use general and specialized curricula for teaching
across curricular content areas to individualize learning for students with and without
exceptionalities.
3.7 19 100%
B. Student Learning
Candidate knows and understands similarities and differences in human development and
learning and uses this knowledge to provide meaningful and challenging learning experiences
for students with exceptionalities.
3.7 19 100%
C. Diversity of Learners
Candidate uses differentiated instruction that demonstrates understanding of how language,
culture, and family background influence the learning of students including students with and
without disabilities.
3.5 19 100%
D. Environment for Learning
Through collaboration with colleagues, candidate creates safe, inclusive, culturally responsive
learning environments to engage students with and without exceptionalities.
3.4 19 95%
E. Assessment
Candidate demonstrates the ability to select, adapt, monitor, analyze and use multiple methods
of assessment in making educational decisions for students with and without exceptionalities.
3.2 19 95%
Skills
A. Planning Instruction
Candidate is able to select, adapt, and use a repertoire of evidenced-based instructional
strategies to advance language development and communication skills.
3.7 19 100%
13
B, Strategies/Technologies
Candidate uses a variety of instructional and assistive technologies, and augmentative and
alternative communication systems to enhance language and communication skills of students
with exceptionalities.
3.7 19 100%
C. Learning Environment
Through collaboration with general educators and other colleagues, candidate creates a safe,
inclusive, and culturally responsive learning environment to engage students with and without
exceptionalities, in meaningful learning activities and social interactions.
3.6 19 100%
D. Communication
Candidate effectively uses verbal, non-verbal, written language, and media communication
strategies to support and enhance language development and communication skills of students
with exceptionalities.
3.2 19 89%
E. Assessment
Candidate selects, adapts and modifies formal and informal assessments to guide educational
decisions for students with disabilities.
2.9 19 89%
F. Collaboration and Relationships
Candidate demonstrates across a range of settings, the ability to consult, and collaborate with
school colleagues, families and community personnel, to promote the learning and well-being
of students with exceptionalities.
2.9 19 89%
G. Reflection and Professional Development
Candidate uses knowledge of historical, current and cultural and ethical issues and reflects on
the impact of his/her practice to improve instruction for students with disabilities and to guide
their own professional growth, practice, and significance of participating in professional
activities and learning communities.
2.9 19 89%
Dispositions/Values
A. Diversity/Individual Differences
Candidate demonstrates that he/she can create consistent culturally responsive learning
environments that promote increased independence, self-motivation, self-direction, self-
advocacy and personal empowerment of students with exceptionalities.
3.6 19 100%
B. High Expectations
Candidate demonstrates commitment and upholds high standards of competence and integrity
to developing the highest education and quality-of-life potential of students with and without
disabilities.
3.7 19 100%
14
3. Satisfaction of Employers and Employment Milestones
Even though this report is not available at this time, it is planned to be available in one year.
4. Satisfaction of Completers
Even though this report is not available at this time, it is planned to be available in one year.
15
5. Graduation Rates
The data below presents the completion (graduation) and dropout rates for the MSED program. All of the rates track new students
entering the program.
The Academic Year (AY) starts with the summer, followed by the fall, and lastly the spring semester. Example: 2014-2015 academic
year includes the summer 2014, fall 2014, and spring 2015.
Dropout rates do not include students who are still enrolled at the program. Still enrolled at the program included students who are
were enrolled for the spring 2019 semester or enrolled for the summer 2019 semester as of 4/23/19.
Table 1. Completion Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY
Academic Year Admitted Cohort
Completed
Cohort Percentage Completed
2014-2015 27 23 85%
2015-2016 18 13 72%
2016-2017 23 22 96%
2017-2018 18 12 67%
16
Figure 1. Completion Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY
17
Table 2. Dropout Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY
Academic Year Admitted Cohort
Dropped Out
Cohort Percentage Dropped Out
2014-2015 27 4 15%
2015-2016 18 5 28%
2016-2017 23 1 4%
2017-2018 18 5 28%
18
Figure 2. Dropout Rates for the MSED Program from 2014-2015 AY - 2017-2018 AY
19
6. Ability of Completers to Meet Certification and State Requirements
Besides the edTPA, New York State requires candidates who are seeking certifications in Childhood Education 1-6 and Special
Education (Students with Disabilities) to pass the following NYSTCE assessments:
201- Educating All Students
060 – Students with Disabilities
221/222/245 – Multi-Subject: Teachers of Childhood (Grades 1-6)
Note: The former required 202 – Academic Literacy Skills Test was eliminated as a state requirement for certification. In addition,
the choice of passing either the former “safety net” assessment versions, and/or passing the new reformulated versions were still in
place during 2016-17 but ended on June 30, 2017.
The following are the 2019 Title II Assessment Pass Rates:
20
Assessment Pass Rates
Assessment code - Assessment name Test Company Group
Number taking tests
Avg. scaled score
Number passing tests
Pass rate (%)
202 -ACADEMIC LITERACY SKILLS TEST
Evaluation Systems group of Pearson
All program completers, 2015-16
11 515 7 64
201 -EDUCATING ALL STUDENTS
Evaluation Systems group of Pearson
Other enrolled students
12 509 8 67
21
Assessment Pass Rates
Assessment code - Assessment name Test Company Group
Number taking tests
Avg. scaled score
Number passing tests
Pass rate (%)
201 -EDUCATING ALL STUDENTS
Evaluation Systems group of Pearson
All program completers, 2017-18
16 508 12 75
201 -EDUCATING ALL STUDENTS
Evaluation Systems group of Pearson
All program completers, 2016-17
13 517 13 100
201 -EDUCATING ALL STUDENTS
Evaluation Systems group of Pearson
All program completers, 2015-16
20 521 19 95
22
Assessment Pass Rates
Assessment code - Assessment name Test Company Group
Number taking tests
Avg. scaled score
Number passing tests
Pass rate (%)
002 -MULTI-SUBJECT CST
Evaluation Systems group of Pearson
All program completers, 2017-18
1
1221 -MULTI-SUBJECT GRADES 1 - 6
Evaluation Systems group of Pearson
Other enrolled students
3
1221 -MULTI-SUBJECT GRADES 1 - 6
Evaluation Systems group of Pearson
All program completers, 2017-18
9
23
Assessment Pass Rates
Assessment code - Assessment name Test Company Group
Number taking tests
Avg. scaled score
Number passing tests
Pass rate (%)
1221 -MULTI-SUBJECT GRADES 1 - 6
Evaluation Systems group of Pearson
All program completers, 2016-17
8
1221 -MULTI-SUBJECT GRADES 1 - 6
Evaluation Systems group of Pearson
All program completers, 2015-16
13 1631 12 92
902 -SAFETY NET MULTI-SUBJECT
Evaluation Systems group of Pearson
All program completers, 2017-18
3
24
Assessment Pass Rates
Assessment code - Assessment name Test Company Group
Number taking tests
Avg. scaled score
Number passing tests
Pass rate (%)
902 -SAFETY NET MULTI-SUBJECT
Evaluation Systems group of Pearson
All program completers, 2015-16
3
960 -SAFETY NET STUDENTS WITH DISABILITIES
Evaluation Systems group of Pearson
All program completers, 2017-18
2
960 -SAFETY NET STUDENTS WITH DISABILITIES
Evaluation Systems group of Pearson
All program completers, 2016-17
1
25
Assessment Pass Rates
Assessment code - Assessment name Test Company Group
Number taking tests
Avg. scaled score
Number passing tests
Pass rate (%)
TP012 -SPECIAL EDUCATION
Evaluation Systems group of Pearson
Other enrolled students
2
TP012 -SPECIAL EDUCATION
Evaluation Systems group of Pearson
All program completers, 2016-17
1
060.1 -STUDENTS WITH DISABILITIES CST.1
Evaluation Systems group of Pearson
Other enrolled students
7
26
Assessment Pass Rates
Assessment code - Assessment name Test Company Group
Number taking tests
Avg. scaled score
Number passing tests
Pass rate (%)
060.1 -STUDENTS WITH DISABILITIES CST.1
Evaluation Systems group of Pearson
All program completers, 2017-18
15 520 10 67
060.1 -STUDENTS WITH DISABILITIES CST.1
Evaluation Systems group of Pearson
All program completers, 2016-17
11 535 9 82
060.1 -STUDENTS WITH DISABILITIES CST.1
Evaluation Systems group of Pearson
All program completers, 2015-16
15 535 14 93
27
7. Ability of Completers to Be Hired in Education Positions for Which They Have Prepared
The New York State Department of Education does not provide teacher preparation programs with any data on the teaching
employment of graduates. MCNY was able to determine that 17 out of the 19 program completers in 2017-18 were hired in
positions for which they have prepared. Even with the unknown results for two completers, the rate of employment is 89%.
Completer Employment
A Public District Elementary School
B Public Elementary Charter School
C Unknown
D Public Elementary Charter School
E Public Elementary Charter School
F Public Elementary Charter School
G Public District Elementary School
H Public District Elementary School
I Public District Elementary School
J Public District Elementary School
K Public District Elementary School
L Public District Elementary School
M Public District Elementary School
N Public District Elementary School
O Public District Elementary School
P Public Elementary Charter School
Q Public District Elementary School
R Unknown
S Public Elementary Charter School
28
8. Student Loan Default
MCNY Default Rates (2013-2015)
The USDOE Cohort Default Rate History List provides information on Metropolitan College of New York’s student default rate. Each
USDOE reports listed are for cohorts over a three-year period. The most recent cohort report (2015) indicates a low default rate of
4%.
29