teacher effectiveness measure (tem): summative process

1

Teacher Effectiveness Measure (TEM): Summative Process

By now, all Memphis City Schools educators are familiar with the Teacher Effectiveness Measure (TEM) as our

district’s new, multi-dimensional teacher evaluation system. This document is intended to prepare you for the

upcoming summative evaluation process during which all teachers will receive data and ratings for each TEM

component as well as an overall TEM score.

Details are provided on 1) the different TEM components each educator group will use for the summative score;

2) how each component will be scored in the TEM profile; and 3) the timeline for the summative evaluation

process. If you have additional questions, please visit the TEI website (www.mcstei.com) or contact the

Department of Teacher Talent & Effectiveness (DTTE) ([email protected]; 416-0135)

TEM Components by Educator Group “Educator Group” refers to which group a teacher has been assigned to for his or her observation rubric. These

groups include General Education, Exceptional Children (Self-Contained or Inclusion), Library/Media Specialists,

Guidance Counselors, Psych/Social Workers, and Instructional Facilitators. Although the majority of teachers

will use the same TEM components, these components vary for some educator groups. These distinctions are

provided in detail below:

General Education Teachers in Tested Subjects

TEM Component Measurement Used Weighting

Student Growth

Individual Value-Added (TVAAS) Data 35%

(State Mandated)

Student Achievement Menu of Options Provided by Tennessee Dept.

of Education

15%

(State Mandated)

Observations TEM General Education Rubric 40%

Stakeholder Perceptions Spring 2012 Tripod Student Survey Results 5%

Teacher Knowledge Menu of Options Provided by MCS 5%

General Education Teachers in Non-Tested Subjects


Student Growth

PreK – 3: School-Level Literacy or Math TVAAS

9 – 12: School-Level Literacy or Math TVAAS

CTE: School-Level Literacy or Math TVAAS

World Language: School-Level Literacy TVAAS

ELL: School-Level Literacy TVAAS

P.E./Health: School-Level Overall TVAAS

Comp Tech: School-Level Overall TVAAS

Fine Arts: School-Level Overall TVAAS

35%

(State Mandated)

Student Achievement Menu of Options Provided by Tennessee Dept. of

Education

15%

(State Mandated)

Observations TEM General Education Rubric 40%

Stakeholder Perceptions Spring 2012 Tripod Student Survey Results 5%


http://www.mcstei.com/

mailto:[email protected]

2


Teachers of Exceptional Children

1


Student Growth

School-Level Overall TVAAS 35%

(State Mandated)


of Education

15%

(State Mandated)

Observations TEM Exceptional Children Rubric (Inclusion

or Self-Contained)

45%

Stakeholder Perceptions Not Applicable 0%


Library/Media Specialists


Student Growth

School-Level Literacy TVAAS 35%

(State Mandated)


of Education

15%

(State Mandated)

Observations TEM Library/Media Specialists Rubric 50%


Teacher Knowledge Not Applicable 0%

Guidance Counselors


Student Growth

School-Level Overall TVAAS 35%

(State Mandated)


of Education

15%

(State Mandated)

Observations TEM Guidance Counselor Rubric 50%



Psych/Social Workers


Student Growth

School- or District-Level Overall TVAAS 35%

(State Mandated)


of Education

15%

(State Mandated)

Observations TEM Psych/Social Workers Rubric 50%



1 CLUE teachers are to use the General Education TEM classification instead of Exceptional Children educator group.

3


Instructional Facilitators


Student Growth School-Level Literacy, Math or Overall

TVAAS

35%

(State Mandated)


of Education

15%

(State Mandated)

Observations TEM Psych/Social Workers Rubric 50%



Scoring the TEM As with TVAAS and TEM observations, each component of a teacher’s TEM profile will receive a score on a

scale of 1 – 5. Those scores will then be weighted2 and added together as demonstrated in the table below:

Range of TEM Component Scores (Minimum – Maximum)

Weighting Weighted Score

(Min – Max)

Student Growth (TVAAS) 1 - 5 x 35 35 – 175

Student Achievement 1 - 5 x 15 15 – 75

Observations 1.00 – 5.00

(average rounded to the nearest .01)

x 40

40 – 200

Stakeholder Perceptions 1 – 5 x 5 5 – 25

Teacher Knowledge 1 – 5 x 5 5 – 25

TOTAL 100 - 500

Once the total weighted score is calculated, a TEM teacher effectiveness rating is assigned based on the ranges

provided below:

TEM Score Range

Effectiveness Rating

100 - 200 TEM 1 Performing Significantly Below Expectations

200 – 274.99 TEM 2 Performing Below Expectations

275 – 349.99 TEM 3 Meeting Expectations

350 – 424.99 TEM 4 Performing Above Expectations

425 – 500 TEM 5 Performing Significantly Above Expectations

2 This table reflects the TEM profile of a teacher with all five possible TEM components. The weighted calculations will

vary somewhat for educator groups with fewer than five TEM components.

4


Scoring Student Growth (TVAAS)

What is TVAAS (Student Growth)?

TVAAS is a measure of growth in performance from one year to the next on state-mandated tests. The TVAAS

system calculates growth from tests covered under the Tennessee Comprehensive Assessment Program (TCAP) in

reading/language arts, math, science, and social studies and other core subjects. The general expectation for

TVAAS is that a year’s worth of instruction should result in a year’s worth of student growth.

Is TVAAS the same as Adequate Yearly Progress (AYP)?

No. AYP is a required measurement under the No Child Left Behind (NCLB) Act. In short, AYP refers to

whether or not a student, class, school, district, or state met a pre-set target of proficiency on a criterion-

referenced test (CRT). TVAAS, by contrast, is a growth measure. It is based on the gains students make (or

don’t make) compared to their test outcomes in previous years.

Which teachers have individual-level TVAAS results?

Teachers whose students are assessed at the end of the year by a state-mandated test have TVAAS results. At the

present time, that means core subject teachers in grades 3 – 8 and teachers in Algebra I, Algebra II, Biology I,

English I, English II, and U.S. History have TVAAS results. In 2011–12, English III will be a tested subject that

counts for TVAAS.

What if I don’t have individual-level TVAAS?

According to state law, teachers in non-tested subjects must use school-level TVAAS data for 35% of their

evaluation (TEM) score until the state approves a system for measuring value-add in non-tested subjects. In 2012,

we are expecting that the work of several state-level committees will provide a more direct measurement of the

impact of teachers in non-tested subjects. For the first time this year, a school-level composite TVAAS score will

be calculated. This new statistic will combine TVAAS results from all the teachers in a school and report that as

a single number: the TVAAS Composite.

How will Student Growth (35% of TEM) be scored?

As with all TEM components, TVAAS will be scored on a scale of one through five, and these scores will be

provided by the state of Tennessee. All teachers will use their most stable TVAAS data from the 2010-11 school

year. That means that those who have taught a tested subject for two years will receive a two year composite.

Those who have taught a tested subject for three or more years will receive a three-year composite. If you have

taught a subject Those teachers and schools whose students demonstrate less than a year of growth will either

receive a score of 1 (“Significantly Below Expectations”) or 2 (“Below Expectations”). Teachers and schools

with students demonstrating one year of growth will receive a score of 3 (“Meeting Expectations”). Those with

Students demonstrating more than one year of growth will receive a score of 4 (“Above Expectations”) or 5

(“Significantly Above Expectations”).

When will TVAAS scores be available?

MCS will use the TVAAS data that is currently available to calculate individual teachers’ TEM scores. These

TVAAS data became available to principals and teachers in tested subjects in November 2011.

5


Scoring Student Achievement

How will Student Achievement (15% of TEM) be scored?

15% of the TEM evaluation score must include a measure of student achievement outcomes based on a menu of

options that the state of Tennessee has deemed appropriate for a teacher’s educator group as outlined in the table

below. As with all TEM components, each these achievement outcomes will be scored on a scale of one through

five, and these scores will be provided by Memphis City Schools. The options3 identified by the state are:

Stat

e as

sess

men

ts

(dis

cip

line-

spec

ific

/TC

AP

)

Sch

oo

l-w

ide

TVA

AS

or

ind

ivid

ual

TV

AA

S fo

r

teac

her

s in

to

p 3

qu

inti

les

AC

T su

ite

of

asse

ssm

ents

/SA

T su

ite

of

asse

ssm

ents

Nat

ion

al/S

tate

-use

d “

off

th

e sh

elf”

ass

essm

ents

AP

/IB

/NIC

su

ites

of

asse

ssm

ents

Gra

du

atio

n r

ate

/ C

TE c

on

cen

trat

or

grad

uat

ion

rate

Po

stse

con

dar

y m

atri

cula

tio

n/p

ers

iste

nce

/ p

lace

men

t

Co

mp

leti

on

/su

cces

s in

adva

nce

d c

ou

rsew

ork

, in

clu

din

g d

ual

cre

dit

an

d

du

al e

nro

llmen

t

9th

gra

de

pro

mo

tio

n t

o

the

10

th g

rad

e/ 9

th g

rad

e re

ten

tio

n r

ate

Teachers with TVAAS (4-8) X X X X X

Teachers with TVAAS (9-12) X X X X X X X X X

PK-3 X X X

Fine Arts X X X X X X X X X

Middle/High School non-assessed courses

X X X X X X X X X

World Languages X X X X X X X X X

Computer Technology X X X X X X X X X

PE and Health Educators X X X X X X X X

Instructional Facilitators X X X X X X X X

Library Media Specialists X X X X X X X X

ELL X X X X X X X X X

Exceptional Children X X X X X X X X X

CTE X X X X X X X X X

Psych/Social Workers X X X X X X X X

3 National/State “off the shelf” tests include: PreK-12 diagnostic or achievement/attainment assessments (e.g. SAT 10,

Dibels, DRA, Kindergarten-readiness, end of course, etc.) DOE will develop standard criteria for approval of tests submitted

by LEAs.

AP/IB/NIC suites of assessments include: Courses designed for Advanced Placement (AP), International Baccalaureate (IB),

National Industry Certification (NIC) assessments

Participation in advanced coursework includes: School level calculated secondary rates (e.g. Honors, AP, IB, NIC,

college/high school dual enrollment and dual credit) according to SBE uniform grading policy.

6


What is the selection process for the Student Achievement measure?

In Spring 2012, teachers will be able to select an appropriate Student Achievement measure using their Randa

Tower software accounts. By state regulation, principals must agree with teachers’ selections. If a teacher and

principal cannot reach consensus, the principal shall choose the Student Achievement measure. MCS will provide

principals with all available Student Achievement data and corresponding TEM scores from the 2010-11

academic year prior to the selection deadline to assist teachers and principals in this process.

When will Student Achievement scores be available?

As with Student Growth, MCS will use the Student Achievement data that is currently available to calculate

individual teachers’ TEM scores. MCS will provide principals with all available school-level Student

Achievement data and corresponding TEM scores from the 2010-11 academic year in Spring 2012 to assist

teachers and principals in the selection process.

Scoring Observations

How will Observations be scored?

As with all TEM components, observation ratings are scored on a scale of one to five. MCS will average all of

the indicators that have been rated across all of a teacher’s observations to generate a composite observation

score. Please note that any unrated indicators will not count towards a teacher’s final composite score. MCS will

only average those indicators that have been rated a 1 – 5. Unlike the other TEM components, teachers’

observation composite averages will be rounded to the nearest 0.01 instead of the nearest whole number.

Teachers in the General Education educator group will only receive scores for the indicators in the Teach and

Cultivate a Learning Environment domains, whereas other educator groups will receive scores for the domains in

their specialized rubrics.4

What are the requirements for a complete observation score?

According to state regulations, teachers with Professional licensure should be observed a minimum of 4 times

each academic year for a combined total of no fewer than 60 minutes. Teachers with Apprentice licensure should

be observed a minimum of 6 times for a combined total of no fewer than 90 minutes. Half of a teacher’s

observations should be announced and half of the observations should be unannounced. All teachers should

receive a post conference following each observation. Teachers should also have the following documentation,

located in the Randa Tower software:

Self-Assessment – Completed by the teacher during the first observation cycle

Professional Growth & Support Plan (PGSP) – Completed jointly by the teacher and a school

administrator during the first observation cycle

Observation Scores – Completed by the observer after each observation

Post-Observation Conference Record – Submitted by the observer after incorporating the teacher’s

reflections and any adjustments to the PGSP following an observation post conference

4 Librarians are exempt from the “Plan” domain.

7


When will the composite TEM observation score be available?

Administrators have been instructed to complete all observations by March 30th. Individual observation scores

have been made available to teachers on a continual basis following each observation. MCS will provide teachers

and principals with the overall observation composite score along with all TEM data by April 30th, 2012.

Scoring Stakeholder Perceptions

How will Stakeholder Perceptions be scored?

For 2011-12, Stakeholder Perceptions will be measured using the Spring 2012 results from the Tripod Student

Survey, which will be administered in early March. The survey asks students to assess observable teaching

practices in their classrooms according to the “Seven Cs”:

Caring about students (encouragement and support)

o Ex: “My teacher in this class makes me feel that s/he really cares about me”

Controlling behavior (press for cooperation and peer support)

o Ex: “Our class stays busy and doesn’t waste time”

Clarifying lessons (success seems feasible)

o Ex: “My teacher explains difficult things clearly”

Challenging lessons (press for effort, perseverance and rigor)

o Ex: “My teacher wants me to explain my answers – why I think what I think”

Captivating students (learning seems interesting and relevant)

o Ex: “My teacher makes learning enjoyable”

Conferring with students (students sense their ideas are respected)

o Ex: “My teacher wants us to share our thoughts”

Consolidating knowledge (ideas get connected and integrated)

o Ex: “My teacher takes the time to summarize what we learn each day”

MCS will then compare individual teachers’ Tripod results to the average Tripod results for all teachers in similar

content areas and grade levels and create an index score of 1 – 5, like with TVAAS scores. An example is

provided in the table below.

Teaching

Practice

Individual

Teacher’s Score

Avg. for MCS

Peer Teachers

Caring 70 55

Controlling 54 52

Clarifying 43 40

Challenging 55 51

Captivating 62 48

Conferring 45 41

Consolidating 57 54

Overall 56 49 Index:

Level 4

8


What if I don’t have students that will take the Tripod survey this year?

Teachers in the Exceptional Children, Library/Media Specialist, Guidance Counselor, Psych/Social Worker and

Instructional Facilitator educator groups will not receive Tripod survey results in 2011-12. These groups will not

have Stakeholder Perceptions as part of their TEM profiles, and the adjusted TEM component weightings are

reflected on pages 2 – 3 of this document.

When will the Stakeholder Perceptions score be available?

MCS will provide teachers and principals with Tripod results and the index score along with all TEM data by

April 30th, 2012.

Scoring Teacher Knowledge

How will Teacher Knowledge be scored?

As with Student Achievement, teachers will select from a menu of options for the Teacher Knowledge

component. Please note that this menu of options will be revisited on an annual basis and potentially revised as

additional measures become available. The menu of options for 2011-12 is detailed in the table below.

Option Description Scoring Method Content-Specific

Professional

Development

For 2011-12, this option entails the successful

completion of 10 hours of the Common Core

online modules already required as part of

teachers’ 57 PD hours. The deadline for

completion is March 1st, 2012.

Parts 1 & 2: AVATAR Course # 10490.12500

Part 3: AVATAR Course # 11225.13658

Part 4: AVATAR Course # 11226.13965

Teachers who successfully complete

all four Common Core courses by

March 1st, will receive a 5. Those

who do not complete all four

courses by March 1st will receive a

1.

Portfolio of Work Teachers may submit a portfolio of work that

will be reviewed by a committee of peers in

their content area that have been identified by

the district. These committees have developed

the rubrics used to score portfolios for each

content area. The rubrics are available on the

TEI website under the “Content Knowledge”

section of the TEM page:

(www.mcstei.com/tem/content-knowledge)

Portfolios must be submitted no later than

March 16th

by mail or in person to:

Office of Teacher Effectiveness Measurement

2597 Avery Avenue Room 260

Memphis, TN 38112

Teachers will receive a score of 1 –

5 on their portfolios based on the

rubric for their content area.

http://www.mcstei.com/tem/content-knowledge

9


Observation by a

Content Specialist

Teachers may request an observation from a

content specialist in their subject area to count

towards both Teacher Knowledge and

Observation TEM components. It should be

noted that this option is limited by Content

Specialists’ availability and case load capacity.

If one of a teachers 4-6 required classroom

observations was conducted by a content

specialist, the observation’s scores will satisfy

the knowledge requirement. If a teacher has not

been observed by a content specialist, an

additional classroom observation using the

TEM rubric will be conducted to satisfy this

requirement. These observations will be

conducted during the week of April 2nd

-April

5th and April 9

th-April 13

th.

All rated indicators of the Content

Specialist’s observation will be

averaged and rounded to the nearest

hundredth place, as will the

composite Observation score.

What is the selection process for Teacher Knowledge measure?

In early February, teachers will receive an email with a Survey Monkey link to select their Teacher Knowledge

component. Unlike the Student Achievement component, teachers do not need approval from their principals to

select an option. Teachers who select a Content Specialist observation as their top choice will receive a follow-up

email from a district administrator confirming whether or not a Specialist will be available to assist with this

component. If capacity for observations becomes limited, teachers may be asked to select another option.

What if my educator group does not have a Teacher Knowledge component for the TEM?

Teachers in the Library/Media Specialist, Guidance Counselor, Psych/Social Worker and Instructional Facilitator

educator groups are exempt from the Teacher Knowledge components of the TEM in 2011-12. The adjusted

TEM component weightings for these groups are reflected on pages 2 – 3 of this document.

When will Teacher Knowledge scores be available?

Although the timeline may vary by the option selected, MCS will provide teachers and principals with Teacher

Knowledge scores along with all TEM data by April 30th, 2012.

The Summative TEM Process and Timeline

What is the decision-making process for teacher reelections based on TEM scores?

By April 30, 2012, you and your principal will receive your TEM profile with scores for each component and

composite TEM scores of 1 - 5. Student Growth and Achievement data for the current year will not be available,

so teachers will receive scores for these components based on existing 2010-11 data where available. Scores for

the other TEM components (Observations, Stakeholder Perceptions and Teacher Content Knowledge) will be

provided for the current year by April 30th. Once teachers and principals receive these TEM Profiles, school-

level administrators will schedule a summative evaluation meeting with each teacher to review his or her TEM

data and notify teachers of reelection recommendations.

10


Guidance on Proper Use of the TEM for Reelection Recommendations

The TEM is a management tool providing school leaders the ability to readily identify appropriate support,

development and recognition opportunities for teachers aligned to their individual performance data. Tennessee

law states that a district may also use teacher performance data as grounds for a teacher’s dismissal if he or she is

performing “Below Expectations” or “Significantly Below Expectations.” For MCS, this means that teachers

scoring a TEM 1 or TEM 2 may be considered for non-reelection. Teachers scoring a TEM 3 or higher will not be

eligible for dismissal based on their TEM evaluation scores because they are meeting performance expectations.

However, a teacher’s TEM score alone does not determine whether or not he or she will be reelected. If a

teacher scores a TEM 1 or 2, school leaders must ultimately make recommendations based on whether or not a

teacher’s performance or potential to improve will lead to positive student outcomes. Further, please note that a

teacher’s TEM score is not the only factor influencing reelection recommendations. As with prior school years,

any teacher may be considered for non-reelection for criteria other than, or, in combination with TEM scores

including 1) incompetence, 2) inefficiency, 3) neglect of duty, 4) unprofessional or improper conduct, and 5)

insubordination. As with previous years, teachers who are recommended for non-reelection based on TEM scores

or for any other reason will have their files reviewed by district personnel before a final determination is made.

Teachers who are reelected in May will remain with the district for the 2012-13 school year. As with previous

years, teachers who are recommended for non-reelection based on TEM scores or for any other reason will have

their files reviewed by district personnel before a final determination is made. Teachers with tenure are

guaranteed due process. If teachers are recommended for non-reelection for reasons other than or in addition to

their TEM scores, Human Resources will make a final determination as with prior years. A flow chart of this

process is provided below:

11


A timeline for this summative TEM process is provided below:

March Deadline for teachers and principals to select teachers’ Student Achievement

option in Randa Tower software. Details forthcoming.

March 5 - 9 Tripod student survey is administered for Stakeholder Perceptions TEM

component.

March 30 Deadline for completing all Observations.

April 30 All teachers receive TEM Profiles. Individual summative meetings with school-

level administrators begin.

May 15 Deadline for principals to complete summative TEM meetings and submit all

teacher reelection recommendations.

June 15 Current-year Student Growth and Achievement data is available from the state.

TEM Profiles are updated.

Please note that teachers must file any grievances based on evaluations between April 1st and May 14

th. Teachers

may only grieve errors related to the evaluation process and the accuracy of the evaluation data assigned to

them. To learn more about the grievance process or to get forms to file a grievance, please contact the Memphis

Education Association (MEA), 901-454-0966.

teacher effectiveness measure (tem): summative process

Documents