CHAPTER 3

By: Lauren ChadwickED 450

Monday September 9, 2012

BellworkO Please use a pen

or pencil and draw 2 appointment slots on the plate I gave you when you entered the classroom:

Once you are finished open the folded paper and fill in the blank with the type of word written on the outside.

Put paper in box.

1 2

SLO’sO Students will …

O Understand reliability in assessment.O Describe the 3 types of reliability

evidence.O Explain the importance of standard

error of measurement.

Let’s have some fun!

O Everyone get up and stand in a circle!

Reliability?

Reliability = Consistency

O 2 important ways reliability may be useful:

O How reliable are your assessments for you and your students?

O Parents may ask how reliable standardized tests may be.

O Reliability is an important criteria used when judging standardized tests.

3 Types of Reliability Evidence

OStabilityOAlternate FormOInternal Consistency

Stability ReliabilityO Consistency of

test results over time

O The goal is to see close scores in tests administered even if they are given on different occasions.

O Example: Midterm on TuesdayO A masked man grabs your

briefcase containing the students’ tests!

O The next day you tell the students that their tests were stolen and that they need to re-take the test!

O Because there has been no new information taught since the last test, you expect that the test scores would be similar.

O So, if you were able to see that the scores were not similar, then your exam would be judged to have little stability reliability.

Test-RetestO To see how stable an assessment’s

results are over time, we usually test students, then in a week or two retest with the same form of assessment.

O It is important though that there is no significant event taking place in between the two assessments. Example: field trip, movie, speaker- exposing student to test-relevant information.

So, how are scores computed?Correlation

Coefficient (r)So…..

O Reflects the degree of similarity between scores on two tests/

O Ranges from: +1.0 to -1.0

(close to 1.0 = strong relationship)(closer to 0 = weak relationship)

O …if the test–retest r for two sets of test scores were .84 this would show that the students’ performance on both tests were similar.

O If the r was instead .23 this would show that the results were not as similar.

Classification- Consistency

O To decide which students would be exempted from further assessment.

O Example: If you decide on an 80% answers correct required for exemption, then on a test-retest approach, the teacher would find the % of students who scored ≥ 80% on both assessments.

Student

Test 1 Test 2

Jill 84% 99%

Jack 65% 82%

Classification for Jill: ≥ / ≥

Classification for Jack : ≠ / ≥

How to determine Classification-ConsistencyA % of students exempt 42%B % of students not exempt 46%C % of students different classification 12%D % of classification – consistency ( A + B) ----------------------------------------------------------------------------------------------

88% Classification- Consistency of that test

Alternate-Form Reliability

O Simply put:

O Providing an equivalent retest for students in another form other than the first and failed assessment.

O Usually found in settings where a student is testing to receive his/her diploma.

O BUT keep in mind when you are reviewing a state-developed test which claims to have equivalent forms, that those tests assess the same information as the original.

Internal Consistency Reliability

O Reflects the degree of consistency of the items on a given test.

O Homogeneous

O Each of the test’s items should be assessing the same variable.

O Example:

20 item test on problem solving:

Students who are strong problem solvers should get most of the 20 items correct

FYIO Tests containing items

which a students can either be right or wrong such as multiple-choice, the most commonly used approaches are the:

Kuder-Richardson procedures

O Tests containing items where students can be given different grades such as in essay items, the most common approach is:

Cronbach’s coefficient alpha

Just a thought…O The more times you taste from a pot

of soup, the more accurate will be your estimate of what the ingredients are.

O One taste might fool you, twenty tastes will give you a much better idea of what’s in the pot.

O More items on a test will yield more reliable estimates of understanding.

Choose one…Stability Reliability ≠ Alternate-Form Reliability ≠ Internal Consistency Reliability

These three forms of reliability evidence should not be used together. Instead, keep in mind the kind of educational decision linked to the results of the assessment.

Standard Error of Measurement

O Supplies information of test consistency of an individual student by estimating their score consistency.

O Problem: Teacher see’s student results from a pervious IQ test. They expect that the student will perform on that

exact level

O SEM helps to remind teachers that scores earned on commercial or classroom tests are not always exact on student’s understanding.

O Another area where SEM is used is in reporting of student performance on state tests.

60 item Math test

O Jack answers 53 out of the 60 questions correctly, so what would he be classified as?

Classification

# correct

Advanced 54 – 60

Proficient 44 – 53

Basic 37 – 43

Below Basic 36 and below a proficient student.

Appointment Time!Everyone find a classmate for your 1st appointment. You have 3 minutes to decide whether the following is stability, alternate-form, or internal consistency reliability.

Suppose you were trying to show that the items on an educational assessment all measured the same variable. What type of reliability evidence would you use?

Internal consistency reliability

Appointment Time!Everyone find a classmate for your 2nd appointment. You have 3 minutes to decide whether the following is stability, alternate-form, or internal consistency reliability.

A new nationally standardized reading test for junior high students has been created, but the test developers want to make sure that the three different forms of tests are performing in essentially the same way. What sort of reliability evidence should they rely on?

Alternate-form reliability

Why do you need to know?

O You may be called on to explain to parents the meaning of their child’s test scores.

O At least be knowledgeable about the meaning of reliability in educational assessments.

O Understanding standard error of measurement will be helpful when you need to explain to parents and students how to make sense of their scores on accountability tests.

Check your understanding..

O Do you understand assessment reliability?

O Do you know the 3 types of reliability evidence?

O Can you explain the importance of standard error of measurement?

Please turn over your appointment clock and leave me

some feedback!

THANK YOU!!