chapter 6 - standardized measurement and assessment
DESCRIPTION
What are the four different levels or scales of measurement? N ominal Scale O rdinal Scale I nterval Scale R atio ScaleTRANSCRIPT
![Page 1: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/1.jpg)
Chapter 6 - Standardized Measurement and
Assessment
http://www.offthemarkcartoons.com/cartoons/2005-08-21.gif
![Page 2: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/2.jpg)
What is measurement?
• the act of measuring• assigning symbols or numbers to
something according to a specific set of rules
![Page 3: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/3.jpg)
What are the four different levels or scales of measurement?
•Nominal Scale
•Ordinal Scale
•Interval Scale
•Ratio Scale
![Page 4: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/4.jpg)
What is essential to know about the Nominal Scale?
• it’s the simplest form of measurement• it uses symbols, such as words or
numbers• it measures categorical variables
LABELIDENTIFY
CLASSIFY
![Page 5: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/5.jpg)
What is essential to know about the Ordinal Scale?
• it’s a rank-order scale• it doesn’t indicate how much greater
one ranking is over another
![Page 6: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/6.jpg)
What is essential to know about the Interval Scale?
• it’s also a rank-order scale• includes equal distances
or intervals between adjacent numbers
• the absence of a zero points means you cannot make “ratio statements”
![Page 7: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/7.jpg)
What is essential to know about the Ratio Scale?
• it’s the highest level of quantitative measure
• it has all the properties of the nominal, ordinal, and interval scales plus it has a true zero point
• it is not often used in educational research
![Page 8: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/8.jpg)
Nominal-labels things-
Ordinal-ranks things-
Interval-ranks w/ equal distances
Ratio-ranks & labels-
Scales of Measurement
![Page 9: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/9.jpg)
How do we define testing?
• the measurement of variables
![Page 10: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/10.jpg)
How do we define assessment?
• gathering data to make evaluations
![Page 11: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/11.jpg)
How do we define error?
• the difference between true scores and observed scores
![Page 12: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/12.jpg)
How do we define traits?
• distinguishable, enduring ways in which one individual differs from another
![Page 13: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/13.jpg)
How do we define states?
• distinguishable but less enduring ways in which individuals vary
![Page 14: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/14.jpg)
What are the twelve assumptions underlying testing and measurement?
psychologicaltraits & states
exist
psychological traits & states
can be measured
various approachesto measurement
can be useful
assessments can answer
some of life’s most important
questions
1 32
4
![Page 15: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/15.jpg)
What are the twelve assumptions underlying testing and measurement?assessment can pinpoint phenomena that require further study
various sourcesof data
enrich & are partof the assessment
process
various sourcesof error
are alwayspart of the assessment
process
measurementtechniques
have strengths&
weaknesses
5
7
6
8
![Page 16: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/16.jpg)
What are the twelve assumptions underlying testing and measurement?test-related
behaviorpredicts non-test
relatedbehavior
testing &
assessmentcan be done in a fair
and unbiasedway
present-day samplingpredicts
future behavior
testing &
assessmentbenefitsociety
9
10
11
12
![Page 17: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/17.jpg)
What is the difference between reliability and validity?
• Reliability refers to the consistency of a score
• Validity refers to the accuracy of the interpretations you make from the scores
If you want validity, you must have reliability.
![Page 18: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/18.jpg)
What is a reliability coefficient?
• a correlation coefficient that is used as an index of reliability
• researchers want reliability coefficients to be as close to +1.00 as possible
![Page 19: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/19.jpg)
What are four different ways of assessing reliability?
1. Test-Retest Reliability2. Equivalent Forms Reliability3. Internal Consistency Reliability4. Interscorer Reliability
![Page 20: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/20.jpg)
What is test-retest reliability?
• a measure of the consistency of scores over time
• the time interval can have an effect on test-retest reliability because people change over time
![Page 21: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/21.jpg)
What is equivalent forms reliability?
• the consistency of a group of individuals’ scores on two equivalent forms of a test measuring the same thing
• the success of this method depends on the ability to construct two equivalent forms of the same test
![Page 22: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/22.jpg)
What is internal consistency reliability?
• the consistency with which the items on a test measure a single construct
![Page 23: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/23.jpg)
What is split-half reliability?
• splitting a test into two equivalent halves and then assessing the consistency of the scores across the two halves of the test
• each half needs to be equal to the other in format, style, content, and other aspects
![Page 24: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/24.jpg)
What is coefficient alpha?
• a formula that provides an estimate of the reliability of a homogeneous test or an estimate of the reliability of each dimension in a multidimensional test
• tells you the degree to which the items are interrelated• need to consider the number of items; don’t just
assume that because the coefficient alpha is large, the items are strongly related
![Page 25: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/25.jpg)
What is interscorer reliability?
• the degree of agreement or consistency between two or more scorers, judges, or raters
• some degree of training and practice for the scorers is advised to improve the reliability of an evaluation
![Page 26: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/26.jpg)
What is the definition of validity?
• the accuracy of the inferences, interpretations, or actions made on the basis of test scores
• to make sure that our test is measuring what we intended it to measure for the particular people in a particular context and that the interpretations we make on the basis of the test scores are correct
• we want our inferences to be accurate and our actions to be appropriate
![Page 27: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/27.jpg)
What is the definition of validity evidence?
• the empirical evidence and theoretical rationales that support the inferences or interpretations made from the test scores
![Page 28: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/28.jpg)
What is the definition of validation?
• the process of gathering evidence that supports inferences made on the basis of test scores
• the best rule is to collect multiple sources of evidence
• validation should be viewed as a never-ending process
![Page 29: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/29.jpg)
What are the characteristics of the different ways of obtaining validity evidence?
• Evidence based on content• Evidence based on internal structure• Evidence based on relations to other variables
![Page 30: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/30.jpg)
What are the characteristics of evidence based on content?
• content-related evidence is when you make a judgment of the degree to which the evidence suggests that the items, tasks, or questions on the test adequately represent the domain of interest
• it’s based on item content, but it is also based on the formatting, working, administration, and storing of the test
![Page 31: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/31.jpg)
What are the characteristics of evidence based on internal structure?
• Factor Analysis – a statistical procedure that analyzes the relationships among items to determine whether a test is unidimensional or multidimensional
• Homogeneity – refers to how well the different items in a test measure the same construct or trait
![Page 32: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/32.jpg)
What are the characteristics of evidence based on relations to other variables?• Criterion• Criterion-Related Evidence• Validity coefficient• Concurrent evidence• Predictive evidence• Convergent evidence• Discriminant evidence• Known groups evidence
![Page 33: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/33.jpg)
Define criterion.
the standard or benchmark that you want to predict accurately on the basis of the test scores
![Page 34: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/34.jpg)
Define criterion-related evidence.
validity evidence based on the extent to which scores from a test can be used to predict or infer performance on some criterion such as a test or future performance
![Page 35: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/35.jpg)
Define validity coefficient.
a correlation coefficient that is computed to provide validity evidence, such as the correlation between test scores and criterion scores
![Page 36: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/36.jpg)
Define concurrent evidence.
validity evidence based on the relationship between test scores and criterion scores obtained at the same time
![Page 37: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/37.jpg)
Define predictive evidence.
validity evidence based on the relationship between test scores collected at one point in time and criterion scores obtained at a later time
![Page 38: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/38.jpg)
Define convergent evidence.
validity evidence based on the relationship between the focal test scores and independent measure of the same construct
![Page 39: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/39.jpg)
Define discriminant evidence.
evidence that the scores on your focal test are not highly related to the scores from other tests that are designed to measure theoretically different constructs
![Page 40: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/40.jpg)
Define known groups evidence.
evidence that groups that are known to differ on the construct do differ on the test in the hypothesized direction
![Page 41: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/41.jpg)
What are the characteristics of the different ways of obtaining validity evidence?
• norming group - the specific group for whaich the test publishers or researcher provides evidence for test validity and reliability
• it is not wise to rely solely on previously reported reliability and validity information
• the characteristics of you participants should closely match the characteristics of the norming group
![Page 42: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/42.jpg)
What are some different types of psychological tests?
• intelligence tests - the ability to think abstractly and to learn readily from experience
• personality tests - patterns that characterize and classify people– self-report - participants rate themselves– performance measures - participants perform some
real-life, observable behavior– projective measures - participants provide
responses to ambiguous stimuli
![Page 43: Chapter 6 - Standardized Measurement and Assessment](https://reader035.vdocuments.us/reader035/viewer/2022062401/5a4d1b0f7f8b9ab05998e3b7/html5/thumbnails/43.jpg)
What are some different types of educational assessment tests?• preschool assessment tests - assess the
various behaviors and cognitive skills of young children
• achievement tests - designed to measure the degree of learning that has taken place after a person is exposed to a specific learning experience
• aptitude tests - focuses on the information acquired through the informal learning that goes on through life
• diagnostic tests - identify where a student is having difficulty