fair, fish, fowl, or fraud? - association of american ... ppt.pdf · cornell university...
TRANSCRIPT
Cornell University
Standardized Testing in College Admissions
Fair, Fish, Fowl, or Fraud?
Robert J. Sternberg
Cornell University
Cornell University
Problems with Standardized Tests
• Content
• Response format
• Proxy for SES
• Assume abilities are static
• Self-fulfilling prophecies
• Inertia
Cornell University
Main Messages
• Current tests are narrow and inequitable for certain groups
• We can do better
– Create better prediction of academic and nonacademic success
– Reduce racial/ethnic/SES differences
– Increase goodwill of applicants
Cornell University
Goal
• To measure equitably and with
high validity and reliability the
skills needed for
–Deep, reflective thinking
–Concerned, active citizenship
–Ethical leadership
Cornell University
Organization of Talk
• What is Successful Intelligence?
• Demonstration Projects
• HA*BIT: Holistic Assessment *
Broad Indicator of Talents
• Conclusions
Cornell University
Successful Intelligence
Students are SUCCESSFULLY INTELLIGENT when they have the skills to set and achieve their own goals in life according to their own standards within their life context, by:
1. Recognizing and capitalizing on strengths;
2. Recognizing and correcting or compensating for weaknesses.
Cornell University
Successful Intelligence
Capitalization and compensation enable
people to balance:
1. Adaptation to existing environments;
2. Shaping environments to improve
them;
3. Selecting new environments.
Cornell University
Successful Intelligence
• In order to succeed in school and life, you
need:
–Creative skills
–Analytical skills
–Practical skills
–Wisdom-based/ethical skills
Cornell University
Analytical Skills
• analyze
• compare and contrast
• evaluate
• explain
• judge
• critique
Cornell University
Wisdom-Based Skills
• Seek a common good
• Balance one’s own, others’, and
larger interests over the long and
short terms
• Find an ethical solution
Cornell University
The Rainbow Project with the Rainbow Project Collaborators
• Develop a battery assessing
creative, analytical (SAT/ACT-like),
and practical intellectual skills that is
–Reliable and valid
–Practical
–Enjoyable to students
Cornell University
Movies
• The Unwanted Guest
• Jerry’s “Beauty-rest Sleeper”
• Letter of Recommendation
Cornell University
Item Types
Creative
• Multiple-choice items--verbal, quantitative, and figural
• Performance tasks—written stories, oral stories, cartoon captions
Cornell University
Written Stories
• “A Fifth Chance”
• “2983”
• “Beyond the Edge”
• “It’s Moving Backwards”
• “Not Enough Time”
Cornell University
Participants
• Total sample
–777 students from 13 colleges
and universities around the
country varying widely in
geographic area, race, ethnicity,
and ability level
Cornell University
Predicting GPA: SAT + Analytical
Step 1: SAT-Verbal, SAT-Math
Step 2: Analytic (STAT)
9.8 9.9
0
5
10
15
20
Step 1 Step 2
R s
qu
are
d (
%)
Cornell University
Predicting GPA: SAT + Practical
Step 1: SAT-Verbal, SAT-Math
Step 2: Practical (STAT + Separate Tasks)
9.8
12.9
0
5
10
15
20
Step 1 Step 2
R s
qu
are
d (
%)
Cornell University
Predicting GPA: SAT + Creative
Step 1: SAT-Verbal, SAT-Math
Step 2: Creative (STAT + Separate Tasks) 9.8
18.6
0
5
10
15
20
Step 1 Step 2
R s
qu
are
d (
%)
Cornell University
Predicting GPA: SAT + Analytic, Creative, Practical
Step 1: SAT-Verbal, SAT-Math
Step 2: All Rainbow Project Items
(STAT Analytic, Practical, Creative,
Practical Tasks,
Creative Tasks)
9.8
20.9
0
5
10
15
20
Step 1 Step 2
R s
qu
are
d (
%)
Cornell University
Predicting GPA: All measures (practical before creative)*
Step 1: SAT-M
SAT-V
HSGPA
Step 2: + Analytic
Step 3: + Practical
Step 4: + Creative
*Controlling for school quality in
dependent variable
15.6 15.2 15.9
24.8
0
5
10
15
20
25
30
Step
1
Step
2
Step
3
Step
4
R s
qu
are
d
Cornell University
Amount of Each Measure That Is Predicted
by Racial / Ethnic Differences (ω²)
0.09
0.04
0.00
0.02
0.01
0.02
0.03
0.01
0.02
0.01
0.03
0.00
0.05
0.10
Verb
al
Math
ST
AT
ST
AT
Written
Sto
ries
Ora
l
Sto
ries
Cart
oons
ST
AT
Movie
s
Colle
ge
Life
Com
mon
Sense
SAT Analytic Creative Practical
Pro
po
rtio
n e
xp
lain
ed
by
ra
ce
dif
fere
nc
es
Cornell University
Kaleidoscope With Lee Coffin and the Kaleidoscope Collaborators
• Insert creative, analytical, practical,
and wisdom-based essays as part
of the Tufts-specific admissions
application in order to broaden the
way we think about applicants
Cornell University
Types of Items
• Suppose the Nazis had won WW II.
What would the world be like today?
• Design a science experiment.
• Analyze a book you have read.
• How do you convince people of
ideas they do not initially like?
• How can you help improve the
world?
Cornell University
More Creative Story
• If the Trojans had heeded Laocoon’s
advice and thrown Odysseus’ wooden
horse into the sea, they would have
defeated the Greeks at Troy. Aeneas
would then never have had reason to flee
the city, and he would never have
ventured to Italy to found Rome. Without
Rome, neither the Roman Republic nor a
Roman Empire would have existed.
Cornell University
More Creative Story
• Concrete, the arch, plumbing, and the sauna
might never have been invented. The modern
implications of Rome never having existed are
indeed drastic. Lacking even concrete floors,
people would resort to sleeping in the mud,
and, without plumbing or saunas, they would
be perpetually filthy and, generally, quite chilly.
France could not have built the base of the
Eiffel Tower without arches, so tourists would
be unable to purchase miniature collectible
Towers in Parisian convenience stores.
Cornell University
Less Creative Story
• What if the ratification of the nineteenth
amendment did not pass and women were
never given the right to vote? What would life
for women, like me, be like in the United
States? For one thing, I probably would not be
writing this essay. If women were not given
their right to vote, I probably would stop going
to school after this year and it would be unlikely
that I would receive a college education.
Cornell University
Less Creative Story
Without suffrage, my career options would be
limited, if a career were a possibility at all. My
accepted practices would be limited to staying
home and taking care of the family. Rather than
being equals, women would be subservient to men.
I might not drive, I might not dress in the way in
which I choose to, and I might not be able to live
the way I want.
Cornell University
Data
• Kaleidoscope
– improves prediction of academic and
nonacademic success
– produces an academically stronger
class
– Greatly reduces ethnic and other group
differences
– Is enjoyable for and meaningful to
applicants
Cornell University
Data
• Number of applications rose
• Bottom third of old application pool
greatly diminished; many more top
applicants
• Average SATs rose slightly
• African-American applications up 25%,
acceptances up 30%
• Hispanic-American applications and
acceptances up 15%
Cornell University
Panorama With Kyle Wray and the Panorama Collaborators
• Project done at Oklahoma State
University (when I was provost and
senior vice president) to identify students
who would not be admitted by normal
criteria of ACT and/or high school GPA
• We admitted substantial number of
students with skills for success who
otherwise would have been rejected, and
who have succeeded at OSU
Cornell University
HA*BIT
• New generation of assessment that
measures creative, analytical, practical,
and wisdom-based skills
• Scored for each of these skills as well as
writing skills and passion for learning
Cornell University
OTHER PROJECTS
• Graduate-school admissions
• Business-school admissions
• Medical-school admissions
Cornell University
Conclusions
• Traditional abilities tests are narrow and
limited
• Our new measures can
– Broaden the range of skills tested for
educational purposes
– Increase predictive validity
– Decrease ethnic-group differences
– Increase customer satisfaction
Cornell University
Conclusions
• If you are interested in learning more, or
in collaborating in experimenting with
new admissions assessments for your
institution, please contact:
• Robert J. Sternberg