maximizing the utility of english assessment scores · aptis-lexile linking studies • lexile...
TRANSCRIPT
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Maximizing the Utility of English Assessment ScoresNew Directions in English Language Assessment Conference: Mexico City
March 21-22, 2019
Jason Turner
Sr. Vice-President, Global Services
MetaMetrics
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Overview
• Linking Aptis with the Lexile Framework for
Reading• Aptis Teens Advanced, Aptis General, Aptis Advanced
• CEFR
• Aligning Readers and Texts• Graded Readers
• S. Korean Textbooks, U.K. and U.S. texts
• U.S. text level expectations
2
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Why Link?
• Provides supplementary information to extend the utility
of the test score and CEFR information
• Provides an additional layer of feedback, connecting the
test to real-world reading
• Provides a way of benchmarking real-world reading
demands and a way to evaluate the gap between reading
ability and reading demand of texts
3
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
What is the Lexile Framework?
4
The most widely used standard
for matching readers with with
texts, and connecting learners
to level-appropriate resources.
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Lexile Scale & Concept
Student Measures
Advanced Readers
Beginning Readers
More Complex Text
Less Complex Text
Book Measures
200L
680L
940L
1190L
1410L
1050L
230L
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Reader & Text Relationship
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 7
Obtained
through
measuring text
with the
Lexile Analyzer
Software
Evaluates Text
Characteristics
Obtained
through
Tests of
English
Reading
Ability
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 8
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis
Score Reports
• Scaled scores for each component assessed (e.g.
Reading, Writing) on a 0-50 score scale
• CEFR levels reported for each skill (e.g. Listening,
Reading)
• Overall CEFR level when all 4 skills taken
9
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis Reading Test – Example Tasks
10
Skill Focus CEFR
Level
Task Description Cognitive Processes
Inter-sentence
cohesion
A2 Reorder jumbled sentences
to form a cohesive text
Careful reading, global,
inferencing, building a mental
model
Reading short
paragraphs-main idea
B1 Matching description of
person with main idea of
paragraph
Careful reading, global
Reading longer text,
integrating related
propositions
B2 Matching headings to
paragraphs within a longer
text
Expeditious reading, global
Careful reading, global
Reading across
extended sections of
related text
C1 Select text to complete gaps
in texts within two related
selections
Careful reading, global
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis-Lexile Linking Studies
• Lexile linking items administered at the same time as
Aptis reading items
• Aptis General and Advanced – October - November 2015
• Aptis Teens Advanced – November 2017 - April 2018
• Lexile linking items, N = 25 ranging in difficulty to align
with the Aptis test being linked.
11
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Sample Lexile Item970L
12
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis-Lexile Linking Studies
Samples
• Aptis General and Advanced
• 563 examinees from 8 countries
• Austria, Columbia, Great Britain, Greece, Malaysia,
Romania, Turkey, and Vietnam
• Aptis Teens Advanced
• 1,662 examinees from China
13
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis-Lexile Linking Function
Linking function determined by Rasch
(Winsteps) analysis and then smoothing at
the ends of the distribution (when necessary)
14
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis General Linking Function
15
0
200
400
600
800
1000
1200
1400
1600
1800
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis Advanced Linking Function
16
0
200
400
600
800
1000
1200
1400
1600
1800
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis Teens Advanced Linking Function
17
0
200
400
600
800
1000
1200
1400
1600
1800
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50
CEFR
B1
805L
985L
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis CEFR Levels on the Lexile Scale
18
0
200
400
600
800
1000
1200
1400
1600
1800
2000
A1 A2 B1 B2 C1 C2
Lexile
Measure
s
Advanced
Teens Advanced
General
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Maximizing Utility
19
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Provide Targeted Practice
A Placement Test places your
customer on the Lexile scale
Your Student
Your Content
Your measured content provides targeted practice for your
customers
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
What Else Do Lexile Measures Tell Us?
Lexile Measures allow a customer to compare their current ability with real
world English reading demands and opportunities.
Reading Demand Examples in the World
Your customer
Cu
sto
me
r Le
xile
Me
asu
re
Reading Demand Examples
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Lexile Reading Stories:
What does reading demand look like around the world?
Daily Reading
Reading Demand Examples in the World
College Entrance
Career and Workplace Readiness
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
• Benchmarking
• Commonly read materials – Lexile map
• Graded readers
• Textbooks and exam materials
• International newspapers
• U.S. grade level text expectations
23
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Lexile Map
24
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Lexile Map
25
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Graded Readers
26
• Internationally prominent graded reader publishers
• Four series
• 422 graded readers across 7 levels
• 319 had a single CEFR designation
• Research Questions
• Is there a systematic increase in reading challenge across
graded reader levels and CEFR levels?
• Is there variation across publishers in the reading challenge of
books designated at similar levels?
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Graded Readers
27
0
200
400
600
800
1000
1200
1400
A C D A B C A B C D A B C D A B
Lexile
Measure
Series and CEFR Level
A1 A2 B2B1 C1
Series and CEFR Level
A1 A2 B2B1 C1
Series and CEFR Level
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Graded Readers – Aptis Teens CEFR Levels
28
0
200
400
600
800
1000
1200
1400
A C D A B C A B C D A B C D A B
Lexile
Measure
Series and CEFR Level
A1 A2 B2B1 C1
Series and CEFR Level
A1 A2 B2B1 C1
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Textbooks and Exam Materials
• English Medium Contexts in China – Research Study
• 10 textbooks were analyzed for text complexity
• Biology, chemistry
• English language, World modern history
• Maths core and extended, Pure Maths 1, Pure Maths 2 & 3
• Mechanics
• 18iGSCE exam papers taken at English medium schools
• Research Questions
• Does the complexity in the texts match the exam?
• Is the complexity comparable to the reading material in other
English-speaking and non-English speaking countries?
29
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 30
Comparing with Aptis Teens Examinee Scores
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 31
Comparing with Aptis Teens Examinee Scores
Aptis Teens Advanced
Mean B2 (1033L)
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 32
English Textbooks used in South Korea
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 33
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 34
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 35
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Lexile Text Measures by US K-12 Grade
Levels
36
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Lexile Text Measures and US K-12 Text
Levels
37
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Summary
Linking an existing assessment to the Lexile Framework
extends the utility of the test scores
- Matching readers to text
- Benchmarking against existing text sets to anticipate
need for scaffolding
- Understanding reading demands of international
reading materials
38
-
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
THANK YOU
39