a new aptitude test -...
TRANSCRIPT
![Page 1: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/1.jpg)
A New Aptitude Test• CASL Research on Language Aptitude
DLAB 2 Increase prediction of basic proficiency (ILR 2) & predict attritionALAB Language neutral testHi-LAB Predict ultimate attainment (ILR3+/4)ATI Aptitude-by-treatment interaction: Match training to aptitude
![Page 2: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/2.jpg)
Theoretical Framework Develop a research program for getting USG language professionals to very high levels of proficiency?
Aptitude‐by‐Treatment Interaction!
Little research directly investigating aptitude for high‐level language.
![Page 3: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/3.jpg)
Uses of Aptitude Tests
3
• Selection
• Diagnosis
• Placement
• Counseling
• Tailored Training (ATI)
• Predicting performance outside classroom
• SLA Research
![Page 4: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/4.jpg)
4
Predicting Ultimate AttainmentProblem
Need to identify individuals who can attain near‐native proficiency, given motivation and opportunity.
RelevanceAdult language learning is time consuming, effortful and expensive.
SolutionCognitive abilities that underlie advanced language learning processes are measurable. High levels of cognitive abilities may predict near‐native attainment.
![Page 5: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/5.jpg)
Why is language learning so hard?
5
![Page 6: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/6.jpg)
What predicts LL success?
Talent: Cognitive
Ability
Experience
Motivation
Personality Facets
Many factors
Natural abilityDrivePerseveranceTrainingCoaching
![Page 7: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/7.jpg)
“L2 aptitude is the individual’s initial state of readiness and capacity for learning a foreign language, and probable facility in doing so [given the presence of motivation and opportunity].”
J. B. Carroll, 1981
7
Second Language Aptitude
Talent: Cognitive
Ability
![Page 8: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/8.jpg)
What language aptitude explains
• Speed of language acquisition
under intensive learning conditions (rate)
(Carroll and others following)
• Individual differences in
post‐CP ultimate attainment (ceiling)
(CASL)
8
![Page 9: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/9.jpg)
Carroll’s 4 Aptitude Constructs
DLAB MLAT PLAB
Phonetic Coding Ability
Part 2, Recognition of Stress PatternsPart 3, Foreign Language Grammar
Part 2, Phonetic Script Part 3, Spelling Clues
Part 5, Sound DiscriminationPart 6, Sound-Symbol Association
Grammatical Sensitivity
Part 3, Foreign Language GrammarPart 4, Foreign Language Concept Formation
Part 4, Words in Sentences
Part 4, Language Analysis
Rote Learning Ability
Part 3, Foreign Language Grammar
Part 1, Number Learning Part 5, Paired Associates
Part 4, Language Analysis
Explicit Inductive Language Learning Ability
Part 4, Foreign Language Concept Formation
Part 1, Number Learning
Part 4, Language Analysis
9
![Page 10: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/10.jpg)
Sources of Individual Differences
10
![Page 11: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/11.jpg)
Hi‐LAB = Cognitive Aptitude
the working memory system
11
![Page 12: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/12.jpg)
Hi‐LAB’s unique prediction
• Hi‐LAB is designed to predict the ultimate level of attainment by adult starters.
• While recognizing the importance of other individual differences and experience
• to success in language learning, the purpose of Hi‐LAB is to measure cognitive aptitude.
12
![Page 13: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/13.jpg)
Summary
• Critical period offsets disrupt language learning.
• Empirical studies show wide variation in ultimate attainment post‐critical period.
• Successful individuals may be those with language aptitude which helps to counteract critical period effects.
![Page 14: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/14.jpg)
14
InstrumentationHi‐LAB is a more direct battery
•Innovative: computer‐delivered behavioral measures (E‐prime CRO)
•Data types: accuracy, reaction time
•Not transparent to learners
•Instructions provided & response type and individual responses PRACTICED
•Fun!?
![Page 15: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/15.jpg)
Hi‐LAB Constructs and MeasuresHi-LAB Constructs and MeasuresWorking Memory
Executive Functioning
Updating Running Memory Span
Inhibitory Control Antisaccade
Stroop Simon
Task Switching Task Switching Numbers (mix cost and switch cost)
Phonological Short-term Memory Letter Span
Non-Word Span
Associative Memory Paired Associates
Long-term Memory Retrieval (Priming) ALTM Synonym
Implicit Induction (have added explicit) Serial Reaction Time (Letter Sets)
Processing Speed Serial Reaction Time (could be from other measures)
Auditory Perceptual Acuity Phonemic Discrimination: Hindi, English Pseudo-Contrastive
Phonemic Categorization: Russian
15
![Page 16: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/16.jpg)
Our iterative research approach
• Propose new constructs (motivated by theory)
• Develop measures, assess usabilityusability revise
• Gather evidence for reliabilityreliability revise
• Gather evidence for validityvalidity revise
• Gather information about testing environment construct operational tests
![Page 17: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/17.jpg)
Aims of Reliability Research
• Improve the reliability of Hi‐LAB tests
• Reduce the size of the test battery to fit within reasonable testing time limit
(7 2.5 1.5 hours)
• Regularize scoring procedures
• Examine theoretical construct validity
(factor analysis)
17Doughty et al (2007) (RS1) and (2009) (RS2)
![Page 18: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/18.jpg)
Study MethodParticipants
• RS1: 195 NSs of English (UMD)• RS2: 142 NSs of English (DLIFLC)Materials
• 42 measures 22 measures (based on RS1)• 2 parallel versions of the battery: Hi‐LAB Version A and Hi‐LAB Version B
Design
18
TestAB 2 weeks + Re-test
BA
![Page 19: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/19.jpg)
Hi‐LAB Factors and Measures
19
![Page 20: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/20.jpg)
Hi‐LAB ConstructsCognitive Construct Factor Reliable Stable
Jointly Acceptable
Short Term Memory 2 of 2 1 of 2 1 of 2
Updating 2 of 2 1 of 2 1 of 2
Inhibition ? 2 of 2 1 of 2 1 of 2
Task‐switching 2 of 2 1 of 2 1 of 2
Rote Memory 2 of 2 1 of 2 1 of 2
Perceptual Acuity 3 of 4 2 of 4 2 of 4
Speed 2 of 2 1 of 2 1 of 2
Semantic Primability 2 of 2 1 of 2 1 of 2
Implicit Induction 1 of 2 1of 2 1 of 2
Explicit Induction Letter Sets
20
20
Simon
![Page 21: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/21.jpg)
Aims of Hi‐LAB Validity Research
• Phase 1: Factor analyses to theoretical construct validity
• Phase 2: Group discrimination
• (Phase 3: Gather longitudinal validity evidence )
• Phase 4: Operational validation (October 2013)
Mislevy et al. 2010 - AUA
![Page 22: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/22.jpg)
Study Method
Aim – Investigate whether Hi‐LAB discriminates groups with known levels of L2 ability
(Pairs – propensity score matching;
Group discrimination – logistic regression)
Moderately Successful Group
Highly Successful Group
Age Gender Level of Education
Linck, Hughes, Campbell, Silbert, Tare, Jackson, Smith, Bunting & Doughty (in press, Language Learning 2013).
![Page 23: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/23.jpg)
Hi‐LAB correctly classifies high‐level and normal attainment
GROUPListening (n = 152)
Reading(n = 188 )
Either(n = 206)
High‐Level vs.Normal Attainers
**70% **59% **67%
the working memory system
** p < .01
Validity: Hi-LAB is accurate in group discrimination
Linck, Hughes, Campbell, Silbert, Tare, Jackson, Smith, Bunting & Doughty (in press, Language Learning 2013).
![Page 24: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/24.jpg)
Best predictors in regression modelListening Reading Either
Task switching
Letter Span Letter span Letter span
Paired Associates Paired Associates Paired Associates
Serial Reaction Time Serial Reaction Time Serial Reaction Time
24Linck, Hughes, Campbell, Silbert, Tare, Jackson, Smith, Bunting & Doughty
(in press, Language Learning 2013).
![Page 25: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/25.jpg)
Further Test Development
• Can the aptitude dimensions be further reduced?
• Is language aptitude distinct from IQ?
25
![Page 26: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/26.jpg)
Study methodParticipants
• 428 NSs of English
Analyses• Principal components analysis
• Hierarchical and non‐hierarchical cluster analyses
26
![Page 27: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/27.jpg)
Hi‐LAB Component Reduction?
27
Not yet …
![Page 28: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/28.jpg)
Language Aptitude distinct from IQ?CQBN Measures (n = 132)
Hi-Lab Measures (n = 428) Variable NameDescription
Processing Speed CQBN1PMPattern Matching
Serial Reaction Time CQBN2MT Matrices
Phonemic Discrimination CQBN3EU English Usage
Phonemic Categorization CQBN4WM Wording Meaning
Phonemic Cat. Learning CQBN5SPSpelling
Mix-Cost RT CQBN6DRDirections
Switch-Cost RT CQBN7AR Arithmetic
Anti-Saccade Accuracy CQBN8NN Name & Number Check
Stroop CQBN9AL Artificial Language
Letter Span
Running Span
Non-Word Span
ALTM
28
![Page 29: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/29.jpg)
Is language aptitude unique?
29
Hi‐LAB CQBN
Hi‐LAB + CQBN
Significant
Non‐significant
Hi-LAB accounts for substantial unique variance
![Page 30: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/30.jpg)
Quartimax rotated loadings, 4 Factors, Hi‐LAB only
Measure Factor 1 Factor 2 Factor 3 Factor 4
Processing.Speed.TS0.761
Serial.Reaction.Time
Phonemic.Disc0.576
Phonemic.Cat0.630
Phonemic.Learn0.364
MixCost.RT‐0.564
SwitchCost.RT0.514 0.404
Anti.ACC‐0.522
Stroop.RT
Letter.Span0.959
Running.Span0.512
NonWord.Span
ALTM‐0.528
30
![Page 31: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/31.jpg)
Conservative and less conservative analyses of # of components to retain
31
• CQBN 2 or 8 of 9 components
• Hi‐LAB 4 or 10 of the 13 components
• Hi‐LAB
and CQBN 7 or 21 of the 22 components
![Page 32: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/32.jpg)
32
32
![Page 33: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/33.jpg)
33
33
![Page 34: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/34.jpg)
3434
![Page 35: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/35.jpg)
35
![Page 36: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/36.jpg)
Theoretical Framework : ATI
Assess effectiveness
Apply treatments (training)
Identify groups based on aptitudes
Collect data on participants
All participants
Group A
Training A
Training B
Group B
Training A
Training B
36
![Page 37: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/37.jpg)
37
Aptitude‐by‐Treatment InteractionProblem
Individuals who are already selected for intelligence and motivation and have attained functional ability need to acquire higher levels of language
RelevanceHigher‐level ability is essential to the job
SolutionMatch cognitive aptitude profile to language training procedures to optimize training and boost outcome levels
QuestionTo what extent do aptitude profiles exist?
![Page 38: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/38.jpg)
Cluster Analyses
• Solutions from hierarchical and non‐hierarchical clustering algorithms indicate that some clusters may usefully be defined with respect to average aptitude profiles.
38
![Page 39: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/39.jpg)
Aptitude Profiles: 2 Cluster Solution: (1) all high (2) all low
39
![Page 40: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/40.jpg)
Aptitude Profiles: 3 Cluster Solution: (1) all high (2) all low (3) mixed (speed)
40
Mixed group strength = memory and perceptual acuity
![Page 41: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/41.jpg)
4 Cluster Solution
41
1 Cluster good almost across the board; 4th poor3rd cluster: good at only STM, Updating, Perceptual Acuity4th cluster: good only at Speed, Induction, Inhibition, Priming, STM
![Page 42: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/42.jpg)
Steps to develop ATI materialsTAILORING SECOND LANGUAGE TRAINING: STEPS
Step 1. Identify a learning problem relevant for the learners’ and instructors’ needs
Step 2. Consider the learners’ aptitude profile card
Step 3. Consider practical constraints
Step 4. Design the training
Step 5. Determine the language outcome measures (pre‐ and post‐training tests)
Step 6. Implement the training
42
![Page 43: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/43.jpg)
Example of ATI: Variety in speech
• L2 language use need: Interact with people speaking many registers and dialects of the target language.
• SLA principle: Rich and varied input is necessary for second language acquisition to high levels.
• General learning principle: Complex information is best learned in a structured way at first, moving on to variety.
• Aptitude interaction: People with high aptitude can benefit from the variety sooner. People with low aptitude may need to learn only in the structured format.
43
![Page 44: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/44.jpg)
44
![Page 45: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/45.jpg)
45
![Page 46: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/46.jpg)
46
![Page 47: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/47.jpg)
How to choose which aptitudes and treatments to investigate?
• Information about the participants– Focus on abilities that will impact language use– Target abilities that differ most
• Information from science– Research‐based training techniques– SLA measures (in addition to proficiency)– Stronger predictions for real‐world performance, beyond the lab
47
![Page 48: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/48.jpg)
Triangulation: Needs and constraints
48
Needs analysis• Use• Learning
Practical limits on interventions• time available• time for design &
implementation• instructor role• accessibility• expertise availability
Aptitude profiles• distribution• groupings?• aptitude predictions
testable by intervention
![Page 49: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/49.jpg)
Needs Analysis
• Russian is a highly‐inflected language complex inflectional morphology
– Instruction bias: focus on vocabulary, don’t pay attention to the ’endings’
– Students develop a strategy to ignore inflectional morphology
49
![Page 50: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/50.jpg)
Needs Analysis
• Grammatical case especially problematic: – Nominal inflectional morphology marks grammatical roles of nouns in
a sentence;
– Russian allows scrambling/limited constraints on word order inflections are the only reliable marking of grammatical function.
• What makes it worse:‐ existing L1 bias – English has strict SVO word‐order;‐ a universal processing strategy to interpret (or to assign) the role of
the agent to the first noun (for L2: VanPatten’s First Noun Processing Strategy, FNPS);
‐ Shallow Structure Processing hypothesis (Clahsen) – L2 semantic processing bias (e.g., animacy)
50
![Page 51: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/51.jpg)
Pre‐testPerception TasksTask 1: Picture/audio match (CASES/WO) (auditory presentation)‐ to assess whether students can correctly interpret the meaning of SVO and OVS
sentences.
Мама читаетмальчику. Мальчик читает маме.
The Mom.NOM reads to the boy.ACC The boy.NOM reads to the Mom.ACC
51
![Page 52: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/52.jpg)
Pre‐test
Task 2: Plausibility task (CASES/WO) (visual presentation)‐ similar to Task 1, but allows for inanimate objects to be subjects in grammatically
correct sentences. Мужчину держит ложка.
The man.ACC holds the spoon.NOM
Task 3: Auditory Grammaticality Judgment task (CASES)‐ assesses how sensitive learners are to case violations (e.g., two nouns both
marked for nominative case).
На завтрак бабушка варила кашу.
For breakfast the grandmother.NOM was cooking oatmeal.ACC.
52
![Page 53: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/53.jpg)
Pre‐test
Production TasksTask 1: Picture description/translation (CASES/WO) (visual presentation)• uses English/Russian translation to force participants to produce both SVO and
OVS sentences with appropriate inflections.
Prompt: dog/girl/brush
Собаку причесывает девочка.The dog.ACC brushes the girl.NOM
Task 2: Elicited imitation (CASES/WO) (auditory presentation)‐ are able to correctly reproduce grammatical inflections.
53
![Page 54: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/54.jpg)
Aptitudes• Implicit/Explicit learning designated as possibility
• Profiles (Hi‐LAB) confirmed split in class, though not fully crossed
• All students showed strength in Explicit Learning
• Half also showed strength in Implicit Learning
• Intervention design adjusted to take this into account
54
![Page 55: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/55.jpg)
Intervention challenges
• “Case” too broad for narrow intervention• Instrumental case chosen• Within‐subjects design needed• Two target constructions for instrumental available for contrast: passive subjects and instruments
• Explicit intervention: what instructor already had planned
• Implicit intervention: “murder mystery” logic puzzles
55
![Page 56: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/56.jpg)
Design details
• Study design:– All participants received both interventions– Implicit intervention only on instruments (not passive subjects)
– Matched training (implicit training for implicit aptitudes) should show greater benefit
– Training designed to be useful for all participants (just more useful for some)
– Existing instruction plans disturbed very little
56
![Page 57: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/57.jpg)
Intervention details
• Critical design features:– Implicit: no overt instruction on or excessive attention towards instrumental case
– Meaning‐based: students needed to comprehend the statements for meaning to engage in the task
– Exposure: frequent use of weapons in sentence provided high‐density exposure to the instrumental case, using instruments
57
![Page 58: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/58.jpg)
Murder mystery
• Like “passive Clue”
• Had to guess murderer, location, weapon, etc.
• Students listened to a number of statements (between 4 and 15)
• Used witness statements to eliminate possibilities
58
1. The mechanic.NOMki lled the cook.ACC Механик убил повара.2. The nurse.NOMki lled the cook.ACCwith a belt.INSTR Медсестра убила кого‐то ремнём.3. The nurse.NOM ki lled the victim.ACC in the school.PREPwith a spade.INSTR Медсестра убила кого‐то в школе лопатой.4. Someone.NOMki lled the professor.ACC in the school.PREP Кто‐то убил профессора в школе.5. The mechanic.NOMki lled the victim.ACCwith a spade.INSTR Механик убил кого‐то лопатой.6. The nurse.NOMki lled the professor.ACC Медсестра убила профессора.7. The mechanic .NOM ki lled the professor.ACCwith a belt.INSTR Механикубил профессораремнём.
![Page 59: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/59.jpg)
Outcome measure details
• Auditory grammaticality judgment task (GJT)
• Taps into knowledge of students without asking specific questions on case
• Auditory chosen to match auditory intervention
• Quickly designed and implementable
• Provides specific measure for outcomes
59
![Page 60: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/60.jpg)
Procedure
• Monday: Pre‐test GJT
• Tuesday, Wednesday, Thursday: Implicit instruction,3 x 20 minutes
• Friday: Mid‐test GJT (after the implicit instruction)
• Later on Friday: Explicit instruction, 1 hour• Tuesday (following week): Post‐test GJT (after the explicit instruction)
• 3 weeks later: Delayed post‐test GJT
60
![Page 61: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/61.jpg)
Results
• Learners with explicit and implicit learning profiles take advantage of both types of instruction, and (tentatively) benefit more from the combination of the two.
• Learners with an explicit aptitude profile also show a gain in accuracy as a result of the explicit instruction.
• The information on the individual differences among learners predicts different learning outcomes as a result of methodological intervention
feasibility of effective task tailoring to individual strength for accelerated learning outcomes
61
![Page 62: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/62.jpg)
RESEARCH PROBLEM
• Learners have incomplete knowledge of lexical items, and when they sound similar to other items, they confuse the two.
• ATI study will leverage participants’ aptitude profiles to determine whether instruction designed to match their aptitudes will promote increased mastery of confusable words as measured by accuracy & automaticity of perception and retrieval of words.
62
![Page 63: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/63.jpg)
63
Study Design
![Page 64: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/64.jpg)
• What constitutes exceptional ability?
• How many different aptitude profiles are there?
• At what level should the intervention be – approach, technique?
• Are there genetic correlates of language aptitude?
64 64
![Page 65: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/65.jpg)
Solutions?
• Select people with talent, that is cognitive aptitude for language
• Tailor training to aptitude profiles, language learning needs, and workplace language needs
• Train the cognitive aptitude
• Train the job performance
• Coach the job performance
65
![Page 66: A New Aptitude Test - gralseminar2013.weebly.comgralseminar2013.weebly.com/uploads/1/8/6/2/18628688/gral_doughty... · A New Aptitude Test • CASL Research on Language Aptitude DLAB](https://reader031.vdocuments.us/reader031/viewer/2022021711/5cb773db88c99379328c085b/html5/thumbnails/66.jpg)
66 66
Thank you!Questions?