developing measures in algebra foege… · 17 + 6d + 2d Ð 9 r Ð 4 = 4 r = 4 b ... what is the...
TRANSCRIPT
1
DevelopingDevelopingMeasuresMeasuresin Algebrain Algebra
Anne FoegenAnne FoegenIowa State UniversityIowa State University
Pacific Coast Research ConferencePacific Coast Research ConferenceFeb. 3 - 5, 2005Feb. 3 - 5, 2005
Funded by OSEP Field Initiated award #H324C030060Funded by OSEP Field Initiated award #H324C030060
Session OverviewSession Overview
Project backgroundProject background 5 algebra measures5 algebra measures Initial findingsInitial findings Questions and issuesQuestions and issues
Guidance and suggestionsGuidance and suggestions
2
Why Algebra?Why Algebra?
Challenging academic standardsChallenging academic standards Increased graduation requirementsIncreased graduation requirements Gatekeeper functionGatekeeper function Linked to success in technology fieldsLinked to success in technology fields ““21st century civil right21st century civil right”” (Moses & Cobb, (Moses & Cobb,
2003)2003)
Project AAIMSProject AAIMS
Algebra Assessment and Instruction--Algebra Assessment and Instruction--Meeting StandardsMeeting Standards
Descriptions of current algebraDescriptions of current algebrainstruction for students with and withoutinstruction for students with and withoutdisabilitiesdisabilities
Development and validation of progressDevelopment and validation of progressmonitoring measures in algebramonitoring measures in algebra
3
Identifying MeasuresIdentifying Measures
Compare 5 alternative measures toCompare 5 alternative measures toidentify those that are most promisingidentify those that are most promising
Initial work: examine distributions,Initial work: examine distributions,reliability, criterion validityreliability, criterion validity
Subsequent research activities willSubsequent research activities willexamine the viability of using the mostexamine the viability of using the mostpromising measures for progresspromising measures for progressmonitoringmonitoring
A - Basic/Tool Skills
9 + a = 15
a =
Simplify:
y2 + y – 4y + 3y2
Simplify:
2x + 4 + 3x + 5
12 – 6 = e
e =
12 – e = 4
e =
Simplify:
12 + ( - 8) + 3
Simplify:
(- 5) + ( - 4) – 1
!
z
5= 5
z =
6 • 9 = d
d =
Simplify:
4d + 7 d – 9 + 2
Simplify:
4(3 + s) - 7
!
54
w= 9
w =
2s = 16
s =
Simplify:
8 – (- 6) – 4
Simplify:
4 – (- 2) + 8
6 + 7 = v
v =
8m = 72
m =
Simplify:
6r – 5 – 2r + 6
Simplify:
3 (c + 2) – 2c
f – 7 = 3
f =
!
63
c
= 9
c =
Simplify:
-5 + 6 - 6
Simplify:
17 + 6d + 2d – 9
r – 4 = 4
r =
4
B - Basic ConceptsAlgebra Probe B -1 Page 1
What is the slope?
What is the y
intercept?
Fill in the
empty box:
s 3s
6 18
7 21
8
9 27
Fill in the
empty box:
n 4n+7
1 11
2 15
3
4 23
Fill in the
empty box:
b
5 2
3 0
0 -3
-2 -5
What is the slope?
What is the y intercept?
If y > 9, two
possible values for
y are _____ and
_____
6 • 4 + 1 =
________
Simplify:
7f + (2f + f)
Solve:
n + 3 = 8
n = ________
Evaluate 4 b + 2
when
b = 1 ________
and when
b = 3 ________
Write the
expression for this
phrase:
6 less th an a
number
(-2) • ( -4) =
________
Graph the expression
m > 6
Write a word
phrase for this
expression:
n + 9
8 ÷ 2 + 4 • 3 =
________
23 = ________
Write the expression for
this phrase:
9 times a number
-8 -6 -4 -2 0 2 4 6 8
C - Curriculum Sampling (open)Evaluate 5 y + x2
when y = 3 and x = 2
Simplify:
8(-4)(b)(b)(b)
Solve:
!
x
2
= ( - 6 )
Solve:
2t – 4 = 6
Graph x = 3
Graph the equation:
y + 2x = 4
Write the equation in
slope -
intercept form:
m =( -8) b = 4
Write the equation in
standard form:
(-2y) +
!
1
2
x = 4
6
A Tale of TwoA Tale of TwoRounds of DataRounds of Data
CollectionCollection……
Round 1: Spring 2004Round 1: Spring 2004
District 1: small District 1: small ““bedroom community;bedroom community;””predominately white; 16% free/reduced lunchpredominately white; 16% free/reduced lunch
High rate of student participation (82%)High rate of student participation (82%) Included 113 students in 4 types of classesIncluded 113 students in 4 types of classes
8th grade algebra8th grade algebra Algebra 1Algebra 1 Algebra 1A/1BAlgebra 1A/1B Algebra taught by special education teachers for IEPAlgebra taught by special education teachers for IEP
studentsstudents Administered probes A, B, and C and gatheredAdministered probes A, B, and C and gathered
data on criterion measuresdata on criterion measures
7
Data Collection ProceduresData Collection Procedures
Students completed 2 forms of probes AStudents completed 2 forms of probes Aand B, 1 form of C in Mayand B, 1 form of C in May
Probe administration process repeated 1Probe administration process repeated 1week laterweek later
Criterion measures included algebraCriterion measures included algebragrade, GPA, standardized test scores,grade, GPA, standardized test scores,teacher ratings, Iowa Algebra Aptitudeteacher ratings, Iowa Algebra AptitudeTest scoresTest scores
Means and Std. DeviationsMeans and Std. Deviations
N=6N=43N=49N=15N=113
3.0(3.2)
10.8(5.1)
14.7(6.7)
34.5(2.6)
15.3(9.8)
CCurric-open
9.8(4.7)
22.1(6.7)
30.8(6.7)
41.4(5.1)
27.7(10.2)
BBasic
Concepts
12.7(7.7)
14.2(5.1)
19.7(7.2)
26.5(7.4)
18.1(7.7)
ATool Skills
SpEd1A/1BAlg 18th gr.FullProbe
8
Probe DistributionsProbe Distributions
A3MNMEAN
38.0
36.0
34.0
32.0
30.0
28.0
26.0
24.0
22.0
20.0
18.0
16.0
14.0
12.0
10.0
8.0
6.0
4.0
16
14
12
10
8
6
4
2
0
Std. Dev = 7.70
Mean = 18.1
N = 113.00
Mean of all 4 B 6 min probes
50.0
47.5
45.0
42.5
40.0
37.5
35.0
32.5
30.0
27.5
25.0
22.5
20.0
17.5
15.0
12. 5
10.0
7.5
5.0
2.5
20
10
0
Std. Dev = 10.24
Mean = 27.8
N = 113.00
Avg of C across sessions
40.037.5
35.032.5
30.027.5
25.022.5
20.017.5
15.012.5
10.07.5
5.02.5
0.0
30
20
10
0
Std. Dev = 9.82
Mean = 15.3
N = 112.00
Differentiating BetweenDifferentiating BetweenGroups: Probes A, B, CGroups: Probes A, B, C
0
5
10
15
20
25
30
35
40
45
50
Form
A
Form
B
Form
C
Pro
be S
co
re
SpEd
Slower Pace
Typical
Advanced
9
Reliability of Probes A, B, CReliability of Probes A, B, C
.86.85.90†.71†C
Curric-open
BBasic
Concepts
ATool Skills
Probe
.73
.71
Week 1
.92
.86
Week 2
.86
.84
Form 1
.83
.81
Form 2
Criterion Validity for MeansCriterion Validity for Meansfor Teacher 1for Teacher 1
.62.59CCurric-open
BBasic Concepts
ATool Skills
Probe
.54
.32
Teacher Rating
.72
.52
IAAT
10
Conclusions from Round 1Conclusions from Round 1
Reliability is acceptable for probes afterReliability is acceptable for probes afterthe first task administeredthe first task administered
Technical adequacy of forms B and CTechnical adequacy of forms B and Cmore promising than form Amore promising than form A
Concerns about form CConcerns about form C Restricted range of scores for May dataRestricted range of scores for May data Lower Lower interscorerinterscorer reliability rates reliability rates Extremely time consuming to scoreExtremely time consuming to score
Round 2: Fall 2004Round 2: Fall 2004
11
District 1District 1
Lower rate of student participation (38%)Lower rate of student participation (38%)and smaller sample size (N=48)and smaller sample size (N=48)
Students completed 2 forms each ofStudents completed 2 forms each ofprobes D and E on two consecutive weeksprobes D and E on two consecutive weeks
Criterion validity data available forCriterion validity data available forteacher rating, IAAT (other measuresteacher rating, IAAT (other measurescurrently being gathered)currently being gathered)
Means and Std. DeviationsMeans and Std. Deviations
N=4N=8N=30N=5N=47
8.0(3.6)
6.4(3.8)
9.9(3.9)
11.9(3.2)
9.3(4.0)
ECurric-
MC
2.0(1.9)
4.2(2.1)
6.9(4.6)
9.3(3.7)
6.3(4.3)
DReform-oriented
SpEd1A/1BAlg 18th gr.FullProbe
12
Distributions for Probes D and EDistributions for Probes D and E
Mean of all 4 D probes
16.0
15.0
14.0
13.0
12.0
11.0
10.0
9.0
8.0
7.0
6.0
5.0
4.0
3.0
2.0
1.0
0.0
7
6
5
4
3
2
1
0
Std. Dev = 4.34
Mean = 6.3
N = 47.00
Mean of all 4 E probes
18.0
16.0
14.0
12.0
10.0
8.0
6.0
4.0
2.0
0.0
12
10
8
6
4
2
0
Std. Dev = 3.98
Mean = 9.3
N = 47.00
Reliability of Single ProbesReliability of Single Probes
ECurric- MC
DReform-oriented
Probe
.41
.40
Week 1
.43
.48
Week 2
.49
.61
Form 1
NS
NS
Form 2
13
Reliability of Aggregated ProbesReliability of Aggregated Probes
ECurric- MC
DReform-oriented
Probe
.45
.62
By Week
.53
.61
By Form
Criterion Validity for Teacher 1Criterion Validity for Teacher 1
ECurric- MC
DReform-oriented
Probe
.54
.46
Teacher Rating
.59
.61
IAAT
14
District 2District 2
Blue collar town; 85% white; 23%Blue collar town; 85% white; 23%free/reduced lunchfree/reduced lunch
64% student participation rate; 81 students64% student participation rate; 81 students 4 sections of Algebra 1A4 sections of Algebra 1A 2 sections of Algebra 1B2 sections of Algebra 1B
Means and Std. DeviationsMeans and Std. Deviations
18.0(6.8)
15.2(5.1)
16.0(5.8)
ECurric- MC
8.6(5.3)
8.9(5.4)
8.1(5.3)
DReform-oriented
N=24N=55N=79
16.1(6.5)
15.3(3.8)
15.5(4.7)
CCurric-open
18.0(6.2)
22.3(5.2)
21.0(5.8)
BBasic Concepts
14.5(4.6)
14.2(5.0)
14.3(4.8)
ATool Skills
Alg 1BAlg 1AFullProbe
15
Reliability of Single ProbesReliability of Single Probes
.56
.63
.92
.68
.70
Form 2
.57.56.46ECurric- MC
.49.72.53DReform-oriented
.44.80†NS†CCurric-open
.68.66.62BBasic Concepts
.73.78.64ATool Skills
Form 1Week 2Week 1Probe
Reliability of Aggregated ProbesReliability of Aggregated Probes
.58
.75
--
.69
.76
By Form
.63ECurric- MC
.70DReform-oriented
--CCurric-open
.72BBasic Concepts
.78ATool Skills
By WeekProbe
16
Criterion Validity for District 2Criterion Validity for District 2
.43
NS
.45
.33
.34Rating
Teacher 2IAATIAATRating
.40.61.41ECurric- MC
NSNSNSDReform-oriented
.38.61.67CCurric-open
.44.68NSBBasic Concepts
NS.46.40ATool Skills
Teacher 1Probe
Summary by Probe TypeSummary by Probe Type
.46-.57
.49-.72
.44-.92
.62-.68
.64-.78Reliability
District 2ValidityValidityReliability
.40-.61.54-.59.41-.49ECurric- MC
NS.46-.61.40-.61DReform-oriented
.38-.67.59-.62.71-.90CCurric-open
.33-.68.54-.72.73-.92BBasic Concepts
.34-.46.32-.52.71-.86ATool Skills
District 1Probe
Data in yellow collected in 2003-4; data in white collected in 2004-05.
17
Conclusions from Round 2Conclusions from Round 2
Serious concerns about probe D andSerious concerns about probe D andmismatch with instruction at these sitesmismatch with instruction at these sites(guessing)(guessing)
Reliability issuesReliability issues Round 2 data lower than Round 1Round 2 data lower than Round 1 D and E probes especially lowD and E probes especially low
Validity estimates differed by teacher inValidity estimates differed by teacher inDistrict 2District 2
Questions and IssuesQuestions and Issues
ReliabilityReliability Probe construction process (problem types,Probe construction process (problem types,
practice activities)practice activities) Student factors (motivation, fatigue, showingStudent factors (motivation, fatigue, showing
work)work) Teacher factors (buy-in)Teacher factors (buy-in)
ValidityValidity Complete analyses for Round 2 withComplete analyses for Round 2 with
additional variables (ITED, grades, IAATadditional variables (ITED, grades, IAATsubtests)subtests)
18
Common QuestionsCommon Questions
Global indicators or curriculum-sampling?Global indicators or curriculum-sampling? No clear pattern favoring one over the otherNo clear pattern favoring one over the other Most promising measures are B (global) andMost promising measures are B (global) and
E (curriculum sampling)E (curriculum sampling) Difficulty levelDifficulty level
No clear patternNo clear pattern B is an easier task; E represents currentB is an easier task; E represents current
instructional levelinstructional level
Next Steps/Future ResearchNext Steps/Future Research Explore strategies to increase reliabilityExplore strategies to increase reliability
(focuses samples of problem types per(focuses samples of problem types perquestion; student motivation; teacherquestion; student motivation; teacherencouragement)encouragement)
Examine additional criterion measures forExamine additional criterion measures forYear 2 to see if diminished magnitude ofYear 2 to see if diminished magnitude ofcorrelations holdscorrelations holds
Replicate static measures study in DistrictReplicate static measures study in District22
Conduct 10 weeks of progress monitoringConduct 10 weeks of progress monitoringin District 1in District 1