asymmetric triangulation scaling: asymmetric mds for visualizing inter-item dependency structure
DESCRIPTION
Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure. SHOJIMA Kojiro The National Center for University Entrance Examinations [email protected]. Purpose of Research. Development of method for visualizing inter-item dependency structure - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/1.jpg)
Asymmetric Triangulation Scaling:
Asymmetric MDS for Visualizing Inter-Item Dependency Structure
SHOJIMA KojiroThe National Center for University Entrance Examinations
![Page 2: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/2.jpg)
Purpose of Research
• Development of method for visualizing inter-item dependency structure– Especially important for analyzing math test data
• Proposal: ATRISCAL– Asymmetric Triangulation Scaling
– An asymmetric multidimensional scaling
– Data: Conditional correct response rate matrix
![Page 3: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/3.jpg)
Joint correct response rate matrix
• n×n symmetry matrix• The j-th diagonal element P(j,j)=P(j) – Correct response rate of item j
• The ij-th off-diagonal element P(i,j) – Joint correct response rate of items i and j– symmetry P(i,j)=P(j,i)
Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1) P(n,2) ⋯ P(n,n)
![Page 4: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/4.jpg)
Conditional correct response rate matrix
• n×n asymmetry matrix• The j-th diagonal element P(j|j)=P(j)/P(j)=1.0• The ij-th off-diagonal element P(j|i)=P(i,j)/P(i)– The correct response rate of item j when item i is
answered correctly– P(i|j)≠P(j|i): Usually asymmetric
Item 1 Item 2 ⋯ Item nItem 1 P(1,1) P(1,2) ⋯ P(1,n)Item 2 P(2,1) P(2,2) ⋯ P(2,n)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1) P(n,2) ⋯ P(n,n)
Item 1 Item 2 ⋯ Item nItem 1 P(1,1)/P(1) P(1,2)/P(1) ⋯ P(1,n)/P(1)Item 2 P(2,1)/P(2) P(2,2)/P(2) ⋯ P(2,n)/P(2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(n,1)/P(n) P(n,2)/P(n) ⋯ P(n,n)/P(n)
Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
![Page 5: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/5.jpg)
Multidimensional scaling (MDS)
X12
X3X9
X7
X13
X10
X8
X11
X2
X4
X6
X5
X14
X1
X15
Q1
QM Q2
O
Asymmetry conditional correct response rate matrixItem 1 Item 2 ⋯ Item n
Item 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
![Page 6: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/6.jpg)
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ⊥𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= 𝑘𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ+ሺ1− 𝑘ሻ𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ (0 ≤ 𝑘 ≤ 1)
|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
X j
Xi
O
Xij
𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ= −𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ∙𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ| 𝑋𝑖𝑋𝑗ሬሬሬሬሬሬሬሬԦ+ 𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ
Relationship betweenitems i and j
![Page 7: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/7.jpg)
|𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ| = 𝑃(𝑖) |𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ| = 𝑃(𝑗)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ| = 𝑃(𝑖,𝑗)
Relationship betweenitems i and j
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
X j
Xi
O
Xij
𝑃ሺ𝑖ȁ�𝑗ሻ= 𝑃(𝑖,𝑗)𝑃(𝑗)
𝑃ሺ𝑗ȁ�𝑖ሻ= 𝑃(𝑖,𝑗)𝑃(𝑖)
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑖ሬሬሬሬሬሬሬԦ|
|𝑂𝑋𝑖𝑗ሬሬሬሬሬሬሬሬԦ||𝑂𝑋𝑗ሬሬሬሬሬሬሬԦ|
𝑐𝑜𝑠∠𝑋𝑖𝑂𝑋𝑖𝑗
𝑐𝑜𝑠∠𝑋𝑗𝑂𝑋𝑖𝑗
![Page 8: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/8.jpg)
Asymmetric correct response rate matrix
• The asymmetric matrix lacks information about the correct response rate of each item
• So we add the imaginary n+1-th item whose correct response rate is 1.0
– P(j|n+1)=P(j,n+1)/P(n+1)=P(j)– P(n+1|j)=P(j,n+1)/P(j)=1.0
Item 1 Item 2 ⋯ Item nItem 1 1 P(2|1) ⋯ P(n|1)Item 2 P(1|2) 1 ⋯ P(n|2)⋮ ⋮ ⋮ ⋱ ⋮
Item n P(1|n) P(2|n) ⋯ 1
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) P(n+1|1)Item 2 P(1|2) 1 ⋯ P(n|2) P(n+1|2)⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 P(n+1|n)Item n+1 P(1|n+1) P(2|n+1) ⋯ P(n|n+1) 1
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1
Expanded
![Page 9: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/9.jpg)
Stress function
)1(
)|(1
)(,
nn
jipp
n
jiji
||
||)|(
j
ij
OX
OXji
)()(
)()( X
X
XX U
T
SF
1
)(,
2)|()|()(n
jiji ij jijipλS
X
1
)(,
2)|()(n
jiji ijij pjiλδT
X
21
)(,arccosexp)|(1)|(1)( ij
n
jijiijpjipU
X
![Page 10: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/10.jpg)
Xij
δ (delta)Well-formed triangle
• The perpendicular foot from O falls on line segment XiXj
Not well-formed triangle
• The foot from O does NOT fall on line segment XiXj
Xi
X j
O
Xij
XiX j
O
δij=δji=1
δij=δji=0
![Page 11: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/11.jpg)
λ ( lambda )
Item 1 Item 2 ⋯ Item n Item n+1Item 1 1 P(2|1) ⋯ P(n|1) 1Item 2 P(1|2) 1 ⋯ P(n|2) 1⋮ ⋮ ⋮ ⋱ ⋮ ⋮
Item n P(1|n) P(2|n) ⋯ 1 1Item n+1 P(1) P(2) ⋯ P(n) 1
0.5
0.51
1
otherwisen
njiifij 1
,1
![Page 12: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/12.jpg)
Spatial indeterminacy and fixed coordinates
• Number of dimensions=3
• Coordinates of item n+1– (xn+1=0, yn+1=0, zn+1=1)
• Coordinate of item k, which has the lowest correct response rate– (xk=0, yk>0, zk)
• Coordinate of item l, which has a moderate P( ・ |k)– (xl>0, yl, zl)
![Page 13: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/13.jpg)
Optimization of Stress Function
• Two-stage optimization– Stage 1: Simple genetic algorithm (SGA)
– Stage 2: Steepest descent method (SDM)
The author is grateful to Dr. Akinori Okada (Tama University) for speaking about this strategy at the spring seminar of the Behaviormetric Society of Japan (Gotemba, Japan) in March 2010.
![Page 14: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/14.jpg)
Demonstration of exametrika
14
www.rd.dnc.ac.jp/~shojima/exmk/index.htm
![Page 15: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/15.jpg)
Result of Analysis: Radial Map
• Red dots– Estimated coordinates
• Orange dots – Points of intersections
of extensions of red line segments and the surface of the hemisphere
![Page 16: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/16.jpg)
Relationship betweenimaginary item n+1and item j
• P(j)→1.0• P(k)→0.0
Item j Item n+1Item j 1 1
Item n+1 P(j) 1
Xj
P(j)1
P(k)
Xk
Xn+1
O
![Page 17: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/17.jpg)
Relashinship between items i and j
• P(i)<P(j)• P(i|j)→1.0• P(i|j)→0.0
Item i Item jItem i 1 P(j|i)Item j P(i|j) 1
Xn+1
Xi
O
Xj
![Page 18: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/18.jpg)
Topographic Map
• The coordinates of orange points are projected onto the XY plane
• Voronoi tessellation
• Lift each Voronoi region by the length of the orange line segment
– Separate height with different colors
![Page 19: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/19.jpg)
Mastery Maps
• For each examinee
![Page 20: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/20.jpg)
Demonstration of exametrika
20
www.rd.dnc.ac.jp/~shojima/exmk/index.htm
![Page 21: Asymmetric Triangulation Scaling: Asymmetric MDS for Visualizing Inter-Item Dependency Structure](https://reader036.vdocuments.us/reader036/viewer/2022062314/56812bd6550346895d903f43/html5/thumbnails/21.jpg)
Thank you for listening.
SHOJIMA KojiroThe National Center for University Entrance Examinations
Tokyo Institute of Technology