programming aptitude testing materials, data analysis saeed dehnadi middlesex university

Programming Aptitude Testing Materials, Data analysis

Saeed Dehnadi

Middlesex University

I talk about:

• Test materials– Questions– Mental Models– Answer Sheet– Mark Sheet– Interpretation Algorithms

• Results– Middlesex University (initial test) - 2005– University of Newcastle (Australia) - 2006– York - 2006– Strathclyde - 2006– Middlesex University (second test) - 2006

• Collaborators• Summary

Questions 1 (single assignment)

(M1) Mental model “move value”

a = b;

• The value of b is given to a and b changes its value to zero.

a b // b 0

Ans ( a = 20 , b = 0 ) 8th Answer

(M2) Mental model“copy value”

a = b;

• The value of b is given to a and b keeps its original value.

a b // b unchanged

Correct conception of LHS RHS

Ans ( a = 20 , b = 20 ) 4th Answer

(M3) Mental model “move value”

a = b;

• The value of a is given to b and a changes its value to zero.

b a // a 0

Ans ( a = 0 , b = 10 ) 3rd Answer

(M4) Mental model“copy value”

a = b;

• The value of a is given to b and a keeps its original value.

b a // a unchanged

Ans ( a = 10 , b = 10 ) 1st Answer

Mental models (M5) (copy + add) value

a = b;

• The sum of a and b is given to a, and b keeps its original value.

a (a + b) // b unchanged

Ans ( a = 30 , b = 20 ) 2nd Answer

Mental models (M6) (move + add) value

a = b;

• The sum of a and b is given to a, and b changes its value to zero.

a (a + b) // b 0

Ans ( a = 30 , b = 0 ) 10th Answer

Mental models (M7) (copy + add) value

a = b;

• The sum of a and b is given to b, and a keeps its original value.

b (a + b) // a unchanged

Ans ( a = 10 , b = 30 ) 9th Answer

Mental models (M8) (move + add) value

a = b;

• The sum of a and b is given to b, and a changes its value to zero.

b (a + b) // a 0

Ans ( a = 0 , b = 30 ) 5th Answer

Mental models (M9) “no changes”

a = b;

• a and b keep their original values.

a unchanged // b unchanged

Ans ( a = 10 , b = 20 ) 6th Answer

Mental models (M10) Equality

a = b;

• Assignment is a simple equation, and then all equal values of a and b are acceptable.

Ans ( a = 10 , b = 10 ) and

Ans ( a = 20 , b = 20 )

both 1st and 4th answers

Mental models (M11) “Swap values”

a = b;

• a and b swap their values simultaneously.

a b a gets b’s value

b a b gets a’s value

Ans ( a = 20 , b = 10 ) 7th Answer

Models for a single assignment int a = 10; int b = 20;

a = b;

Answer Sheet (Question1)

Question 4 (two assignments)

Mental model“M1 + Sequence”

a = b;b = a;

Model is M1 that applies sequentially through both statements:

• L1) a b and b 0 then a = 20 and b = 0 • L2) b a and a 0 then b = 20 and a = 0

Single answer: a = 0 , b = 20

Mental model“M1 + Independent (M1 I)”

a = b;b = a;

• Model is M1 that applies independently for each individual line.

• L1) a b and b 0 • L2) b a and a 0

Multiple answers:a = 20 and b = 0a = 0 and b = 10

Mental model“M1 + Simultaneous-single (M1 Ss)”

a = b;b = a;

• M1 Ss Same as (M1 I), subjects only interested on Left-hand-side values of statements and ignores the right-hand-side values.

• L1) a b and b ignores • L2) b a and a ignores

Single answer: a = 20 , b = 10

Answer Sheet (question4)

Question 7 (three assignments)

Mark Sheet

Mark Sheet Interpretation(Tree Structure)

Mark Sheet Interpretation (Mode Structure)

Data sample


Barnet CollegeFirst Experiment

2005

Data provided by: Saeed DehnadiMiddlesex University

Experiment

• Subjects:– 30 students from Middlesex University

Introductory to programming (Java)– 31 students from Barnet College Further

Education Programming Course

• Time:– Week 0 of the course

• Questions– 12 questions

CategoriesWeek 0

• Consistent (45%)

• Inconsistent (39%)

• Blank (16%)

CategoriesWeek 3

• Consistent (67%)

• Inconsistent (33%)

• Blank (0%)

Stable group membership(Week3)

Data provided by: Saeed Dehnadi


Pass/Fail statistics(Initial Result)

P < = 0.01



Two populations(Initial Result)

Australian Data

University of Newcastle2006

Data provided by: SimonSchool of DCIT (Design, Communication, and

Information Technology)

Pass/Fail statistics(First glance)

P < = 0.01

Data provided by: Simon

The University of Newcastle

Pass/Fail statistics Exclude m2

P < = 0.025



Pass/Fail statistics Exclude m2

P < = 0.001



Strathclyde Data

The University of Strathclyde

2006

Data provided by: Linxiao Mu

Mental models in week 20



Value assignment

Reference assignment

Value assignment mental models in final exam

P < = 0.05



Value assignment mental models in 4 in-course tests

Data provided by: Linxiao Mu The University of Strathclyde

P < = 0.01

Reference assignment mental models in final exam

P < = 0.025



Reference assignment mental models in 4 in-course tests

Data provided by: Linxiao Mu The University of Strathclyde

P < = 0.05

YorkThe University of York

2006

Data provided by: Dimitar Kazakov

Pass/Fail statistics

P < = 0.001

Data provided by: Dimitar Kazakov

The University of York

Middlesex UniversitySecond Experiment

2006

Data provided by: Saeed Dehnadi Middlesex University

Pass/Fail (Quiz 1)

P < = 0.20



Pass/Fail (Quiz 2)

P < = 0.01



Collaborators

• Ireland - School of Computing in DCU• Australia - Charles Stuart University• Mozambique - Instituto Superior de Transportes e

Comunicações, Maputo• UK- Bournemouth University (Royal School of Signals in

Blandford) • UK – University of Birmingham • UK – University of Sheffield• UK – King’s College, London• Canada - University of Toronto• Denmark – University of Aarhus • USA – A-level in Computing, High school• UK - A-level in Computing, North West College• Germany – Computing study, teacher training Institution

Summary

• OriginalityWe are the first to have an aptitude test that works.

• ObjectivityWe have objective test materials that can be used by others.

• SpeculationStudents who can handle meaningless rules learn programming more easily.

To be continue …..

programming aptitude testing materials, data analysis saeed dehnadi middlesex university

Documents

b slide

value of b

b assignment

l2 b aanda

bs value b ab

unchanged b unchanged

value ans

th answer slide