Download - NCME April 11, 2007
![Page 1: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/1.jpg)
NCME April 11, 2007
Assessing Knowledge in a Learning Space:
Validity and/or Reliability
Jean-Claude Falmagne
University of California, Irvine
![Page 2: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/2.jpg)
Eric CosynChris DobleNicolas ThieryHasan Uzun
![Page 3: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/3.jpg)
Content:
Assessing Knowledge in a Learning Space What is a Learning Space? The Fringe Theorem The Projection Theorem Uncovering a Knowledge State Validity/Reliability of the Assessment Summary and Discussion
![Page 4: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/4.jpg)
Content:
Assessing Knowledge in a Learning Space What is a Learning Space? The Fringe Theorem The Projection Theorem Uncovering a Knowledge State Validity/Reliability of the Assessment Summary and Discussion
![Page 5: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/5.jpg)
Content:
Assessing Knowledge in a Learning Space What is a Learning Space? The Fringe Theorem The Projection Theorem Uncovering a Knowledge State Validity/Reliability of the Assessment Summary and Discussion
![Page 6: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/6.jpg)
Assessing Knowledge in a Learning Space
In its principle, a learning space enables a very accurate assessment engine capable of uncovering the knowledge state of a student among
many possible ones.
![Page 7: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/7.jpg)
In beginning algebra, for example, there are around 250 different types of problems (concepts, skills) that a student must master.
![Page 8: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/8.jpg)
What do we mean by a “type of problem”?
![Page 9: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/9.jpg)
Here is an example of a type of word problem.
One story line--out of 5--goes as follows:
"Abdul works mowing lawns and raking. He earns $5.40 anhour for mowing and $4.40 an hour for raking. How much will he earn for 5 hours of mowing and 1 hour of raking?"
![Page 10: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/10.jpg)
Here is an example of a type of word problem.
One story line--out of 5--goes as follows:
"Abdul works mowing lawns and raking. He earns $5.40 anhour for mowing and $4.40 an hour for raking. How much will he earn for 5 hours of mowing and 1 hour of raking?”
Another story line for the same type of word problem is:
"In the past month Felipe rented 1 video cassette and 7 DVDs.The rental price for the video cassette was $2.80 . The rental price for each DVD was $3.20 . What is the total amount that Felipe spent on video cassette and DVD rentals in the past month?"
![Page 11: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/11.jpg)
Let us say that there are five such story lines for that problem type. When selecting an instance of that word problem, one of those five story lines has to bechosen randomly, and the particular numbers involved(dollar amounts and whole numbers), must also be chosen. (Integer dollar amounts are excluded.)
Overall, there are 28,125 instances to choose from, randomly, for that particular word problem.
![Page 12: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/12.jpg)
In other words, in this particular case, one type ofword problem corresponds to 28,125 items in the usual sense of a standardized test.
![Page 13: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/13.jpg)
A knowledge state is a feasible subset of thatset of 250 problem types.
![Page 14: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/14.jpg)
The number of feasible knowledge states is typically very large. In beginning algebra, this number is on the order of 107.
![Page 15: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/15.jpg)
Despite this very large number of states, it is nevertheless possible to assess a student’sknowledge state in the topic, accurately, in the span of 25-35 questions.
![Page 16: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/16.jpg)
Content:
Assessing knowledge in a learning space What is a Learning Space?
![Page 17: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/17.jpg)
A learning space is a collection K of knowledge states.Each knowledge state is a subset of a set Q of problem types. Any knowledge state contains all the types of problems that some student in the population of referencemight be capable of solving.
forming the collection K.
The set Q
(Not a partition)
![Page 18: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/18.jpg)
There are serious constraints on this concept. We suppose that a learning space always contains the empty set and the full set Q of problem types.
Thus, it is in principle possible for a student not to know anything, and for some other student to know everything.
![Page 19: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/19.jpg)
Moreover, any learning space K satisfies two pedagogically reasonable principles:
![Page 20: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/20.jpg)
First Principle:
If a knowledge state K is included in some otherknowledge state K’
K
K’
![Page 21: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/21.jpg)
First Principle:
If a knowledge state K is included in some otherknowledge state K’, then
K
K’
![Page 22: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/22.jpg)
First Principle:
If a knowledge state K is included in some otherknowledge state K’, then
K
K’
![Page 23: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/23.jpg)
First Principle:
If a knowledge state K is included in some otherknowledge state K’, then
K
K’
![Page 24: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/24.jpg)
First Principle:
If a knowledge state K is included in some otherknowledge state K’, then
K
K’
![Page 25: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/25.jpg)
First Principle:
If a knowledge state K is included in some otherknowledge state K’, then
K
K’
![Page 26: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/26.jpg)
First Principle:
If a knowledge state K is included in some otherknowledge state K’, then
K
K’
there exists at least one chain of states going from K to K’, each differing from the previous one by a single problem, so that learning can proceed from K to K’ by mastering the concepts one at a time.
![Page 27: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/27.jpg)
In other words, the material must belearnable one step at a time, no matterwhere you are, and where you want to go beyond that.
![Page 28: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/28.jpg)
Second Principle:
If a problem type c is learnable from someknowledge state K
K
![Page 29: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/29.jpg)
Second Principle:
If a problem type c is learnable from someknowledge state K
c
K
![Page 30: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/30.jpg)
Second Principle:
If a problem type c is learnable from someknowledge state K and if that knowledge state K is included in some knowledge state K’,
c
K
K’
![Page 31: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/31.jpg)
Second Principle:
If a problem type c is learnable from someknowledge state K and if that knowledge state K is included in some knowledge state K’,
c
K’
then c should also be learnable from thestate K’.
![Page 32: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/32.jpg)
In other words:
Knowing more can only help to learn something new.
![Page 33: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/33.jpg)
These axioms have a Guttman’s scale flavor, but thatintuition would be misleading: if a Guttman’s scale interpretation of a learning space was attempted for any realistic learning space, literally billions ofGuttman’s scales would ensue. Such a construction would not be helpful.
Also, a formal concept of skill maps has been developed within this framework (Doignon and Falmagne, 1999). So far, we do not use it.
![Page 34: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/34.jpg)
A learning space onthe set of problems{a,b,c,d,e,f }
![Page 35: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/35.jpg)
Content:
Assessing Knowledge in a learning space What is a Learning Space? The Fringe Theorem
![Page 36: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/36.jpg)
Except for the empty state (the empty set) and the full set Q, each knowledge state has two fringes: the outer fringe and the inner fringe.
![Page 37: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/37.jpg)
Except for the empty state (the empty set) and the full set Q, each knowledge state has two fringes: the outer fringe and the inner fringe.
The outer fringe of a state K in a learning space is theset of all problem types q such that K + {q} is also astate.
In other words, intuitively: the outer fringe of a state K is the set of all problem types that are learnable from K.
Outer fringe
![Page 38: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/38.jpg)
Except for the empty state (the empty set) and the full set Q, each knowledge state has two fringes: the outer fringe and the inner fringe.
The inner fringe of a state K in a learning space is theset of all problem types q such that K - {q} is also astate.
In other words, intuitively: the inner fringe of a state K is the set of all problem types that could have beenlearned last.
Inner fringe
![Page 39: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/39.jpg)
Outer fringe of state K set of problem types learnable from K
Inner fringe of state K set of most recently learned problem types in K
![Page 40: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/40.jpg)
EXAMPLE OF AN INNER AND OUTER FRINGE OF A STATE IN ARITHMETIC
(High Points)
![Page 41: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/41.jpg)
The Fringe Theorem:In a learning space, any state is characterized (or can be accurately summarized) by its inner and outer fringe.
A FUNDAMENTAL RESULT
![Page 42: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/42.jpg)
The Fringe Theorem:In a learning space, any state is characterized (or can be accurately summarized) by its inner and outer fringe.
A FUNDAMENTAL RESULT
The two fringes provide a very good summary of an assessment, immediately giving an entry intolearning: the student may be offered the choiceof studying any problem type in the outer fringe ofhis or her state.
![Page 43: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/43.jpg)
The Fringe Theorem:In a learning space, any state is characterized (or can be accurately summarized) by its inner and outer fringe.
A FUNDAMENTAL RESULT
The two fringes provide a very good summary of an assessment, immediately giving an entry intolearning: the student may be offered the choiceof studying any problem type in the outer fringe ofhis or her state. One can obviously also give a total score, or a score in any of the subtopics.
![Page 44: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/44.jpg)
![Page 45: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/45.jpg)
![Page 46: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/46.jpg)
![Page 47: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/47.jpg)
Content:
What is ALEKS? What is a Learning Space? What are the fringes? The Fringe Theorem The Projection Theorem
![Page 48: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/48.jpg)
What is a Projection of a Learning Space?
Intuitively, it is a smaller learning space giving a macroscopic view of the bigger one, a concept useful for a placementtest or similar tests.
![Page 49: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/49.jpg)
A learning space onthe set of problems{a,b,c,d,e,f }
![Page 50: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/50.jpg)
We begin by selecting a subsetof {a,b,c,d,e,f}. We first choose Problem type b.This choice determines a bipartition of the set of states.
![Page 51: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/51.jpg)
![Page 52: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/52.jpg)
Projection Theorem (Informal Statement.) If K is a learning space on Q, then any subset T of Q defines a partition of Q such that each class of that partition corresponds to a subset of T forming a knowledge state of a new learning space U on T. This learning space has fewer states and fewer problem types if T is a proper subset of Q.
The learning space U is called the projection of thelearning space K under T.
![Page 53: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/53.jpg)
Projection Theorem (Informal Statement.) If K is a learning space on Q, then any subset T of Q defines a partition of Q such that each class of that partition corresponds to a subset of T forming a knowledge state of a new learning space U on T. This learning space has fewer states and fewer problem types if T is a proper subset of Q. Moreover, each class of the partition is a collectionsatisfying all the conditions of a learning space, except that the empty set is not necessarily a state. (Dan Cavagnaro, 2006, submitted to the JMP.)
The learning space U is called the projection of thelearning space K under T.
![Page 54: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/54.jpg)
Why a projection?
![Page 55: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/55.jpg)
Think about the learning space for the complete mathematics curriculum for all of K-12.
We have such a learning space, based on around900 problems.
![Page 56: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/56.jpg)
Using a projection, based on selecting a subsetof problem types, a placement test can be manufactured in a matter of a couple of days.
![Page 57: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/57.jpg)
Using a projection, based on selecting a subsetof problem types, a placement test can be manufactured in a matter of a couple of days.
With a little more programming, it would be in acouple of hours.
![Page 58: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/58.jpg)
Content:
Assessing knowledge in a learning space What is a Learning Space? The Fringe Theorem The Projection Theorem Uncovering a Knowledge State
![Page 59: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/59.jpg)
Content:
What is ALEKS? What is a Learning Space? What are the fringes? The Fringe Theorem The Projection Theorem Uncovering a Knowledge State
NOTE THAT I COMPLETELY BYPASS THE VERY DIFFICULT PROBLEM OF HOW WE CAN, IN PRACTICE, CONSTRUCT A LEARNING SPACE.
![Page 60: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/60.jpg)
Content:
What is ALEKS? What is a Learning Space? What are the fringes? The Fringe Theorem The Projection Theorem Uncovering a Knowledge State
![Page 61: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/61.jpg)
Schema of an assessment algorithm
![Page 62: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/62.jpg)
Schema of an assessment algorithm
The most informativequestion is asked
ALSO, BEFORE I FORGET: NO MULTIPLE CHOICE.
![Page 63: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/63.jpg)
Schema of an assessment algorithm
Bayesian Rule
![Page 64: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/64.jpg)
Content:
What is ALEKS? What is a Learning Space? What are the fringes? The Fringe Theorem The Projection Theorem Uncovering a knowledge state Is the assessment valid/reliable?
![Page 65: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/65.jpg)
Beginning Algebra
Basic idea: in each assessment, an additional,randomly selected problem is asked, the response to which is not taken into account to gauge the knowledge state. However, the assessed state can be used to predict the student’s response.
![Page 66: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/66.jpg)
![Page 67: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/67.jpg)
Beginning Algebra
210,102 assessments204 problem types, out of 250
![Page 68: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/68.jpg)
Beginning Algebra
210,102 assessments204 problem types, out of 250
46 problems were discarded because the relevant data were too meager.
![Page 69: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/69.jpg)
Beginning Algebra
210,102 assessments204 problem types, out of 250.
85% college students15% high school students.
So, we have on the average210,102/250 = 1,030… assessmentsper problem.
![Page 70: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/70.jpg)
204 beginning algebra problems
![Page 71: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/71.jpg)
204 beginning algebra problems
![Page 72: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/72.jpg)
On the suggestion of Brian Junker (Thanks!)we also used the log odds ratio to evaluate thecorrelation.
![Page 73: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/73.jpg)
Based on 185 problems(19 problems were discardedbecause one of the cellscontained a zero)
![Page 74: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/74.jpg)
However, the tetrachoric coefficient and thelog odds ratio tell essentially the same story.
![Page 75: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/75.jpg)
![Page 76: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/76.jpg)
Finally, still another, very important way of evaluating the accuracy of the assessment is to look at the efficiency of the learning process guided by the assessment.
![Page 77: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/77.jpg)
The assessments are taking place at well chosen moments determined by the system or by the teacher.But, in addition to those assessments, the system also updates the student’s knowledge states continuously.
This is what some of you call “a system that teaches as it assesses.”
![Page 78: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/78.jpg)
When a student has successfully learned a new problem type, the system adds that problem type to the student’s state, and displays the new statewith its outer fringe on the computer screen.
![Page 79: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/79.jpg)
![Page 80: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/80.jpg)
The student then picks, in that outer fringe, a newproblem type to learn.
A natural questions is: how successful is the systemat predicting what the student is ready to learn?
![Page 81: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/81.jpg)
Distribution of the conditional probabilities that a student successfully learnsa problem type chosen in the outer fringe of his or her knowledge state.Data based on 1,564,296 learning occasions (for the 250 problem types).
![Page 82: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/82.jpg)
Distribution of the conditional probabilities that a student successfully learnsa problem type chosen in the outer fringe of his or her knowledge state.Data based on 1,564,296 learning occasions (for the 250 problem types).
![Page 83: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/83.jpg)
Major differences with a psychometric test
1. The objectives and the philosophy.
![Page 84: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/84.jpg)
Major differences with a psychometric test
1. The objectives and the philosophy.2. The theories.
![Page 85: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/85.jpg)
Major differences with a psychometric test
1. The objectives and the philosophy.2. The theories.3. The result of the assessment or the test.
![Page 86: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/86.jpg)
Major differences with a psychometric test
1. The objectives and the philosophy.2. The theories.3. The result of the assessment or the test.4. The principles underlying the construction of the two instruments.
![Page 87: NCME April 11, 2007](https://reader034.vdocuments.us/reader034/viewer/2022051417/56814e2d550346895dbb90d3/html5/thumbnails/87.jpg)
THANK YOU!