: flexible open learner modeling sergey sosnovsky, paws@sis@pitt
Post on 18-Jan-2016
219 Views
Preview:
TRANSCRIPT
:Flexible Open Learner Modeling
Sergey Sosnovsky,PAWS@SIS@PITT
References
• Susan Bull, , UK.
• Mabbott, A. & Bull, S. (2004). Alternative Views on Knowledge: Presentation of Open Learner Models, ITS2004, 689-698.
• Mabbott, A. & Bull, S. (2006). Student Preferences for Editing, Persuading and Negotiating the Open Learner Model, ITS2006, 481-490.
• Kerly, A. & Bull, S. (2006). The Potential for Chatbots in Negotiated Learner Modelling, ITS2006, 443-452.
Outline
• Open User Model• Flexi-OLM:
– Viewing LM– Editing LM– Persuading LM– Negotiating LM– Multiple LM Presentations– Evaluation
• Demo
• What:Visualization of the learner model, providing a learner with a mechanism to explore it, sometimes, negotiate it.
• Why:When a learner is engaged in the analysis of the learner model he is reflecting upon his domain knowledge and experience re-calling and re-considering ideas of which he is aware.
Open Learner Modeling
Flexi-OLM• Models student understanding of basic C programming• Uses color coding for telling students about the concept
knowledge levels: – While – limited understanding– Pale yellow – somewhat limited– Yellow/green – moderate– Bright green – excellent– Red – misconception– Grey – insufficient data
• Large topics include smaller concepts. Clicking on a topic in the model brings more detailed concept-wise information about this topic understanding.
• Knowledge are assessed with the help of short questions• After playing with the system:
– Questions correspond to only one concept– No knowledge Inference between concepts– Very simple modeling formula (seems like average with linear
thresholds for knowledge levels)
Editing LM
• Flexi-OLM allows direct editing of LM
• Possible scenarios for this will be:– new learner wishes to inform
the system about topics she already understands;
– the learner grasps a concept outside the system and wants LM to reflect this;
– the learner correctly guesses a series of answers => LM has a higher knowledge level than she believes she has.
Scrutinizing and Persuading LM• Less direct method:
– A learner registers a disagreement with the LM and propose a change
– Flexi-OLM explains its believes by presenting the evidence supporting these beliefs
– If the learner still wishes to proceed, she has the opportunity to ‘persuade’ the LM by answering a series of test questions.
• Possible Scenarios:– A learner believes her
knowledge may be different than the system asserts, but lacks the confidence to edit it unchallenged,
– A learner seeks the satisfaction of proving the system wrong
Negotiating LM• Flexi-OLM supports conversation-based negotiation of LM:
– A learner is chatting with the system (as he thinks)– Flexi-OLM maintains separate believe models for LM and for a
learner– It is ensured that the same dialogue moves are available to both
parties => Each party:• has full control over their own beliefs,• can challenge the other’s belief, • can seek justification for the other’s belief (on the LM side justification
is based on the past learner’s answers),• may request justification before changing their own beliefs,• may ultimately decide to leave their belief unchanged.
– If the difference between LM’s and Student’s beliefs is:• 1 level – The LM accepts the learner’s suggestion• 2 levels – A compromise is offered
(of changing both beliefs by one level)• 3 levels – The systems seeks a justification
(the learner will be asked to answer a question)
• [possible hack] – gradual change of the LM belief by one level
Negotiating LM (cont.)The “Wizard-of-Oz” Paradigm
• Human experimenter takes the role of the chatbot – “Wizard”
• The “Wizard” follows a protocol designed to ensure:– that responses to students remained consistent between users,
and– that the ‘chatbot’ was believable to users.
• To enact the protocol, the Wizard was provided with some 350 pre-authored ‘chatbot’ negotiation initiations and responses to user inputs.
• Typical conversation:
LM Presentations in Flexi-OLM
• hierarchy, a logical grouping of related concepts;• lectures, where topics are organized the same
as in the related lecture course;• concept map showing relationships between the
topics;• prerequisites, showing possible sequences for
studying topics;• index, an alphabetical list;• ranked, where topics are listed in order of
proficiency;• textual summary.
Experiment 1 (2004)
• Two questions:– Is it beneficial for students to have a choice over
presentation of open LM, or it causes information overflow?
– Is there any strong preference for a particular LM view among individuals and if so, can it be predicted on the basis of learning style?
• 23 undergraduate students• Experiment flow:
– Pre-test (control flow in C) to populated LM– Browsing session, where students can choose among
4 different presentations
Experiment 1: Results
Experiment 2 (2006)
• Main question:– What are the students preferences concerning
editing, negotiating and persuading LM?
• 8 third-year undergraduate students• Experiment flow:
– Initial testing to populate LM– 1-hour LM exploring session (edit & persuade)– 20 minutes of negotiating with LM
Editing LM (full student
control)Negotiating LM
Persuading LM (full system
control)
Experiment 2: Results
34
11
23
36
20
Sum
Experiment 3 (2006)
• The goal:– To explore the feasibility of using a chat-based interface in an
OLM system
• 11 final year undergraduates• Experiment flow:
– Self-assessment of student knowledge of each concept, providing the initial user’s beliefs to the system.
– Interacting with the system – to populate LM and provide it with its beliefs about student knowledge
– Then students were shown how to use the chatbot and asked to interact with it for at least 20 minutes.
– Post-experiment questionnaire
Experiment 3(2006)
Thank You for Your Questions
top related