Download - Supporting Flexible Competency Frameworks
Supporting Flexible Competency Frameworks
Erica Melis, Arndt Faulhaber, Ahmad Salim Doost and Carsten Ullrich
DFKI GmbH, SJTU
Supporting Flexible Competency Frameworks - Carsten Ullrich
Environment - ActiveMath
Web-based Learning Environment for Mathematics
Domain/Content Model
User ModelTeaching/Pedagogical
Model
Supporting Flexible Competency Frameworks - Carsten Ullrich
Supporting Flexible Competency Frameworks - Carsten Ullrich
Competency Systems
Solving some exercises require different skills than others
Multitude of Systems• Bloom (Benjamin Bloom, 1956): Knowledge, Comprehension, ...
• PISA (OECD, 1999): think mathematically, argue mathematically, solve problems mathematically, ...
• Revision of Bloom (Anderson & Krathwohl et al., 2001)
• Skills (Pardos, Heffernan et al., e.g. 2007)
No "one fits all"• Different systems for different purposes / perspectives
• E.g. more coarse grained and simple to understand vs. more fine-grained and detailed but harder to use
Supporting Flexible Competency Frameworks - Carsten Ullrich
Competency Systems II
Competency systems differ in various ways• Structure
• e.g. 1-dim Skill vs. 2-dim Pair of (Cognitive Process, Knowledge Element)
• Granularity
• e.g. amount of included cognitive processes
• Perspective
• e.g. pedagogical perspective vs. psychological perspective (i.e. focus on pedagogical goals vs. focus on cognitive processes)
Supporting Flexible Competency Frameworks - Carsten Ullrich
Technology-Enhanced Learning and Competencies
Multitude of different competency systems to assess students' competences
Problems:• Reuse of learning objects
• Interoperability with respect to user modeling
=> Need to extend availability of LOs/usage-data via integration of different systems
=> System to handle/combine data with metadata using various Competency Systems
Supporting Flexible Competency Frameworks - Carsten Ullrich
Goals of our Work
Integrate different learning objects based on various competency systems
Allow for combined evaluation of data annotated with different competency systems
Devise way to integrate new Competency Systems easily
Supporting Flexible Competency Frameworks - Carsten Ullrich
Student Model Example: ActiveMath
Models the competencies of a student i.e. How well can the Student "solve" a "linear equation"
Competencies: Pairs of "Cognitive Process" and "Knowledge" e.g. "solve" + "linear equation"
Derive estimation of competency from performance in exercises Exercises have metadata for required/trained
competencies
Supporting Flexible Competency Frameworks - Carsten Ullrich
Student Model Example: ActiveMath (ctd.)
Fine-grained Overlay Model Each pair (Cognitive Process, Knowledge Item) is
competency (variable) in the student model Many knowledge items, modeled in the domain
ontology, e.g. Add fractions with same denominator Add fractions with different denominators Subtract fractions with same denominator Subtract fractions with different denominators...
Over 30 different Cognitive Processes Recognize, recall, interpret, exemplify, transform...
Supporting Flexible Competency Frameworks - Carsten Ullrich
Repository
“derivation”
Planner / Pedagogical Model
Course Generator
Supporting Flexible Competency Frameworks - Carsten Ullrich
Course Generation: Pedagogical Knowledge
Supporting Flexible Competency Frameworks - Carsten Ullrich
Basic Knowledge
Inserting references to • educational resources
• tools
Generating structure
Accessing information about• learning object
• the learner
About 70 rules
Supporting Flexible Competency Frameworks - Carsten Ullrich
Example / Exercise Selection
About 60 methods
Take into account• competency level
• educational level
• fields of interest
• novelty
• motivation & anxiety
Supporting Flexible Competency Frameworks - Carsten Ullrich
Formalized CG Knowledge: Scenarios
Moderate constructivist scenarios:• Discover
• Rehearse
• Connect
• Train Intensively
• Train Competencies
• Exam Simulation
Based on guidelines of instructional design• Guided Tour
≈ 300 methods and operators
Supporting Flexible Competency Frameworks - Carsten Ullrich
Scenario “Discover”
Introduce
Develop
Prove
Practice
Connect
(discover deriv)
(:method (discover ?f) () ((!startSection Discover ?f) (descriptionScenarioSection ?f) (learnFundamentalsDiscover ?f) (reflect ?f) (!endSection)))
(:method (learnFundamentalDiscover ?c) () ((!startSection Title (?c)) (introduceWithPrereqSection ?c) (developFundamental ?c) (proveSection ?c) (practiceSection ?c) (showConnectionsSection ?c) (!endSection)))
Supporting Flexible Competency Frameworks - Carsten Ullrich
(:method (introduceWithSection! ?c) () ((!startSection Introduction (?c)) (text Introduction (?c)) (motivate! ?c) (problem ?c) (insertIntroductionExample ?c) (!endSection)))
Scenario “Discover”
Introduce
Develop
Prove
Practice
Connect
Motivate
Problem
Illustrate
Prerequisites
(:method (introduceWithPrereqSection! ?c) () ((introduceWithSection! ?c) (learnPrerequisitesFundamentalsShort ?c)))
(introduceWithPrereqSection! deriv)
Supporting Flexible Competency Frameworks - Carsten Ullrich
Scenario Discover
Introduce
Develop
Practice
Connect
Reflect
Motivate
Problem
Illustrate
Prerequisites
(:method (motivate! ?c) ((learnerProperty hasEducationalLevel ?el) (learnerProperty hasAnxiety ?c ?an) (?an <= 2) (GetElement ((class Exercise) (class Introduction) (relation isFor ?c) (property hasLearningContext ?el) (property hasDifficulty very_easy)))) ((insertAuxOnceIfReady! ?element)))
Problem: rules refer to specific competency system
Supporting Flexible Competency Frameworks - Carsten Ullrich
Our Approach
Integration of additional competency systems• Have one internal competency system
• Map additional system to internal competencies
• Before processing input, translate competencies to internal competency system
Extending Planner / Tutorial Component• Parameterize with Competency System
• Assemble Courses for arbitrary Competency Systems
Supporting Flexible Competency Frameworks - Carsten Ullrich
Extending the Student Model
Mapping information "Cognitive Process"-part of competency in system A to
system B Weighting of Competency mapping (for multiple
competencies)
Supporting Flexible Competency Frameworks - Carsten Ullrich
ActiveMath supports three competency systems• Blooms
• PISA competencies
• ActiveMath competency system (revised Bloom)
Supporting Flexible Competency Frameworks - Carsten Ullrich
Extending the Pedagogical Model
Competencies are used when1. inserting a sequence of exercises or examples that
cover all competencies in order to present new content in multiple contexts
2. selecting exercise or examples for a competency given as a parameter
3. selecting an exercise or example for a specific competency.
Supporting Flexible Competency Frameworks - Carsten Ullrich
Generalizing Competencies: Old Rule
Supporting Flexible Competency Frameworks - Carsten Ullrich
Generalizing Competencies: New Rule
Store competency system as a fact
(competency-system (c1,...,cn))
Generalized method:
Supporting Flexible Competency Frameworks - Carsten Ullrich
Case 2: No Change Needed
Supporting Flexible Competency Frameworks - Carsten Ullrich
Case 3: No Change Possible
Very specific, depends on used competency systemCannot be generalized without loss of qualityIn case a competency system can encode a semantically similar case, a pedagogical expert has to add a corresponding method
Supporting Flexible Competency Frameworks - Carsten Ullrich
Conclusion
Approach to close the gap between different TEL systems / competency systems
Course Generator and Student Model can handle multiple competency systems
Data originating from learning objects based on different competency systems can be combined