introduction to the cognitive tutor authoring tools (ctat) and example-tracing tutors bruce mclaren...
TRANSCRIPT
![Page 1: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/1.jpg)
Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors
Bruce McLarenSystems Scientist, Co-Manager of the CTAT Project Team
Human-Computer Interaction InstituteCarnegie Mellon University
![Page 2: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/2.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Kinds of Computer Tutors
Intelligent tutoring systems
e.g., Sherlock Model-tracing tutors
e.g., Andes Cognitive tutors
e.g., Algebra
Tutoring systems
CAI e.g., Microsoft’s PersonalTutor
Constraint-based tutors
e.g., SQL Tutor
![Page 3: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/3.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Cognitive Tutors: Real-world success of Intelligent Tutoring Systems technology
• Cognitive Tutor Courses– Computer-based one-to-one tutoring
• Based on computational models of student thinking• Based on the concept of Model Tracing
• Used by many students in many schools– Algebra Cognitive Tutor: In over 2,000 schools in the USA,
300,000 students per year– Geometry Cognitive Tutor: In 350 schools– Most widely used intelligent tutoring system
• University created a company, Carnegie Learning, to disseminate
![Page 4: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/4.jpg)
Algebra Cognitive Tutor Sample
Use graphs, graphics calculator
Analyze real world problem scenarios
Use table, spreadsheet
Use equations, symbolic calculator
Tutor learns about each student
Tutor follows along, provides context-sensitive Instruction
![Page 5: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/5.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
ACT-R: A Cognitive Theory of Learning and Performance
• Big theory … key tenets:– Learning by doing, not by listening or watching
– Production rules represent performance knowledge:
These units are: Instruction implications:• modular • context specific
isolate skills, concepts, strategiesaddress "when" as well as "how"
Anderson, J.R., & Lebiere, C. (1998). The Atomic Components of Thought. Erlbaum.
![Page 6: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/6.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
• Cognitive Model: A system that can solve problems in the various ways students can
Strategy 1: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + ac = d
Strategy 2: IF the goal is to solve a(bx+c) = d THEN rewrite this as bx + c = d/a
Misconception: IF the goal is to solve a(bx+c) = d THEN rewrite this as abx + c = d
Cognitive Tutor Technology:Use ACT-R theory to individualize instruction
![Page 7: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/7.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor Technology:Use ACT-R theory to individualize instruction• Cognitive Model: A system that can solve problems in
the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
If goal is solve a(bx+c) = dThen rewrite as bx+c = d/a
• Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
![Page 8: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/8.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
3(2x - 5) = 9
6x - 15 = 9 2x - 5 = 3 6x - 5 = 9
Cognitive Tutor Technology:Use ACT-R theory to individualize instruction• Cognitive Model: A system that can solve problems in
the various ways students can
If goal is solve a(bx+c) = dThen rewrite as abx + ac = d
If goal is solve a(bx+c) = dThen rewrite as abx + c = d
• Model Tracing: Follows student through their individual approach to a problem -> context-sensitive instruction
Hint message: “Distribute a across the parentheses.”
Bug message: “You need tomultiply c by a also.”
• Knowledge Tracing: Assesses student's knowledge growth -> individualized activity selection and pacing
Known? = 85% chance Known? = 45%
![Page 9: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/9.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
CTAT motivation: Make tutor development easier and faster!• Development costs of instructional technology are, in
general, quite high– E.g., ~300 dev hours per hour of instruction for Computer Aided
Instruction (Murray, 1999)
• Cognitive Tutors: – Large student learning gains as a result of detailed cognitive
modeling– ~200 dev hours per hour of instruction (Koedinger et al, 1997)– Requires PhD level cog scientists and AI programmers
• Solution: Easy to use Cognitive Tutor Authoring Tools (CTAT)
Murray, T. (1999). Authoring Intelligent Tutoring Systems: An Analysis of the state of the art. The International Journal of Artificial Intelligence in Education, 10, 98-129.
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big
city. The International Journal of Artificial Intelligence in Education, 8, 30-43.
![Page 10: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/10.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
How to reduce the Authoring Cost?
• Less programming, more automation– Drag & drop interface construction– Demonstration-based programming
• Human-Computer Interaction methods– User studies, summer schools, informal & formal
comparison studies
• Exploit tools already in use– Component-based architecture & standard inter-
process communication protocols
![Page 11: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/11.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Authoring with CTAT• Cognitive Tutors
– Difficult to build; for programmers– General for a class of problems
• Example-Tracing Tutors– Much easier to build; for non-programmers– Limited to a single problem instance
• New: “Mass Production” feature greatly facilitates authoring of multiple problem instances (in Excel)
• Typical Approach– Build Example-Tracing Tutor first
• Cognitive Task Analysis• Rapid prototypes - perhaps sufficient for the problem
– Develop Cognitive Tutor from Example-Tracing Tutors• Examples guide planning• Serve as semi-automated test cases
![Page 12: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/12.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Where does CTAT fit into the Realm of Authoring Tools?
Murray survey (1999) discusses 7 categories of authoring tools for intelligent tutoring systems (ITS), including:
• Device Simulation - RIDES (Munro et al 1997)• Tutoring Strategies - REDEEM (Ainsworth et al 2003;
Major et al 1997)• Domain Expert System - Demonstr8 (Blessing 1997)
Cognitive Tutor Authoring Tools:• “Domain Expert System” category of authoring systems• No commercially successful tools for authoring model-
tracing, Cognitive Tutors• First to provide functionality to simplify & speed cog.
tutor development
![Page 13: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/13.jpg)
Behavior Recorder
Data Shop
CTAT’s Modular Architecture
Student Interface(or external problem-solving environment)
Cool Modes
CyclePad
Java Swing
Flash
Learner Management System
LMS
Tutor Shop
Cognitive Model Development Tools
TDK (Lisp-based)
Jess tools (Java-based)
Editor
Eclipse
Tutor Engine
Tertle (Lisp-based) + Model Tracer
Jess (Java-based) + Model Tracer
Example Tracer (Java-based)
Example Tracer (Flash-based)
GUI Builder
IntelliJ
Dreamweaver
Code Warrior
Netbeans
Flash MX 2004
![Page 14: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/14.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
CTAT’s Track Record• CTAT-based tutors have been used in
experiments in Geometry, Chemistry, Chinese, and French
• Over 200 users
• Three releases in past year (1.4, 1.5, 1.6)
• 13 papers published during 2005-2006
• CTAT Web site– 36,000 unique visitors in 2005– 8,800 so far in 2006
• CTAT Downloads– 840 in 2005 (27% CMU & Pitt)– 952 so far in 2006 (4.6% CMU & Pitt)
![Page 15: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/15.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
http://ctat.pact.cs.cmu.edu
![Page 16: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/16.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
PSLC Example-Tracing Tutors built with CTAT
Chinese - tone study
Chinese - listen and readFrench culture
FrenchChemistry (stoichiometry)
Geometry (self-assessment) Geometry (on-line testing)
![Page 17: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/17.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
PSLC Case study - Chemistry (Stoichiometry)
• Goals:
o Test hypotheses about personalization and worked examples
o Exercise and test the Cognitive Tutor Authoring Tools (CTAT) in a live, web-based experiment
• Results:
o Students learned: Significant difference between pre and post
o No sig differences due to personalization or worked examples
McLaren, B. M. et al. (2006). Studying the Effects of Personalized Language and Worked Examples in the Context of a Web-Based Intelligent Tutor. In the Proceedings of the 8th International Conference on Intelligent Tutoring Systems, Jhongli, Taiwan, June 26-30, 2006.
![Page 18: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/18.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
PSLC Case study - Chemistry (Stoichiometry)
Behavior Recorder
Flash authoringenvironment with
tutor interface shown
Web browser withExample-Tracing tutor
embedded in HTML
![Page 19: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/19.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
CTAT Features used in Case Study • Web delivery
– Use of Macromedia Flash MX 2004 to build Student Interface
– Tutor Shop handles problem sequencing on the web
• Template-based tutor development– Use of Excel lowers skill threshold even more– More efficient authoring– Easier maintenance
• Support for experiments– Pre/post tests adminstered and graded automatically– CTAT-built tutors log all student-tutor interactions– PSLC Data Shop provides storage and data analysis
facilities
![Page 20: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/20.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Development time comparison• Past estimates of ITS development
200:1 for robust beta versions used in real classrooms
• Example-Tracing Tutor Development25:1 for initial alpha versions: (Koedinger et al, 2004)
# Of PseudoTutors
DesignTime
Dev.Time
InstructionalTime
Design/Devto Instr.
Economics 11 3600 2190 180 32.2
Math Assistments 20 810 1170 98 20.2
LSAT 3 240 3000 180 18.0
Language Learning 8 210 575 50 15.7
Totals 4860 6935 508 23.2
• If creating a robust beta doubles dev time, we would have ~50:1 ratio -- a reduction of 4 times!
Koedinger, K., Aleven, V., Heffernan, N., McLaren, B. M., and Hockenberry, M (2004). Opening the Door to Non-Programmers: Authoring Intelligent Tutor Behavior by Demonstration;. In the Proceedings of the Seventh International Conference on Intelligent Tutoring Systems (ITS-2004).
![Page 21: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/21.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
CTAT Example-Tracing Tutor Web-Delivery Options
• Java– WebStart– Utility may depend on the situation– We are still gaining experience with this; can be a bit
tricky
• Flash (from Macromedia)– With plug-in, compatible with virtually all browsers– Easy to deploy
![Page 22: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/22.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Flash or Java for Authoring and Delivery?
Cost Web Delivery
Look-and-Feel of Student Interface
Maturity Availability ofDevelopers
Flash - License must be purchased from Macromedia(Academic: $100, $150, for authoring only)
+ More compatible across platforms & browsers
+ Emphasis on presentation; Easy to include movies, graphics, other visuals (Currently 6 widgets)
-First released as part of CTAT in Feb 2005; Much newer than Java
- Less established programming environment, so less programmers available
Java + NetBeans and Java are free downloads
- Less compatible; Use of apps like WebStart for web delivery
- Visual elements not as crisp; more difficult to include other media (Currently 15+ widgets)
+ Available for over 2 1/2 years
+ More established programming language, so more programmers available
![Page 23: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/23.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Movie Showing How an Example-Tracing Tutor is built
![Page 24: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/24.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
Hands-On Assignment: Build your own fraction addition Example-Tracing Tutor• To get started:
– Start Flash MX2004 (From Start menu: All Programs:Programming Tools:MX2004)
– Start the Behavior Recorder (Double click the “Behavior Recorder for Flash” icon on the desktop)
– In Flash open the partially built fraction-addition GUI (W:/Cognitive Tutor Authoring Tools/Projects/Examples/Exercise1-ExampleTracingTutor/fractionAddition.fla)
• To finish the GUI: Follow the hand-out “Building a student interface for fraction addition” (Because you have a partially built user interface, you can just review the first three pages; begin work in the middle of page 4)
• To build the Example-Tracing Tutor: Follow the hand-out: “Building a student interface for fraction addition”
![Page 25: Introduction to the Cognitive Tutor Authoring Tools (CTAT) and Example-Tracing Tutors Bruce McLaren Systems Scientist, Co-Manager of the CTAT Project Team](https://reader036.vdocuments.us/reader036/viewer/2022062301/56649e535503460f94b49720/html5/thumbnails/25.jpg)
PSLC Summer School July-August, 2006© Vincent Aleven and Bruce McLaren, 2006
THE END (Of “Introduction to CTAT and Example-Tracing Tutors”)