intelligent systems and learning centred design
DESCRIPTION
What are intelligent systems for enhancing learning? How do we design them so as to enhance a certain learner's experience? This talk starts with pedagogy theories focusing on designing learning experiences according to learner characteristics. Then it boldly connects them to specific design methodologies, which aims at producing usable and pedagogically effective intelligent systems for delivering enhanced learning material and experiences. Then the real pulp starts: how we designed the intelligent system of the TERENCE FP7 EU project for designing adequate learning material, specifically, smart games for reasoning about stories and adequate to the needs of poor comprehenders. The talk concludes reflecting upon the speaker's change in research areas possibly due to several factors, such as personal reasons, interests and project-dependent requirements. The on-going work is also briefly illustrated, that is, how to design smart games not only for learners but also with learners, engaging and including all.TRANSCRIPT
![Page 1: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/1.jpg)
LEARNER CENTRED DESIGN
INTELLIGENT SYSTEMSand
Rosella Gennarihttp://www.inf.unibz.it/~gennari
Distinguished Speakers Oxford Women in CS
![Page 2: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/2.jpg)
Settingtechnology enhanced learning
ClimaxTERENCE case study
Resolutionreflections
Story outline
![Page 3: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/3.jpg)
Settingtechnology enhanced learning
ClimaxTERENCE case study
Resolutionreflections
Story outline
![Page 4: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/4.jpg)
TECHNOLOGY ENHANCED LEARNING
Technology Enhanced Learning (TEL) is the usage of technology for supporting a learning experience
Herby we take a narrow view: intelligent TEL =
Artificial Intelligence (AI) technology based products for supporting a learning experience
![Page 5: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/5.jpg)
TEL 4 LEARNING EXPERIENCE
How can we design technological products that supports their users’
learning experience?
![Page 6: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/6.jpg)
LET'S SEE EDUCATORS' VIEWPOINT...
Maria Montessori (1870-1952)
Paraphrasing her words, adequate tasks that come in a prepared environment, designed on top of the learner characteristics can effectively support the learner’s learning
was the first Italian woman physician and educator, best known for Montessori pedagogy
![Page 7: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/7.jpg)
ousability of technology learning products
opedagogical effectiveness of technology learning products
TEL 4 LEARNING EXPERIENCE
Adequate tasks that come in a prepared environment designed on top of the learner characteristics can effectively support learning
![Page 8: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/8.jpg)
HOW TO DESIGN USABLE AND PEDAGOGICALLY EFFECTIVE TEL
![Page 9: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/9.jpg)
Based on UCD process diagram (© Tom Wellings)
requirement specification
designevaluation
plan
models + prototypes
intermediate product
final product
HOW TO DESIGN USABLE AND PEDAGOGICALLY EFFECTIVE TEL
USABILITY + P. EFFECTIVENESS
![Page 10: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/10.jpg)
Settingtechnology enhanced learning
ClimaxTERENCE case study
Resolutionreflections
Story outline
![Page 11: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/11.jpg)
TERENCE DESIGN
Based on UCD process diagram (© Tom Wellings)
requirement specification
designevaluation
plan
models + prototypes
intermediate products
final products
TERENCE was an FP7 TEL project blending user centred and evidence based design
USABILITY + P. EFFECTIVENESS
![Page 12: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/12.jpg)
THE PROBLEM
‣ TERENCE developed an adaptive learning system (ALS) that, via a learner GUI, recommends poor comprehenders
- its learning material, i.e., books of stories and games
- its learning tasks, i.e., reading and playing
‣ so as to stimulate their reading comprehension
‣More than 10% of primary school children, older than 8, are diagnosed with deep text comprehension problems
‣ They are referred to as poor comprehenders
![Page 13: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/13.jpg)
THE TERENCE WORLD
a d e q u a t e b o o k o f s t o r i e s
s i g n i n
a d e q u a t e s m a r t g a m e s
r e w a r d
![Page 14: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/14.jpg)
TERENCE INTELLIGENT TEL PRODUCTS
ALS LayerGUI Layer
Learner
EducatorExpert
Learner GUI
Expert GUI
Persistence Layer
OpenRDF
UserManager
OpenRDF
StoryManager
OpenRDF
GameManager
OpenRDF
VisualisationManagerillustrations
NPL
Reasoner
AdaptiveEngine
Visualisation
������������
Reasoning Module
AnnotationModule
VisualisationModule
game generation
adaptation to learners
![Page 15: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/15.jpg)
Requirements Prototypes Analytic + small ev. Int. prod. Analytic+large Fin. prod.
TERENCE DESIGN
![Page 16: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/16.jpg)
TERENCE DESIGN
Requirements Prototypes Analytic + small ev. Int. prod. Analytic+large Fin. prod.
![Page 17: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/17.jpg)
DATA GATHERING METHODS
‣Data for designing the learning material and tasks were from
- contextual inquiries with
‣ IT & UK diagnosis
‣ as well as IT & USA evidence-based medicine therapy experts
- field studies with educators and primary school children
4510
~ 500
![Page 18: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/18.jpg)
CHARACTERI
STICS
Persona Name: Carol
Age: 8
Classroom: year 4
RC levels: low reading levels
Rural/Urban: urban
Deaf/hearing: deaf
First Language: Italian Sign Language
Cochlear Implantation (if deaf): yes
Degree of hearing loss (if deaf): profound
Motor skills (if deaf): average
Summary of the class represented by this persona
A younger deaf girl who is very enthusiastic about using new technology (such as iPhone and IPad) and who adores her Nintendo DS. Her reading RC levels are very low, but she reads together with her parents to learn new words and spelling. She also likes to do many other things such as drawing, taking care of her pets and going to the park.
Quote “I really love Mario and Luigi. And I would love to have an iPhone and an IPad, like my dad.”
Personality Open
Role in classroom Active
Role out of the classroom
Active
Console/Technology Carol and her sister watch TV after school. They like Tom & Jerry, Ben 10, Hello Kitty and Mickey Mouse Clubhouse.
Carol sometimes uses the computer, but only to play minilab games. Her computer is in her bedroom, but her parents don’t allow her to use it all of the time. She can use it only one hour per day. Carol’s dad has an iPhone and an IPad, and Carol would really like to use those as well, but her dad tells her she is a bit too young. Carol likes watching him with his IPad and iPhone though.
Carol doesn’t use a mobile phone.
Carol plays games on the computer and on her Nintendo DS. She plays by herself. She likes the mini-clip games on the computer, and Mario Kart and brain training games on her DS.
She likes games with non-photorealistic human avatars, and prefers fantasy avatars to animal avatars.
Socio-Cultural Level of his/her own family
Medium
School performance Carol has sever reading problems. In her class she is below average in all activities but drawing, where she feels she can truly express her intimate feelings.
Homework After school, Carol does her homework together with her mum.
LIFE
STYLE
Outdoors Activities Carol often goes to the park with her mum.
Indoors Activities Games on the DS Carol reads sometimes. Her mum and dad help her reading in the evening. She likes some of the stories they read together, but mostly, she wants to read because she has to learn new words and spelling. Carol likes drawing and taking care of her pets. Her mum often plays with her.
Home activities Carol also likes to help her mum in the kitchen or in the garden.
Sport activities Carol practices no specific sport.
.
![Page 19: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/19.jpg)
SMART GAME REQUIREMENTSWhat for Description
Difficulty levels Macro levels for learners: - entry: character games; - intermediate: time games; - top: causality games.
Scheduling of reading and playing
1st silent reading; 2nd playing smart games; 3rd playing relaxing games
Constraints on actions Learners should get faster, hence a game has a maximal resolution time
Progress and feedback Monitor and give learners (1) visible idea of progress, (2) explanatory feedback, (3) recall their attention and solicit them to give a resolution (in time)
Representation Production can be impaired hence promote resolution via visual representation and reasoning
![Page 20: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/20.jpg)
Requirements Prototypes Analytic + small ev. Int. prod. Analytic+large Fin. prod.
TERENCE DESIGN
![Page 21: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/21.jpg)
SMART GAME REQUIREMENTSWhat for Description
Difficulty levels Macro levels for learners: - entry: character games; - intermediate: time games; - top: causality games.
Scheduling of reading and playing
1st silent reading; 2nd playing smart games; 3rd playing relaxing games
Constraints on actions Learners should get faster, hence a game has a maximal resolution time
Progress and feedback Monitor and give learners (1) visible idea of progress, (2) explanatory feedback, (3) recall their attention and solicit them to give a resolution (in time)
Representation Production can be impaired hence promote resolution via visual representation and reasoning
![Page 22: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/22.jpg)
who is the actor of … ? what does (a main) character do?
when does … happen in relation to a central event?
why does the central event happen?
![Page 23: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/23.jpg)
SMART GAME REQUIREMENTSWhat for Description
Difficulty levels Macro levels for learners: - entry: character games; - intermediate: time games; - top: causality games.
Scheduling of reading and playing
1st silent reading; 2nd playing smart games; 3rd playing relaxing games
Constraints on actions Learners should get faster, hence a game has a maximal resolution time
Progress and feedback Monitor and give learners (1) visible idea of progress, (2) explanatory feedback, (3) recall their attention and solicit them to give a resolution (in time)
Representation Production can be impaired hence promote resolution via visual representation and reasoning
![Page 24: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/24.jpg)
points for each smart
coins for all smartunlocked if read+play
visual feedback
![Page 25: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/25.jpg)
Instructions Questions Motivational Interaction
Choices Choices for learner Fixed event
Solutions Choices that are either correct (c) or wrong (w)
Feedback Interaction Consistency Explanatory Solution
Smart points Proportional to the learner’s ability in the game level
Relaxing points
Constant
Avatar Happy/sad states
Time solution constant interaction constant
Rules States of the system, actions of the learner, constraints
What for Description
Difficulty levels Macro levels for learners: - entry: character games; - intermediate: time games; - top: causality games.
Scheduling of reading and playing
1st silent reading; 2nd playing smart games; 3rd playing relaxing games
Constraints on actions
Learners should get faster, hence a game has a maximal resolution time
Progress and feedback
Monitor and give learners (1) idea of progress, (2) explanatory feedback, (3) recall their attention and solicit them to give a resolution (in time)
Representation Production can be impaired hence promote resolution via visual representation and reasoning
![Page 26: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/26.jpg)
Requirements Prototypes Analytic + small ev. Int. prod. Analytic+large Fin. prod.
TERENCE DESIGN
![Page 27: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/27.jpg)
EXAMPLE METHODS IN TERENCE
APPROACH WITH WHOM EXAMPLE METHODS WHEN
analyticalHMI experts or domain experts
heuristic evaluationformative, summativeexpert evaluation
cognitive walk-through
small-scale learnersobservations
formativethink aloud
large-scale learners field studies summative
![Page 28: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/28.jpg)
EXAMPLE METHODS IN TERENCE
APPROACH WITH WHOM EXAMPLE METHODS HOW
analyticalHMI experts or domain experts
heuristic evaluationformative, summativeexpert evaluation
cognitive walk-through
small-scale learnersobservations
formativethink aloud
large-scale learners field studies summative
![Page 29: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/29.jpg)
G1. interfaces follow general design guidelines
G2. interfaces support the user’s next step to achieve a task
G3. interfaces provide users with timely feedback
Instructions are not under focus and cannot be easily read
Game question and possible resolutions should be proximally close
Game question and possible resolutions should be proximally close
Evaluation of interfaces
![Page 30: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/30.jpg)
Requirements Prototypes Analytic + small ev. Int. prod. Analytic+large Fin. prod.
TERENCE DESIGN
![Page 31: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/31.jpg)
problem: 256 stories,
each with ~12 games
Smart game design
![Page 32: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/32.jpg)
how can we automatise the
development of smart games via AI (and, hopefully, be efficient)?
Smart game design
![Page 33: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/33.jpg)
enriched annotations
story
annotations
ALS LayerGUI Layer
Learner
EducatorExpert
Learner GUI
Expert GUI
Persistence Layer
OpenRDF
UserManager
OpenRDF
StoryManager
OpenRDF
GameManager
OpenRDF
VisualisationManagerillustrations
NPL
Reasoner
AdaptiveEngine
Visualisation
������������
Reasoning Module
AnnotationModule
VisualisationModule
Semi-automated generation
![Page 34: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/34.jpg)
enriched annotations
story
text
text
text
text
annotations
Semi-automated generationALS LayerGUI Layer
Learner
EducatorExpert
Learner GUI
Expert GUI
Persistence Layer
OpenRDF
UserManager
OpenRDF
StoryManager
OpenRDF
GameManager
OpenRDF
VisualisationManagerillustrations
NPL
Reasoner
AdaptiveEngine
Visualisation
������������
Reasoning Module
AnnotationModule
VisualisationModule
![Page 35: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/35.jpg)
text
text
text
text
image
image image image
enriched annotations
story
annotations
Semi-automated generation
![Page 36: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/36.jpg)
games
template visual
text
text
text
text
image
image image image
enriched annotations
story
annotations
Semi-automated generationALS LayerGUI Layer
Learner
EducatorExpert
Learner GUI
Expert GUI
Persistence Layer
OpenRDF
UserManager
OpenRDF
StoryManager
OpenRDF
GameManager
OpenRDF
VisualisationManagerillustrations
NPL
Reasoner
AdaptiveEngine
Visualisation
������������
Reasoning Module
AnnotationModule
VisualisationModule
![Page 37: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/37.jpg)
text
story
text + visual
games
AUTOM. MANUAL AUTOM.
Semi-automated generation
![Page 38: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/38.jpg)
Requirements Prototypes Analytic + small ev. Int. prod. Analytic+large Fin. prod.
TERENCE DESIGN
![Page 39: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/39.jpg)
APPROACH WITH WHOM EXAMPLE METHODS WHEN
analyticalHMI experts or domain experts
heuristic evaluationformative, summativeexpert evaluation
cognitive walk-through
small-scale learnersobservations
formativethink aloud
large-scale learners field studies summative
EXAMPLE METHODS IN TERENCE
![Page 40: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/40.jpg)
APPROACH WITH WHOM EXAMPLE METHODS WHEN
analyticalHMI experts or domain experts
heuristic evaluationformative, summativeexpert evaluation
cognitive walk-through
small-scale learnersobservations
formativethink aloud
large-scale learners field studies summative
EXAMPLE METHODS IN TERENCE
![Page 41: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/41.jpg)
LARGE-SCALE STUDY DESIGN
Common design of the intervention with TERENCE:
‣ how: pretest/posttest design, with experimental and control groups
‣ hypothesis: TERENCE improves reading comprehension measured with standardized text comprehension tests
ControlExperimental
![Page 42: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/42.jpg)
A 3-PHASE INTERVENTION
‣ Pre-test for (1) assessing txt comprehension, (2) initialising TERENCE
![Page 43: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/43.jpg)
A 3-PHASE INTERVENTION
‣ Pre-test for (1) assessing txt comprehension, (2) initialising TERENCE
‣ Stimulation phase for experimental group with usage sessions so that each -lasts < 45 minutes for attention needs
![Page 44: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/44.jpg)
A 3-PHASE INTERVENTION
‣ Pre-test for (1) assessing txt comprehension, (2) initialising TERENCE
‣ Stimulation phase for experimental group with usage sessions so that each -lasts < 45 minutes for attention needs
-and requires (1) reading
![Page 45: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/45.jpg)
A 3-PHASE INTERVENTION
‣ Pre-test for (1) assessing txt comprehension, (2) initialising TERENCE
‣ Stimulation phase for experimental group with usage sessions so that each -lasts < 45 minutes for attention needs
-and requires (1) reading (2) playing smart
![Page 46: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/46.jpg)
‣ Pre-test for (1) assessing txt comprehension, (2) initialising TERENCE
‣ Stimulation phase for experimental group with usage sessions so that each -lasts < 45 minutes for attention needs
-and requires (1) reading (3) playing relaxing(2) playing smart
A 3-PHASE INTERVENTION
![Page 47: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/47.jpg)
‣ Post-test (pedagogical only) for re-assessing text comprehension
‣ Pre-test for (1) assessing txt comprehension, (2) initialising TERENCE
‣ Stimulation phase for experimental group with usage sessions so that each -lasts < 45 minutes for attention needs
-and requires (1) reading (3) playing relaxing(2) playing smart
A 3-PHASE INTERVENTION
![Page 48: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/48.jpg)
The experimental group is of 344 learners:
‣Avezzano: 270 learners:
- 7-9 years old: 118
- 9-11 years old: 152
‣ Pescina: 74 learners:
- 7-9 years old: 37
- 9-11 years old: 37
‣ They were tested (January-February), stimulated (March-May), and re-tested (may-June)
EXPERIMENTAL GROUP IN IT
![Page 49: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/49.jpg)
Pre-post performances for text comprehension (dependent variable) were as follows:
‣ Pescina:
- pre: 14 poor comprehenders (20.59%)
- post: 6 poor comprehenders (8.82%)
‣ Avezzano:
- pre: 15 poor comprehenders (5.95%)
- post: 2 poor comprehenders (0.79%)
MAIN RESULTS IN IT
PrePescina Avezzano
5,95%
20,59%
‣ Wilcoxon signed-rank test supports that differences are statistically significant
- Pescina: z=-4.904, p<0.0001
- Avezzano: z=-2.266, p=0.0234
![Page 50: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/50.jpg)
EXAMPLE METHODS IN TERENCE
APPROACH WITH WHOM EXAMPLE METHODS HOW
analyticalHMI experts or domain experts
heuristic evaluationformative, summativeexpert evaluation
cognitive walk-through
small-scale learnersobservations
formativethink aloud
large-scale learners field studies summative
![Page 51: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/51.jpg)
EXPERT EVALUATION
Experts of pedagogy: 1 coordinator; 9 evaluators
Sophie'comes'down'the'steps
He had never been beaten before, since he only ever raced with kids who were smaller and slower than him. He wanted a rematch, so the two boys set off again. Ben was paddling as fast as he could, still he didn’t make it to the wall before Luke. It was completely unfair, he thought. Luke was so much faster. No sooner had they climbed out of the water, than he saw his sister coming down the steps. She was smiling at Ben and gave him a playful pat on the shoulder. She also gave Ben a friendly speech about winners and losers.
revise selection of solutions
revise selection of central event
How-to:
1. each pair of evaluators read a story, and edited its games
2. the coordinator revised their work
3. a pair of evaluators was blindly assigned revised games, and another the manually created games
Main edit tasks:
(1) creation of missing games (~recall)
(2) revision of games (~precision)
![Page 52: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/52.jpg)
From D4.2 and D4.3 technical annex
overall assessment of generation
text
story
text + visual
games
revision of Automated Reasoning (AR) selection of central events and solutions
revision of Natural Language Processing (NLP) of text
text
text text text
Edit tasks in details
![Page 53: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/53.jpg)
Requirements Prototypes Analytic + small ev. Int. prod. Analytic+large Fin. prod.
TERENCE DESIGN
![Page 54: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/54.jpg)
From D4.2 and D4.3 technical annex
overall generation
text
story
text + visual
games
AR selection of central events and solutions
revision of NLP text
text
text text text
Analyses of evaluation results
![Page 55: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/55.jpg)
AR selection of central events for games:
>Results: only in 15 out of 250 cases (6%), it was necessary to select a different central event than the automatically generated one
From D4.2 and D4.3 technical annex
>Implications for AR: none picked up
Automated part evaluation-based re-design
![Page 56: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/56.jpg)
AR selection of plausible solutions:
>Results: out of 140 changes of selection of solutions, the majority was for wrong solutions
- generate a wrong solution from correct one by changing participants, e.g.,
<correct_sentence id="2">The man ran and fell on the ground.
</correct_sentence>
<wrong_sentence id="2wh1">Peter ran and fell on the ground.
</wrong_sentence>
>Implications for WP4: new heuristics for wrong plausible solutions in the last part of Y3,
From D4.2 and D4.3 technical annex
Automated part evaluation-based re-design
![Page 57: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/57.jpg)
Overall generation: development times:
>Results for revision time:
- 12’6” per game instance:
↑ 12’8” for time games
↓ 10’6” for who games
>Results for creation time:
- avg. 23” per game instance
text
storytext + visual
games
From D4.2 and D4.3 technical annex
>Implications for AR: the semi-automated development process seems to be promising for optimising development times
Automated part evaluation-based re-design
![Page 58: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/58.jpg)
Game over1st 2nd 3
Sep. 2011 December 2012 September 2013
Sophie'comes'down'the'steps
He had never been beaten before, since he only ever raced with kids who were smaller and slower than him. He wanted a rematch, so the two boys set off again. Ben was paddling as fast as he could, still he didn’t make it to the wall before Luke. It was completely unfair, he thought. Luke was so much faster. No sooner had they climbed out of the water, than he saw his sister coming down the steps. She was smiling at Ben and gave him a playful pat on the shoulder. She also gave Ben a friendly speech about winners and losers.
revise selection of solutions
revise selection of central event
Requirements+for Description
Dif$iculty*levels Macro*levels*for*learners:4*entry:*character*games;4*intermediate:*time*games;4*top:*causality*games.*
Scheduling*of*reading*and*playing
1st*silent*reading;* 2nd* playing* smart* games;*3rd*playing*relaxing*games
Constraints*on*actions
Learners* should* get* faster,* hence* a* game* has* a* maximal*resolution+time
Progress*and*feedback
Monitor* and* give* learners* (1)* idea* of* progress,* (2)*explanatory*feedback,*(3)*recall*their*attention*and*solicit+them*to*give*a*resolution*(in*time)
Representation Production*can*be* impaired* hence*promote* resolution*via*visual*representation+and+reasoning
Instruc(ons Ques%onsQues%ons Mo%va%onalMo%va%onalMo%va%onalMo%va%onal Interac%onInterac%on
Choices Choices3for3learnerChoices3for3learnerChoices3for3learnerChoices3for3learnerChoices3for3learner 3Fixed3event3Fixed3event3Fixed3event
Solu(ons Choices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%onsChoices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%onsChoices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%onsChoices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%onsChoices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%onsChoices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%onsChoices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%onsChoices3or3their3combina%ons3that3are3correct/wrong3(c/w)3solu%ons
Feedback Interac%on Consistency3(c/w)Consistency3(c/w)Consistency3(c/w) ExplanatoryExplanatoryExplanatory Solu%on
Smart6points Propor%onal3to3the3learner’s3ability3in3the3game3levelPropor%onal3to3the3learner’s3ability3in3the3game3levelPropor%onal3to3the3learner’s3ability3in3the3game3levelPropor%onal3to3the3learner’s3ability3in3the3game3levelPropor%onal3to3the3learner’s3ability3in3the3game3levelPropor%onal3to3the3learner’s3ability3in3the3game3levelPropor%onal3to3the3learner’s3ability3in3the3game3levelPropor%onal3to3the3learner’s3ability3in3the3game3level
Relaxing6points ConstantConstantConstantConstantConstantConstantConstantConstant
Avatar Happy/sad3statesHappy/sad3statesHappy/sad3statesHappy/sad3statesHappy/sad3statesHappy/sad3statesHappy/sad3statesHappy/sad3states
Time solu%on3constantsolu%on3constantsolu%on3constant interac%on3constantinterac%on3constantinterac%on3constantinterac%on3constantinterac%on3constant
Rules States3of3the3system,3ac'ons3of3the3learner,3constraintsStates3of3the3system,3ac'ons3of3the3learner,3constraintsStates3of3the3system,3ac'ons3of3the3learner,3constraintsStates3of3the3system,3ac'ons3of3the3learner,3constraintsStates3of3the3system,3ac'ons3of3the3learner,3constraintsStates3of3the3system,3ac'ons3of3the3learner,3constraintsStates3of3the3system,3ac'ons3of3the3learner,3constraintsStates3of3the3system,3ac'ons3of3the3learner,3constraints
data structures
NLP+ AR1 for stories
AR1 for txt games
frameworkAR2 + NLP1 for txt games
AR2 for stories AR3 + NLP2 for txt games
requirements
![Page 59: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/59.jpg)
Settingtechnology enhanced learning
ClimaxTERENCE case study
Resolutionreflections
Story outline
![Page 60: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/60.jpg)
Till 2007
Cate
gory
Axis
AR
HMI
TEL
Game
0 4 8 12 16
Response: work areas Amsterdam U. and CWI
FBK-irstFree U. of Bolzano
![Page 61: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/61.jpg)
From 2007
Cate
gory
Axis
AR
HMI
TEL
Game
0 4 8 12 16
Response: work areas
Free U. of Bolzano
![Page 62: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/62.jpg)
Possible explanation?
![Page 63: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/63.jpg)
co-designgamification cooperative learningHOW
WHY engagement design together inclusion
childrendesigners teachersWHO
WHAT
GACOCO
treestree puzzle
![Page 64: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/64.jpg)
Gamification of protocol (tasks, subtasks and types of feedback)
![Page 65: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/65.jpg)
Gamification (competition for cooperation)MISSIONS
CHALLENGES
REWARDSwell
done!
![Page 66: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/66.jpg)
Acknowledgments to
TERENCE colleagues and schools
Current colleagues and schools
DIARY FOR PRESENTTHE TERENCE BOOK
![Page 67: Intelligent systems and learning centred design](https://reader034.vdocuments.us/reader034/viewer/2022052622/55941f8d1a28ab4a128b45ed/html5/thumbnails/67.jpg)
Settingtechnology enhanced learning
ClimaxTERENCE case study
Resolutionreflections
Story outline
?