1 storytelling alice: presenting programming as a means to the end of storytelling caitlin kelleher...
TRANSCRIPT
1
Storytelling Alice: Presenting Programming as a Means to the End of Storytelling
Caitlin Kelleher
Carnegie Mellon University
Pittsburgh, PA
www.alice.org
2
Shrinking Pool of Computer Scientists
3
There is a strong need for computer scientists
• Three occupational groups are expected to provide 75% of the new jobs between 2004-2014– Health care practitioner and technical
occupations– Education, training, and library occupations– Computer and mathematical science
occupations
4
Projected Science and Engineering Job Openings (new jobs and replacements)
2004-2014
Information Technology
Engineering
Life Science
Physical Science
Natural Science
Managers
5
Other Countries Predict IT Shortages Too
• Canada: the current 35,000 openings per year will grow to 89,000 by 2010. Currently, schools are producing < 7000 per year.
• UK: 50% drop in students since 2000 will lead to hiring difficulties.
6
Computer Science supports progress across many disciplines
7
Simply, we need more computer scientists
…and more diverse computer scientists
8
Broad reach creates a pragmatic need for diversity
9
Lack of diversity can lead to unintentional exclusion
10
Research goal: Design programming
environments that motivate a larger and more diverse group of people to learn to program.
11
Related Work
12
Related WorkSimplify the language:
BASIC
13
Related WorkSimplify the language:
BASIC
Prevent syntax errors: Cornell Program
Synthesizer
14
Related WorkSimplify the language:
BASIC
Prevent syntax errors: Cornell Program
Synthesizer
Make programming concrete:
Karel, Logo
15
Related WorkSimplify the language:
BASIC
Prevent syntax errors: Cornell Program
Synthesizer
Make programming concrete:
Karel, Logo
Construct programs with graphical objects:
Lego Mindstorms
16
Related WorkSimplify the language:
BASIC
Prevent syntax errors: Cornell Program
Synthesizer
Make programming concrete:
Karel, Logo
Construct programs with graphical objects:
Lego Mindstorms
Introduce new programming paradigms:
Smalltalk, Pascal
17
No matter how easy something is, people still need a reason to
want to do it.
18
Approach:Develop programming
environments that present programming as a means to an end that the target audience is
motivated to pursue.
19
Solve the harder problem: make programming motivating for an
under-represented group
• There are many important and under-represented groups in Computer Science.
• Women are currently the largest.
20
Female CS pool is shrinking quickly.
1973 2004
21
Critical Time: Middle School
• In 4th grade, the numbers of girls and boys who like math and science are about equal.
• By the end of 8th grade, twice as many boys as girls are interested in math and science.
22
Approach
Present programming as a means to the end of storytelling.
23
Why Storytelling?
• Given a little bit of time, most girls can come up with a story they want to tell.
• Stories are naturally sequential.• Stories provide girls an opportunity to
experiment with different roles, a central activity during adolescence.
• Non-programming friends can readily understand and appreciate an animated story.
24
Developing Storytelling Alice
25
Started with Alice 2
• Alice 2 is a programming environment targeted at college-level intro CS students– Separate learning of programming concepts and
syntax – Enable novice users to create interactive 3D graphics
programs– Formal results show that use of Alice increases
grades and retention of CS students in subsequent Java-based CS courses.
• Alice 2 provides:– drag and drop program construction that
prevents syntax errors– 3D graphics and animation engine
26
Formative Evaluation: Observe middle school girls (and some boys) creating stories in Alice
Design and Implement:Improve Storytelling Alice
Classic HCI Research ParadigmOver 18 months, ~15 iterations
27
Subjects: Formative Evaluation• ~ 60 Campers at
STEM camps• ~ 30 Local home-
schooled kids
• ~ 120 Girl Scouts from within 1.5 hours of Pittsburgh
28
Storytelling Alice Demo
29
Changes to Enable Storytelling
1. Add high-level animations and scene support.
2. Create a story-based tutorial.
3. Provide a gallery of 3D characters that inspire stories.
30
Changes to Enable Storytelling
1. Add high-level animations and scene support.
2. Create a story-based tutorial.
3. Provide a gallery of 3D characters that inspire stories.
31
Lesson Learned: two-person talk aloud protocol is a poor match for a
creative process like storytelling
32
Storyboarding Process
1. Write the DVD box description
2. Flesh out each scene: setting, purpose, and action
3. Create 6-9 frames/textual descriptions for each scene.
33
Rhetorical Information Helps
• 3 sources for actions:– Users’ descriptions of the action in a given
frame– Differences between drawings of sequential
frames– Users’ descriptions of the action in a scene
(occasionally)
34
People are more important than objects
Kids very rarely animate things.
In Storytelling Alice, humanoids and other non-bipedal characters are a different type and have different methods than non-living “things”.
35
What people can do…
Storytelling Alice:• Say, think• Walk to, Walk offscreen• Walk • Sit On• Lie on• Kneel• Fall Down• Stand Up• Straighten• Look at• Look• Turn to face, Turn away from• Turn• Touch• Keep Touching
Generic Alice:• Move• Turn• Roll • Resize• Play Sound• Move to• Move toward, Move away from• Orient to• Turn to Face, point at• Set point of view to• Set Pose• Stand up• Move at speed, turn at speed, roll at
speed• Constrain to face, Constrain to point at
36
Most character and object interactions can be created using
touch and keep touching
37
Character actions often motivate use of programming constructs like
loops, methods and parameters
38
Changes to Enable Storytelling
1. Add high-level animations and scene support.
2. Create a story-based tutorial.
3. Provide a gallery of 3D characters that inspire stories.
39
Before: After:
40
Click on the “trevor” tile to see what methods trevor knows how to do.
41
42
Stencils enables users to complete tutorials more quickly and with fewer errors.
Users of a Stencils tutorial:• Learned as well as users of a paper
version of the same tutorial• Made 270% fewer errors (p < .05)• Completed tutorial 26% faster
(p = .057)
• Made 88% fewer requests for human assistance (p = .08)
43
Changes to Enable Storytelling
1. Add high-level animations and scene support.
2. Create a story-based tutorial.
3. Provide a gallery of 3D characters that inspire stories.
44
Provide custom animations that require explanation in the story
Animations can be incredibly powerful in helping kids to come up with a story idea.
Harold T. Wireton.crazy go nuts
45
Provide characters with clear roles
Hero or Villain Authority Figure
46
Related Work
47
Programming Systems that enable animation
• ToonTalk (Kahn, 1996)
• Squeak Etoys (Kay et al., 1997)
• HANDS (Pane, 2002)
• Scratch (Resnick et al., 2006)
• Kids Programming Language (Morrison Schwartz, Inc., 2006)
48
Present programming in a motivating context
• Algoarena – programming sumo-wrestlers to fight with each other
• Robocode – programming tanks to fight with each other
• Rocky’s Boots – connecting logic gates to solve puzzles
49
Virtual Family
50
Rapunsel
51
There are no formal results showing that any of these
systems are (more) motivating.
52
Evaluating Storytelling Alice
53
Two Questions
• How much do users learn?
• How motivated are they?
54
Two Questions
• How much do users learn?
• How motivated are they?
55
Summative Evaluation
• 88 Girl Scouts from within 1.5 hours of Pittsburgh
• Compare Storytelling Alice with Generic Alice
56
Evaluating Generic Alice vs. Storytelling Alice
Tutorial Build something to show
in Generic Alice
Take programming quiz and attitude survey
Try Storytelling Alice
Choose Generic or Storytelling Alice to take home
Show a world to everyone
Tutorial Build something to show
in Storytelling Alice
Try Generic Alice
2 hrs, 15 min
30 min
Control Group: Experimental Group:
57
Results
58
Similar Learning in Storytelling Alice and Generic Alice
Mean Programming Quiz Scores
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Sequence DoTogether
SimpleLoop
MoreComplex
Loop
MethodCall
Parameter WorldStartsEvent
Mea
n S
core
Generic Alice Storytelling Alice
59
Since it is as educational, which system do girls find
more motivating?
60
Types of Results
• What girls build
• What girls say
• What girls do
61
Three Activities in Alice
1: Scene Layout
2: Editing Programs
3: Running Programs
62
Users of Storytelling Alice spend 42% more time programming
Average % Time Spent on Alice Activities
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
Scene Layout Editing Program Running Program
Generic Alice
Storytelling Alice
p < 0.001
p< 0.001
63
Storytelling Alice motivates the reluctant programmers
Scene Layout vs. Program Editing
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
% Time - Editing Program
% T
ime
- S
ce
ne L
ayo
ut
Generic Alice Storytelling Alice
64
Generic Alice: Arbitrary motion (28/45 = 62%)
A collection of characters that moves in seemingly arbitrary ways.Sometimes whole characters move, sometimes individual body parts.
65
Generic Alice: Character Motion (7/45 = 15%)
The characters put their arms at their sides and the girl on theleft waves hello.
66
Generic Alice: Story-like sequences (7/45 = 15%)
A penguin moves to the lever, the lever turns and the Christmas tree lights up.
67
Generic Alice: Dance routines (3/45 = 7%)
The penguins turn, jump, and look in different directions in sequence and in parallel.
68
Storytelling Alice: Relationship Stories (22/43 = 51%)
A story about a boy named Dave who has been having relationships with three different girls. They find out and kick his legs in retaliation. The story ends with the statement “And that’s why you don’t cheat on girls. It Makes Your Legs Fall Off!”
69
Storytelling Alice: Good vs. Evil (9/43 = 21%)
A story in which the Big Bad Wolf attempts to befriend the three pigs in the hopes of eating them. The pigs get scared and a ninja appears to frighten the wolf away.
70
Storytelling Alice: other (12/43 = 28%)
• 2 stories about finding lost dogs
• 2 stories depicting running or swimming races
• 3 choreographed routines involving cheerleaders or circus characters
71
Types of Results
• What girls build
• What girls say
• What girls do
72
Ease and Entertainment
• No significant differences between Storytelling Alice and Generic Alice users:– Alice’s ease of use– Alice’s entertainment value
73
Users of Storytelling Alice have stronger interest in future Alice use.
Mean Future Alice Use Scores
0
0.5
1
1.5
2
2.5
3
3.5
One time use Use over the next month Interest in a future Aliceclass
Question
Mean
Sco
re
Generic Alice
Storytelling Alice
p < 0.05
74
Users of Storytelling Alice were slightly (but not significantly) more interested in
computer scienceMean Computer Science Interest Scores
-1
-0.5
0
0.5
1
1.5
Create a w orldyou are proud
of
Learnadvanced
Alice features
LearnJava/C++
Interest in HSCS class
Grow up to CS
Question
Mean
Sco
re
Generic Alice
Storytelling Alice
75
Types of Results
• What girls build
• What girls say
• What girls do
76
Alice Version to Take Home
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Chose Storytelling Alice Chose Generic Alice
Alice Version Selected
% o
f U
sers
Generic Alice Storytelling Alice
Users of Storytelling and Generic Alice prefer Storytelling Alice
p < 0.001
77
Switching Versions
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Show non-main world
Alice Program
Pe
rcen
tag
e o
f U
se
rs
Generic Alice Storytelling Alice
Users of Generic Alice are more likely to show the world they build in 30 minutes.
p < 0.001
78
Sneaking Extra Time
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Show non-main world
Alice Program
Per
cen
tag
e o
f U
sers
Generic Alice Storytelling Alice
Users of Storytelling Alice are more likely to sneak extra time to continue programming.
p < 0.001
79
Summary of Results
• Users of Storytelling Alice– Spent 42% more time programming – Were more than three times as likely to sneak
extra time to work on their programs (51% vs 16%)
– Express stronger interest in future Alice use
• Users of both Storytelling and Generic Alice preferred Storytelling Alice
80
Future Work
81
Research goal: Design programming
environments that motivate a larger and more diverse group of people to learn to program.
82
Other Demographics
• Evaluating and potentially adapting Storytelling Alice for other groups– Boys– Minority students
83
The Real World
• Realistically, most schools will introduce programming in a short unit (if at all)
• Organizations like the Girl/Boy Scouts or a local YMCA will tend towards a small number of sessions.
• Storytelling Alice needs to inspire and enable kids to learn new material on their own.
84
Challenge #1:keep users coming back for more
85
Addictive Educational Software
• Choose and support intrinsically motivating activities.• What extrinsic rewards can reinforce users’ interest in
educational content over long periods of time?– Access to new content?– Positive feedback from peers?
• How do we measure motivation over longer periods?
• Effective extrinsic reward schemes and motivational measures can aid the development of serious games.
86
Challenge #2:motivate users to learn more
programming concepts
87
88
Motivating Use of More Programming Concepts
• More expressive character animations (especially facial and social), to motivate methods and parameterized methods
• Controlling behavior of groups of movie extras to introduce conditionals, iteration, and abstraction.
• Could contribute to systems that enable non-technical users to quickly prototype film pre-visualizations and video games
89
Challenge #3: enable users to teach themselves
90
Using Example Code
• Understanding code: what does this do?
• Re-using code: Copy and modify example code for use in your own program
• Interaction techniques for understanding and re-using example code may lead to improved general-purpose programming environments