Gamifying Teaching and Learning of Software Engineering and Programming

Tao Xie

University of Illinois at Urbana-Champaign

In collaboration with Nikolai Tillmann, Jonathan de Halleux, and Judith Bishop

(Microsoft Research)

http://pex4fun.com

https://www.codehunt.com

Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese

Working for fun

Enjoyment adds to long term retention on a taskDiscovery is a powerful driver, contrasting with direct instructionsGaming joins these two, and is hugely popularCan we add these elements to coding?

Write a program to determine all the sets of effectively identical rooms in a maze. (A page of background, sample input and output given)

Testing Tool Educational Gaming

Dynamic Symbolic Execution (Pex)

Pex for Fun:Interactive Gaming for Teaching and Learning

Support

http://research.microsoft.com/pex/ http://pex4fun.comhttps://www.codehunt.com

Agenda

Background

Coding Duels

Educational Platform

Experiences

Conclusion

Agenda

Background

Coding Duels

Educational Platform

Experiences

Conclusion

http://research.microsoft.com/pex/

Dynamic Symbolic Execution (DSE)aka. Concolic Testing

[Godefroid et al. 05][Sen et al. 05][Tillmann et al. 08]

Instrument code to explore feasible paths

Background

http://research.microsoft.com/pex/

void CoverMe(int[] a){

if (a == null) return;if (a.Length > 0)if (a[0] == 1234567890)

throw new Exception("bug");}

a.Length>0

a[0]==123…

TF

T

F

Fa==null

T

Constraints to solve

a!=null

a!=null &&

a.Length>0

a!=null &&

a.Length>0 &&

a[0]==123456890

Input

null

{}

{0}

{123…}

Execute&MonitorSolve

Choose next path

Observed constraints

a==null

a!=null &&

!(a.Length>0)

a==null &&

a.Length>0 &&

a[0]!=1234567890

a==null &&

a.Length>0 &&

a[0]==1234567890

Done: There is no path left.

Dynamic Symbolic Execution in Pex

http://pex4fun.com/HowDoesPexWork

Pex is Part of Visual Studio 2015!

• As new feature of “Smart Unit Tests”

http://www.visualstudio.com/en-us/news/vs2015-preview-vs#Testing

Coding Duels

1,594,092 clicked 'Ask Pex!'

Coding Duels

Pex computes “semantic diff” in cloud

secret reference implementation vs.

code written in browser

You win when Pex finds no differencesFor more info, see our ICSE 2013 SEE paper: http://web.engr.illinois.edu/~taoxie/publications/icse13see-pex4fun.pdf

Behind the Scene of Pex for Fun

Secret Implementation

class Secret {public static int Puzzle(int x) {

if (x <= 0) return 1;return x * Puzzle(x-1);

}}

Player Implementation

class Player {public static int Puzzle(int x) {

return x;}

}

class Test {public static void Driver(int x) {

if (Secret.Puzzle(x) != Player.Puzzle(x))throw new Exception(“Mismatch”);

}}

behaviorSecret Impl == Player Impl

12

1,594,0921,594,092

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Code Hunt Programming Game

Leaderboard and Dashboard

Publically visible, updated during the contest

Visible only to the organizer

It’s a game!

iterative gameplayadaptivepersonalizedno cheatingclear winning criterion

code

test cases

Audiences

Students: proceed through a sequence on puzzles to learn and practice.

Educators: exercise different parts of a curriculum, and track students’ progress Recruiters: use contests to inspire communities and results to guide hiring

Researchers: mine extensive data in Azure to evaluate how people code and learn

Social Experience

Community

High score lists, leaderboard

Live feed

http://pex4fun.com/Community.aspxhttp://pex4fun.com/Livefeed.aspx

Agenda

Background

Coding Duels

Educational Platform

Experiences

Conclusion

Teaching and Learning

Skills Being Trained

Induction

Problem solving/debugging

Program understanding/programming

Testing

Specification writing

Coding Duels for Course Assignments@Grad Software Engineering Course

http://pexforfun.com/gradsofteng

Observed Benefits• Automatic Grading• Real-time Feedback (for Both Students and Teachers)• Fun Learning Experiences

Coding Duels for Training Testingpublic static string Puzzle(int[] elems, int capacity, int elem) {

if ((maxsize <= 0) || (elems == null) || (elems.Length > (capacity + 1))) return "Assumption Violation!";

Stack s= new Stack(capacity);for (int i = 0; i < elems.Length; i++)

s.Push(elems[i]);int origSize = s.GetNumOfElements();//Please fill in below test scenario on the s stack

//The lines below include assertions to assert the program behaviorPexAssert.IsTrue(s.GetNumOfElements() == origSize + 1);PexAssert.IsTrue(s.Top() == elem); PexAssert.IsTrue(!s.IsEmpty());PexAssert.IsTrue(s.IsMember(elem));return s.GetNumOfElements().ToString() + "; “ + s.Top().ToString() + "; “

+ s.IsMember(elem).ToString() + "; " + s.IsEmpty(); }

Set up a stack with some elements

Cache values used in assertions

Coding Duel Competition

@ICSE 2011

http://pexforfun.com/icse2011

Example User Feedback

“It really got me *excited*. The part that got me most is about spreading interest in teaching CS: I do think that it’s REALLY great for teaching | learning!”

“I used to love the first person shooters and the satisfaction of blowing away a whole team of Noobies playing Rainbow Six, but this is far more fun.”

“I’m afraid I’ll have to constrain myself to spend just an hour or so a day on this really exciting stuff, as I’m really stuffed with work.”

Released since 2010

X

Recap: Code Hunt: A Game for Coding

For individuals (K12, introductory courses, geeks)

For competitions at any level, world-wide or in house

Based on long-term research on symbolic program analysis (Pex, Z3)

Works with Java and C#

Runs in any modern browser

Now working on tablets and phoneswww.codehunt.com

Summary: Testing Tool Educational Gaming

DSE/PexPex for Fun:

Interactive Gaming for

Teaching and Learning

Support

http://pex4fun.comhttps://www.codehunt.com

Testing Tool Educational Gaming

Support

http://research.microsoft.com/pex/ http://pex4fun.com

Related Papers/Resources: https://sites.google.com/site/asergrp/projects/ese

Q & A

Thank you!

Contact: taoxie@illinois.edu http://www.cs.illinois.edu/homes/taoxie/

Summary: Testing Tool Educational Gaming

DSE/PexPex for Fun:

Interactive Gaming for

Teaching and Learning

Support

http://pex4fun.comhttps://www.codehunt.com