the zigzag of innovation a wiki-like open source programming contest foo camp 2006 ned gulley the...

The Zigzag of InnovationA Wiki-like Open SourceProgramming Contest

Foo Camp 2006Ned GulleyThe MathWorks, Inc.

I started Monday after work. I worked late, but not very late. Let's say 4 or 5 hours. The following days I had to work, but I started to become "obsessed". At home, although I am a father of three children, my full-time job was working on the contest. I didn't sleep much. I worked maybe 10 hours after work each day. On Thursday it was clear that I wasn't going to be able to work seriously (for my job), so I took a day off on Friday.

Stijn says…

Nathan says…

Well, this is my first MATLAB contest and it is giving me far too much enjoyment. It's one of the most addictive and compulsive things I have tried. The exercise is improving my programming skill - true - but in return I have a neckache, a backache, a terrible diet, no social life, and not much of a work life either. Also, I have experienced physical trembling while making the final preparations to submit code. Is that normal?

• How does the contest work?

• Visualize results

What is MATLAB?

A matrix-based language suited for algorithm development

Contest goals

• Entertain

• Foster community

• Encourage the exchange of good programming practices

Contest features

• Entries are automatically scored, ranked, and displayed immediately

• Code, author, and score are visible at all times

• Anyone can modify anyone else's code and resubmit it as their own

How it worksrules

standings

standings

1 tim2 carol3 eric

tim.fcn f(x)...

view entry

new entry

ned.fcn f(x)...1 ned

2 tim3 carol4 eric

Joining the contest: tweaks

...y = [1 2 3 4]...

...y = 1:4;...

Joining the contest: leaps

function f = fb(n)a = [0 1; 1 1]^n;f = a(2);

function f = fb(n)if n > 2 f = fb(n-1) + fb(n-2); else f = 1; end

Generating the nth Fibonacci number: 1, 1, 2, 3, 5, 8, 13, ...

Previously…

• Mastermind

• Protein folding

• Mapping Mars

• Golf

• Gerrymandering

What makes a good problem?

• Low cost of admission

• Plenty of headroom

Blockbuster

What to reward?

• A good answer

• Gotten quickly

Better

Time

Wednesday Monday

BetterBetterAnswer

Faster Result

The zigzag of innovation

1000!

One contest: Sudoku…

3061168290144798

Visualizing Mastermind

Visualizing Mastermind

Overlapping clans

Genetic programming

First use of

conv

First leader

with conv

Innovation at the periphery

Darwinian tweak bombs

Tweak bombs up close

Size and similarity over time

Entry names

Collaboration clusters

Social issues

• hacking

• tweaking

• obfuscation

• acknowledgment

• scoring inequities

• tweak bombs

• clone bombs

Obfuscation

function A = domove(A,mv) N(:) = false; findNeighbors(mv(1),mv(2),A(mv(1),mv(2))); anyN = any(N); for j = 1:size(A,2) if anyN(j) …

Obfuscation

zzz1018=zzz1018+zzz1032;for zzz1069=2:zzz1014if zzz1048(zzz1018),zzz1041(zzz1018)=zzz1018;zzz1053=zzz1018-zzz1032;if zzz1048(zzz1053)==zzz1048(zzz1018)zzz1041(zzz1053)=zzz1018;end…

The contest immune system

Two can play at this game. It's sad that this has sunk to this level. Can't we return to the spirit of the contest?

Alan Chalker says…

…"spirit of collaboration" vs "spirit of competition"?! This is a competition right? Otherwise, anyone up for a round of cum-ba-ya?

Dr. Seuss says…

It is a competition, but with certain rules and goals. One of the goals is collaboration and improving other codes. The darkness and twilight zones are added to give people time to develop their own algorithms without being tempted too much for just tweaking others codes.

Stijn Helsen says…

What have we learned?

Collaborative competition works!

Don’t knock tweaking

Format dissolves contemptuous isolation

And…

The Big Lesson: Focus on the small

• Approachable problem

• Low cost of admission

• Rewards for all levels of effort

• Smooth gradient of participation

As a newcomer I was skeptical about the role of "competitive collaboration". However I'm now convinced that it's what makes the contest work. And anyway the really consistent competitors keep returning to the top of the table and staying there with innovations that are more than tweaks. And the fact that it's so popular must mean the basic formula is right. I think there is a self-regulating aspect to it - nobody comments their code overhelpfully, but nobody has gone out of their way to obscure code either.

Additional info at URL:

http://www.mathworks.com/contest/

I can't tell you how much I enjoy the MATLAB contests. I know it takes time out of your busy schedules to conduct this activity, but I believe it is a great way to promote the product and build user loyalty.

END

Bee Conjuring andSwarm Orchestration

Optimizing Collaborative Innovation

April 12, 2006Ned GulleyThe MathWorks, Inc.

1. Conjuring a crowd

2. Keeping them busy

3. Social issues

4. Future directions

1. Conjuring a crowd

2. Keeping them busy

3. Social issues

4. Future directions

Problem design

Micro-rewards

War stories

Next big thing

Titles as conversation

• Scrambled Eggs• Rotten Eggs• I didn't start the fire• don't get obfuscated ... follow the light• You Call This Collaboration? Give

Me A Break

Titles as conversation

• Tweak Tweak Tweak• tweakfest • tweak-o-rama• slippery ground (tweak me)• Seems like everyone's just tweaking• I said to myself not to do this kind of

things....• Poopdeck Pappy gives in to peer

pressure

Titles as conversation

• I wish I knew how this works

Player types

• Spectators, Lurkers

• Drop-ins, Visitors

• Tweakers, Top-grazers

• Algorithm Artists

• Churners, Integrators

• Warriors, Obfuscators

Player types

1. 2. 3. 4. …

Tweaker (many)

Churner (few)

Drop-in (many)

Features we’ve added

• Leaderboard • Lots of realtime stats • Mini-contests • Mid-contest analysis • Darkness and twilight • Newsgroup discussion • Contest blog • File differencing

Future directions

What’s the goal?

• Fun == Optimizing innovation.

How do we optimize?

• Modulate diffusion

• Modulate testing

• Listen to the contestants!

Putting the bees to work

• Contest to create shipping code

• Rent-a-swarm

• Create your own contest

Turning windmills:The MATLAB File Exchange

Similarity Distance

Contest EvolutionVirus Evolution