TopCoder Inc. Developing Software through Crowdsourcing

Group 4Nikhil Goel

Pradeep KhanPrasad Deshpande

Rachit Jain Radhalaxmi R. Iyer

TopCoder Inc. - Inception

• Founded in 2001 by Jack Hughes• Previously owned a traditional software firm

– Tallan• Had been frustrated with the following

business problems:› Time-consuming and expensive process of

talent recruitment› Talent retention – high turnover rate› Development from scratch – unable to

reuse existing components

TopCoder Inc. – company profile

• Runs by providing platform between two sides:› First side – clients. Interfaces with clients to understand their

business needs and generate programming challenges based on it.

› Second side – developers’ community. Arrange contests for the community to create solutions to the challenges for money and skill ratings.

• Arranges two types of competition :› Algorithmic competitions for generating solutions for generic

difficult problems.› Competitions for developing software solutions for specific

client needs.• Evaluate the solutions of the community through bug-testing

and community peer-review• Award money prize and skill ratings to the best solution provider.• Identify reusable components from the software developed for

the clients and store it in a catalog.• Also acts as a source for providing best talents of their

community to their clients.

What’s in it for the customer ?

How about getting it

from a developer

community?

Throw the challenge at them

Arrange

competition for

the best codeGet back high quality

cheap software components

Idea !!

Crowdsourcing, the new software development methodology

• Outsource a challenging software module/component to a large group of people - a community or a crowd.

• Generate interests by giving incentives like money, prizes, skill ratings etc. for the best performance.

• Access the quality of work through various standard procedures like bug tests, peer review etc. and generate scorecards.

• Use the best graded software module/component for integration with other components.

Evolution of Top Coder Community• Growth & Composition

– Growth• >25,000 programmers added each year (2001-2009) • 3 types of coders normal, latent pool(82.5%) & long tail

– Composition• Core of TopCoder’s community was Single Competitive males in there 20’s.• Had competitors world wide. Some coders were full time.

– Profiles & Ratings• Profile

– Programmers maintained a public profile with all demographic data and top coder ratings• Rating

– Ratings based on work done . Red , Yellow, Green color ratings were there .– Success rate of contests & reliability was featured in programmers profile

– Motivating Members• Through regular competitions , earning extra money• Top coders ratings as one important criteria for prospective employers• Work Flexibility & very high end learning

– Community Thinking• Collaboration among top coder community members to help each other grow.• Community help in time of personal exigency

• Low cost & high quality software – key advantage !

• Focus on generation of reusable components.

• Quality of Work – significant motivation.• Generate more algorithmic challenges for

the community to attract talent.

Managing Top Coder• Supply Side

– Unique Management Style• Management through focus on community participation & community growth• Managing clients & top coders programmers simultaneously

– Attracting Community Members• Algorithms Challenges • Advertising on Google for “design contests”• University student participation through “member development days”• Fairness & Quality of competitions

– Community Norms• Enforcement of Integrity , fairness ,transparency & quality through well defined rules & procedures• Elimination of community members in case of cheating • Storing of all competition data to ensure transparency

– Governance• Governance through community thinking • Top Coder managers had rights to reduce, increase money for competition &

reviews– Resource Allocation

• Dynamic resource allocation based on competitions– Retention

• Consistent work streams & prize money• Increasing participation • Work Flexibility

• Low cost & high quality software – key advantage !

• Focus on generation of reusable components.

• Quality of Work – significant motivation.• Generate more algorithmic challenges for

the community to attract talent.

Managing Top Coder• Demand Side

– Platform Managers• Project Planning , Execution & Client Management

– Top Coder Direct• Clients managing there won project development • Concept of “co-pilot” to assist the client

Client Benefits

Better Realization of

Ideas• Concept testing,• Insight into

requirements.

Access to creative

ideas and solutions

Superior Quality

• Delivery of bug free code.

• Rigorous evaluation.

• Documentation process above industry standards.

Decreased Costs.

• Compare the below costs• $350,000 for

large IT firm.• $200,000 for

small IT firm.• $80,000 for

individuals• $35,000 for

TopCoder

Speedy response/ delivery times

• 72 hrs Vs 10 business days

Flexibility• Expand/

Reduce business requirements and development capabilities

Client Concerns

Intellectual Property & Security

• Increased risk to business plans/proprietary ideas to be leaked to competition.

• Community member asking for royalty.

• Solution is stolen/copyrighted.

Cultural Change• CIO’s dream,

programmer’s nightmare

• Loss of control over dev process.

• Members pushy,rude.

Coding Challenges• Co. still required staff

for integration.• Ensure no

bugs/threats• Spend time to

evaluate ideas/approaches

Problem Identification.

• Find the right type of problem.

• Amount of details to be revealed.

• Length of contest(< 2 weeks)

Challenges !

• Competitive Landscape of TopCoder.– Not much competition as of Dec 2009.– uTest – only involved in finding bugs and

check functionality of applications.

Entry barriers for large IT firms ?

Will community members remain with TopCoder ?

Challenges !

• Growth of Business model– Sustainability of model.– Future Growth.

More contests will lead to decreased

quality

Reviewers wanting to compete in contests

– replacements?

Attracting competitors with

increased contests!

Will increased clientele change the

business model ?

What should we do ? - Jack Hughes

Our Thoughts !

• Unique Model• Growth is there !• Low cost & high quality software – key

advantage !• Focus on generation of reusable

components.• Quality of Work – significant motivation.• Generate more algorithmic challenges for

the community to attract talent.