Software Sustainability Institute

www.software.ac.uk

Why developing research software is like a startup(and why this matters)

Neil Chue Hong, N.ChueHong@software.ac.ukISGC 2015, Taipei, 19th March 2015

Institute

SoftwareSustainability

www.software.ac.uk

Supported by Project funding from

Where indicatedslides licensed under

Software Sustainability Institute

www.software.ac.ukThe problem with mushrooms…

Software Sustainability Institute

www.software.ac.uk

The Software Sustainability Institute

A national facility for cultivating better, more sustainable, research software to enable world-class research• Software reaches boundaries in its

development cycle that prevent improvement, growth and adoption

• Providing the expertise and services needed to negotiate to the next stage

• Developing the policy and tools tosupport the community developing andusing research software

Supported by EPSRC Grant EP/H043160/1

Communication

Website & blog

Campaigns

Advice

Guides

Courses

Workshops

Fellowship

Research

Software

Policy

Training

Community

Consultancy41 projects

92 evaluations

4 surgeries

33 UK SWC

workshops

1000+ learners

50,000 readers

41 domain

ambassadors

20+ workshops organised

740 researchers

50,000 grants

analysed

150+ contributed articles

19,000 unique visitors per month

272 RSEs engaged 1700 signatures 13 issues highlighted

Software Sustainability Institute

www.software.ac.uk

Software isn’t special, it’s mainstream

69%92%

Survey of researchers from 15 Russell Group unis conducted by SSI between Aug- Oct 2014.

406 respondents covering representative range of funders, discipline and seniority.

http://www.software.ac.uk/blog/2014-12-04-its-impossible-conduct-research-without-software-say-7-out-10-uk-

researchers

Software Sustainability Institute

www.software.ac.uk

And everyone’s a developer

Survey of researchers from 15 Russell Group unis conducted by SSI between Aug- Oct 2014.

406 respondents covering representative range of funders, discipline and seniority.

http://www.software.ac.uk/blog/2014-12-04-its-impossible-conduct-research-without-software-say-7-out-10-uk-

researchers

56%

Software Sustainability Institute

www.software.ac.ukCan we learn?

Picture by Heisenberg Media Picture by PNNLPicture by nengard

Software Sustainability Institute

www.software.ac.uk

Startup Survival Rules

1. Pick good co-founders

2. Launch fast

3. Let your idea evolve

4. Understand your users

5. Better to make a few users love you than a lot ambivalent

6. Offer surprisingly good

customer service

7. You make what you measure

8. Spend little

9. Get ramen profitable

10.Avoid distractions

11.Don’t get demoralised

12.Don’t give up

13.Deals fall through

http://www.paulgraham.com/13sentences.html

Software Sustainability Institute

www.software.ac.uk

Software Survival Rules

1. Pick good collaborators

2. Release early

3. Let your idea evolve

4. Understand your users

5. Better to make a few users love you than a lot ambivalent

6. Offer surprisingly good

support

7. You make what you measure

8. Spend little

9. Get paper profitable

10.Avoid distractions

11.Don’t get demoralised

12.Don’t give up

13.Funding often falls through

Software Sustainability Institute

www.software.ac.uk

Understand your users

Wealth created / Impact enabled

Nu

mb

er

of

use

rs

How much you improve their lives

“As in science, the hard part is not answering questions but asking them: the hard part is seeing something new that users lack. The better you understand them the better the odds of doing that. That's why so many successful startups make something the founders needed.” Paul Graham, Y-Combinator

Where can you make most difference?

Software Sustainability Institute

www.software.ac.ukStartup companies are all about reaching sustainability by understanding users

Software Sustainability Institute

www.software.ac.uk

Four stages of startups

Discovery

Validation

Efficiency

Scale

Are you solving a problem that others are interested in?

Have you implemented core features that users want?

Can you support new users by refining your processes?

Ready to drive growth.Back-end scalability refactoring

http://blog.startupcompass.co/pages/marmer-stages

Software Sustainability Institute

www.software.ac.uk

Four stages of startups

Discovery

Validation

Efficiency

Scale

Are you solving a problem that others are interested in?

Have you implemented core features that users want?

Can you support new users by refining your processes?

Ready to drive growth.Back-end scalability refactoring

http://blog.startupcompass.co/pages/marmer-stages

Software Sustainability Institute

www.software.ac.uk

Stage 1: Discovery

• Are you solving a problem that others are interested in?

Many pieces of software created by researchers have small user bases – this is particularly true of scripts

Not a problem if you are writing the software for yourself only – but it affects how large the project to support the software can be

• Software development as research and prototyping – is the science interesting?

Software Sustainability Institute

www.software.ac.uk

Case Study: Ligand Binding

• Centre for Computational Chemistry, Bristol New methods for rapid MC sampling of

biomolecular systems modelled using QM/MM Developed two codes ProtoMS (F77) + Sire (C++) Water-Swap Reaction Coordinate method to

calculate absolute protein-ligand binding free energies

• SSI’s work helped assess users + scale devs Ran user observations with 4 different users ASPIRE/ACQUIRE framework has multiple devs

• Split architecture between ASPIRE (adaptive multiresolution hybrid MD simulation) and ACQUIRE (WorkPacket scheduling system with optimisationfor time to result vs “green-ness”

• http://www.software.ac.uk/resources/case-studies/getting-grips-molecules

• http://www.siremol.org/adaptive_dynamics

Software Sustainability Institute

www.software.ac.uk

Four stages of startups

Discovery

Validation

Efficiency

Scale

Are you solving a problem that others are interested in?

Have you implemented core features that users want?

Can you support new users by refining your processes?

Ready to drive growth.Back-end scalability refactoring

http://blog.startupcompass.co/pages/marmer-stages

Software Sustainability Institute

www.software.ac.uk

Stage 2: Validation

• Have you implemented core features that users want? Do you know who’s using your software? Why are they

using it? If you asked them “how would you feel if you can no

longer use this software”, how many would be disappointed?

These are your core users: what do they want and how can you give it to them?

• At this point, research software projects often start giving demonstrations, presentations, workshops

Software Sustainability Institute

www.software.ac.uk

Case Study: Climate Policy Modelling

• CIAS team at Tyndall Centre for Climate Change Research, University of East Anglia Develop linked climate and economic models for

detailed analysis Their software was not ready to be used by other

groups• One researcher/developer at UEA, several users

• SSI’s work means the software is robust enough that it can be installed and used by others Enabled use of the software by the WWFN’s

Climascope project and James Cook University• Documented software to allow extensions by contributors• Made it easier to maintain and backup• Added job scheduling to improve modeling throughput• New modelling framework enables new models i.e. new

science

• http://www.tyndall.ac.uk/research/cias

Software Sustainability Institute

www.software.ac.uk

Four stages of startups

Discovery

Validation

Efficiency

Scale

Are you solving a problem that others are interested in?

Have you implemented core features that users want?

Can you support new users by refining your processes?

Ready to drive growth.Back-end scalability refactoring

http://blog.startupcompass.co/pages/marmer-stages

Software Sustainability Institute

www.software.ac.uk

Stage 3: Efficiency

• Can you support new users by refining your processes?

If you had conflicting requirements from users, how would you deal with them?

What infrastructure changes do you need to make to support new/additional users?

• At this point, research software projects often start designating specific community/product managers, user support staff

Software Sustainability Institute

www.software.ac.uk

Case Study: ICAT

• Science and Technology Facilities Council Metadata catalogue, used by RAL UK (ISIS,

DIAMOND, CLF), SNS US, ELLETRA Italy

ICAT operationally critical at sites

Huge projects looking to use ICAT (PaNdataODI, EuDAT)

Scalability issues and lack of proper processes

• SSI’s work provided 33 recommendations 15 interviews with different stakeholders

92 observations set out in report

“…we must focus on doing the right things, and this report will help us”

• Alistair Mills, STFC

Governance and outreach changes to support additional users

• http://www.software.ac.uk/preparing-icat-thousands-new-users

• http://www.icatproject.org/

Software Sustainability Institute

www.software.ac.uk

Four stages of startups

Discovery

Validation

Efficiency

Scale

Are you solving a problem that others are interested in?

Have you implemented core features that users want?

Can you support new users by refining your processes?

Ready to drive growth.Back-end scalability refactoring

http://blog.startupcompass.co/pages/marmer-stages

Software Sustainability Institute

www.software.ac.uk

Stage 4: Scale

• Are you ready to drive growth of users, to reengineer and refactor on an ongoing basis?

This is when software quality considerations become very important, as you have increased reputational risk

This is also the point where traditionally a PI would step aside to become Chief Technology Officer / Chief Scientist and enlist new management

• After this, next stage is sustain (then conserve)

Though this might be at a small scale if appropriate

Software Sustainability Institute

www.software.ac.uk

Research software at scale

Some of the largest and most popular research software started out as code developed in an academic context designed to address a specific problem

Software Sustainability Institute

www.software.ac.uk

Open Source Software Projects

• “Every good work of software starts byscratching a developer's personal itch.” Eric Raymond, The Cathedral and the Bazaar

• Producers start by having a direct interest in the success of the software Just like in science

• OSS projects need to satisfy two aims: Acquire users (a.k.a. researchers) Acquire contributors (a.k.a. collaborating researchers)

• Producing Open Source Software: How to Run a Successful Free Software Project by Karl Fogel

Software Sustainability Institute

www.software.ac.uk

Why do you need users?

• Funding Direct: fees, subscriptions, … Indirect funding: letters of support,

citations, collaborations Advertising: recommendations and

referrals

• Direction (indirection?) Requirements, bug reports, change

requests

• Community Users supporting other users Users becoming contributors Sustainability and success

Software Sustainability Institute

www.software.ac.uk

Idea Prototype Supported At scale

Seed Angel VC IPO

Self-supported

ResearchGrant

Platform Grant

?

Funding Modes

For-profitNot-for-profit / foundationLicensingServices

Software Sustainability Institute

www.software.ac.uk

The entrepreneur vsthe researcher?

• Entrepreneurs have ideas, and don’t mind if some of them aren’t successful “A person who never made a mistake never tried

anything new” “I have not failed. I’ve just found 10,000 ways that

won’t work”

• Researchers find it difficult to get away from the questions they choose to focus on

• Yet successful researchers are able to switch from one area to another, because it’s interesting to them

Software Sustainability Institute

www.software.ac.uk

Take home messages1. Scratch your own itch2. Understand your users3. Embrace your contributors4. Don’t give up before you start

Software Sustainability Institute

www.software.ac.uk

Find out more about the SSI

• Community Engagement (Lead: Shoaib Sufi) Fellowship Programme Events and Workshops

• Consultancy (Lead: Steve Crouch) Open Call for Projects / Collaborations Software Evaluation

• Policy and Publicity (Lead: Simon Hettrick) Case Studies / Policy Campaigns Software and Research Blog

• Training (Lead: Aleksandra Pawlik) Software Carpentry (300+ students/year) Guides and Top Tips

• Journal of Open Research Software (Editor: Neil Chue Hong)

• Collaboration between universities of Edinburgh, Manchester, Oxford and Southampton