Strengthening the

Effectiveness and Impact of

Collaboration Through Metrics

Mark D. Lim, PhD

Forum on Drug Discovery, Development, and TranslationInstitute of Medicine

October 7, 2014

Shared scientific challenge

Widely-usable tool

Virtual team to create / qualify

research-by-consortium

Sci. Trans. Medicine, June 2014

http://bit.ly/STMConsortia

Operational Framework Landscape

consortiapedia.fastercures.org

Database

Planned release:

end of 2014

Metrics for consortia?

We need to value collective successes of a consortium…

and get rewarded for that.Eric Green, NHGRI Director

Whatever can be done in qualifying how consortium work, how

quality is being measured and recognized, would help big

companies like ours to work in these consortium.Marc Bonnefoi, Sanofi

FasterCures Hill Briefing, July 2014

Objective measurements – performance and outcome

Apples and apples - normalized across consortia

Suite of metrics by project type? Application (biomarker development, clinical trial methods, etc)

Therapeutic area

Stage of innovation

Others?

Considerations:

Since we last met….

Conversations

• Consortium leaders and participants

• Pharmaceutical companies

• FDA

• Collaboration management experts

Presentations

• Science of Team Science

• Pharmaceutical campus

Other

• Hill Briefing on consortia

An evolving conversation…

qualitative

metrics…asking

participants what the

vision/mission is…

what are indicators

for future success?

is this consortium

doing the best it can?

what are common

challenges that all

consortia are facing?

Where we want to go…

Reality check on value of framework

General consensus on:

Design

Implementation

Optimization

Who are implementation partners?

What is the output? What does success look like?

Evidence to inform best practices?

Metrics of collaboration?

Retrospective Prospective

• Return on investment

• Impact on organization

• Impact on society

• Improved efficiency

• Focus vs flexibility

Output and outcomes

Sponsors/Participants

• What did we get from the collaboration?

• Would we participate in a similar collaboration again?

Managers/Organizers

• Were there lessons learned?

• How do we design/execute a better collaboration?

Value of outcomes: who is asking?

Government • public health

• regulatory science

• de-risk innovation

• economic growth

• state-of-science research guidance documents

Industry • accelerate pipeline

• new therapeutic area

• access resources

• de-risk innovation

• access intellectual

capital

Academia • access resources

• opportunities for

publications

• training opportunities

• identify collaborators

Patient

organizations

• accelerate pipelines

• advance basic

research

• de-risk medical

product development

Consortium

researchers

• simplify day jobs

• access resources

• networking

• training / education

Output to outcome metrics

* Adapted from Critical Path Institute and Innovative Medicines Initiative

Non-competitive biomarkers

Clinical trial designs and processes

Data standards

Best practices / procedures

Tools (PRO, prediction, data-sharing)

Output

Implementation / adoption

IP – filed / awarded / licensed

Policy change / regulatory science

Publications

Job / company creation

Impact

De-risked innovation

Decreased R&D cost/time

Meaningful data-sharing

Improved clinical trials

Improved human health

Outcome

Metrics of collaboration?

Retrospective Prospective

• Return on investment

• Impact on organization

• Impact on society

• Improved efficiency

• Focus vs flexibility

Processes

Sponsors/Participants

• Is the collaboration on-track and focused?

• Are resources available/leveraged?

Managers/Organizers

• How can we do better?

• Are we continuing to address an unmet need?

Processes & Team Performance

Wooten, Univ. Houston

Tuckman & Jensen, Group and Organizational Studies (1977)

Burke et al., J. Applied Psychology (2006)

Hall et al, Trans Behavioral Med (2012)

Identifying shared challenge

Consensus solution

Framework for collaboration

Defining success

Convene

Defining accomplishments

Integrating deliverables

Communicating

Tracking adoption/impact

Transition

Execute

Participant/sponsor buy-in

Maintaining vision/focus

Work stream monitoring

Conflict resolution

Discussion points

? Gut reaction?

Structuring discussion to develop framework?

• How? (retrospective/prospective)

• What? (end-user)

• Who? (stakeholders)

• Where? (host)

Next-steps

• Follow-up with consortia, industry, FDA

• Partnering For Cures – Nov 16-18 (NYC)

Seeking insights & expertise

consortiapedia.fastercures.org

mlim@fastercures.org