WELCOME!

Office of Portfolio Analysis (OPA) / DPCPSI / OD

The Office of Portfolio Analysis (OPA)

MISSION STATEMENT

To enhance the impact of NIH-supported research by enabling

NIH research administrators and decision makers to evaluate

and prioritize current, as well as emerging, areas of research

To advance knowledge and improve human health

Office of Portfolio Analysis (OPA) / DPCPSI / OD

The Office of Portfolio Analysis (OPA)

MISSION STATEMENT

To innovate by identifying and developing new, sophisticated tools that

expand and improve NIH-wide efforts in portfolio analysis

To apply and disseminate both current and newly developed tools,

including computational approaches, which are capable of analyzing

a wide range of parameters of biomedical research funding and the

resulting impact

To promote trans-NIH coordination of portfolio analysis activities and

enhance collaboration among all portfolio analysis stakeholders at NIH

Office of Portfolio Analysis (OPA) / DPCPSI / OD

NIH Non-Competing Commitments

% of Total RO1

Funding

Office of Portfolio Analysis (OPA) / DPCPSI / OD

Portfolio Analysis Workshop February 5-6, 2012

>500 attendees Survey results

Highest priority:

Topics included: o Measuring impact 47%

o Gaps and overlap in NIH portfolios 30%

o Identification of emerging areas 18%

o Categorizing portfolios 5%

What needs should OPA try to address?

o Build better tools / easier to use tools 65%

o Provide training and support 50%

o Develop targeted case studies 15%

Office of Portfolio Analysis (OPA) / DPCPSI / OD

Some OPA Goals and Initiatives

Office of Portfolio Analysis (OPA) / DPCPSI / OD

Train NIH staff in the use of computational methods (Fall 2012)

Create a centralized web repository of computational tools and case studies

o http://dpcpsi.nih.gov/portfolio_analysis/

Consult on portfolio analysis projects

Use outcomes of this Symposium to evaluate and continue to develop

data-driven approaches to decision-making at NIH

Encourage use of portfolio analysis as an input in NIH decision-making

OPA

TOOLS

LAB

More OPA Goals and Initiatives

Identify overlap or gaps in funded research

Identify emerging concepts, approaches, and/or fields of research

Develop ways to measure impact

Unique tool development

o Hierarchical clustering tool (CIT)

o SVM classifier (CIT)

o Semantic portfolio analyst (NLM)

o Endeca data cube (link the NIH grants database to

patent, publication, and citation data)

Office of Portfolio Analysis (OPA) / DPCPSI / OD

A Modeler’s View of NIH

INPUT $$

OUTPUT: pubs / patents etc

HEALTH & ECONOMIC

IMPACT

Office of Portfolio Analysis (OPA) / DPCPSI / OD

A Modeler’s View of NIH

INPUT $$

R01

OUTPUT: pubs / patents etc

HEALTH & ECONOMIC

IMPACT

Office of Portfolio Analysis (OPA) / DPCPSI / OD

Citation Profile for 12 Selected Biomed Journals

Four journals from each 1200

category:

High impact factor >25

Medium IF 13-17

Low IF 1-6

All papers in those 12 journals

in a single year: 2007 Tota

l # of citations

800

400

0 0 1000 2000 3000 4000 5000 6000

All citations of those papers since 2007 Rank order of 2007 papers (least to most cited)

Office of Portfolio Analysis (OPA) / DPCPSI / OD

Averages Apply Only to Gaussian Distributions P

erc

en

tage

Adapted from

MEJ Newman

Contemp. Physics

46:322-351 (2005)

Heights of males Speeds of cars

10,000,0005.00E+06 1075

Po

pu

latio

n (

mill

ion

s)

4.00E+064 Size of cities Size of citiesP

op

ula

tio

n 100,000105

1,000,000106

Power law in California in California

f(x) = Cx-3.00E+063 10,000104

1,000103

22.00E+06

100102 (with a tail) 11.00E+06

10101

100.00E+00 10

0 100 200 300 400 500 1 10 100

Rank order (smallest to largest)

Office of Portfolio Analysis (OPA) / DPCPSI / OD

0

HIGH IMPACT FACTOR JOURNAL: 2007

Power Law Distribution for a High IF Journal

Office of Portfolio Analysis (OPA) / DPCPSI / OD

MEDIUM IMPACT FACTOR JOURNAL: 2007

Power Law Distribution for a Medium IF Journal

Office of Portfolio Analysis (OPA) / DPCPSI / OD

LOW IMPACT FACTOR JOURNAL: 2007

Power Law Distribution for a Low IF Journal

Office of Portfolio Analysis (OPA) / DPCPSI / OD

Summary

Accurate bibliometrics must accommodate the three-log range of

accumulated citations, which follow power law dynamics

Impact factor is an unreliable indicator of the impact of papers in

that journal on the field

A more important indicator is specific to each paper: how often

has it been cited?

Citations accumulate rapidly enough to potentially influence

funding decisions

An important goal is to construct a “next gen” database,

comprehensively and accurately linking NIH awards with

associated patents, publications, and citations

Office of Portfolio Analysis (OPA) / DPCPSI / OD

Acknowledgments

PILOT STUDY

Geetha Senthil, Ph.D.

Paula Fearon, Ph.D.

Jean Yuan, Ph.D.

SYMPOSIUM

Trans-NIH Symposium Planning Committee

Carole Christian, Ph.D.

Office of Portfolio Analysis (OPA) / DPCPSI / OD