R&D Evaluation at NIAID:Making the Most of Expert Opinion

Kevin WrightAEA Annual Conference

Nov 4, 2011

OverviewNIAID at a Glance– Highlights– Organization

R&D Evaluation at NIAID– Intent– Implementation– Improvements

Looking Ahead– Opportunities– Threats

NIAID at a Glance

3

NIAID R&D Evaluations

Level 3

Level 2

Level 1 NIAID OD

Division 1

Program A

Solicited Unsolicited

Program B

Division 2

Program C Program D

Initial idea

Program DefinitionReviewResearch

RFA/PA written, grants

submitted

Peer review by

study section, Council

Research conducte

d

Scientific results

Division retreat/ program planning

Concept review by

RIC, Council

NIAID Program Cycle

Primary Conclusion of the Interagency Working Group:

“Expert judgment” remains the best available decision support tool for science policy makers, but a nascent community of practice is emerging in the science policy arena that holdsenormous potential to provide rigorous and quantitative decision support tools in thenear future. – The Science of Science Policy: A Federal Research Roadmap, Nov 2008.

Mission

CoreValues

To enhance the quality, utility, transparency and application of evaluation in support of decision making throughout NIAID.

Quality – The Evaluation Section will strive to meet and exceed expectations

Utility – All services and products will be developed and applied where appropriate

Collaborative – The Evaluation Section will listen to and partner with its customers

Innovative – The Evaluation Section will continually seek to enhance products and services

NIAID R&D Evaluations

Evaluation Planning TemplateProgram

TypeProgram

ObjectivesIndicator(s) Data Source Collection

ScheduleBasic Research

Fund research to fill gap

Projects and Pubs

IMPAC II and Pubmed Quarterly

Applied Research

Identify promising candidates

Press Release, Pubs, Pats

Web, Pubmed, USPTO

Annually

Clinical Research

Evaluate candidates

Protocols, Licenses, NDA

ClinicalTrials.gov, Web, FDA Annually

Physical Capacity

Reduce barriers Indirect costs IMPAC II Annually

Intellectual Capacity

Maintain pipeline of investigators

NI/ESI IMPAC II Annually

9

Analytic Tools

Opportunities

• Improving Tools and Resources• Community of Knowledge• Alignment of Research Priorities• Increased Scrutiny of Results

Threats

• Few Success Stories• Inconsistency– Rigor– Use by Decision Makers

• Culture– Management v. Research– Results v. Process

Contact Info

Kevin D. Wright, MPADeputy ChiefStrategic Planning and Evaluation BranchOSPFM, OSMO, NIAID, NIH, DHHS(p) 301-402-3574wrightk@mail.nih.gov