1

Go Behind the Go Behind the AHRQ/NIH AHRQ/NIH

Study Section Study Section DoorDoor

A Mock ReviewA Mock Review

2

The PanelThe Panel

Linda Greenberg, PhDLinda Greenberg, PhDWillard Manning, PhDWillard Manning, PhDMing Tai-Seale, PhDMing Tai-Seale, PhD

3

The AgendaThe Agenda

Relevant funding mechanisms: Rs, KsRelevant funding mechanisms: Rs, Ks Life of a proposalLife of a proposal Scientific review: who, where, howScientific review: who, where, how Critical areas for improvementCritical areas for improvement Mock reviewMock review Summary statementSummary statement How to work with federal officialsHow to work with federal officials Questions and answersQuestions and answers

4

Funding Funding MechanismsMechanisms

LindaLinda

5

RxxRxx

GRANTS FOR HEALTH GRANTS FOR HEALTH SERVICES DISSERTATION SERVICES DISSERTATION RESEARCH RESEARCH (R36)(R36) Support students seeking a doctorateSupport students seeking a doctorate after successful dissertation defense after successful dissertation defense in areas relevant to health services in areas relevant to health services

research research Total direct costs <$30,000Total direct costs <$30,000

R01, R03, …R01, R03, …

6

Relevant Funding Relevant Funding Mechanisms: K01, K02, Mechanisms: K01, K02,

K08K08, …, … Mentored Clinical Scientist Development Award Mentored Clinical Scientist Development Award

(K08) (K08)

Development of outstanding research scientists.Development of outstanding research scientists. Specialized study support for trained professionals who Specialized study support for trained professionals who

are committed to a career in research and have the are committed to a career in research and have the potential to develop into independent investigatorspotential to develop into independent investigators

Focuses on progression to independenceFocuses on progression to independence Study and development consistent with his/her needs, Study and development consistent with his/her needs,

and previous research or clinical experience. and previous research or clinical experience. The proposed length of the award must be well The proposed length of the award must be well

explained and justified explained and justified Support will only be provided for the period deemed Support will only be provided for the period deemed

necessary to achieve independencenecessary to achieve independence

7

The Life of a The Life of a ProposalProposal

MingMing

8

9

DisciplinesDisciplines Anthropology Anthropology BiostatisticsBiostatistics EconomicsEconomics Epidemiology Epidemiology Health services Health services

researchresearch Medicine Medicine NursingNursing Organizational TheoryOrganizational Theory SociologySociology

MethodologicaMethodological Orientationsl Orientations QuantitativeQuantitative QualitativeQualitative MixedMixed

Stages in Their Stages in Their Own CareersOwn Careers Senior ScholarsSenior Scholars Emergent Emergent

scientistsscientists

Who Serve on Study Sections?

10

11

Where is the Review DoneWhere is the Review Done

NIHNIH AHRQAHRQ

Let’s go there … Let’s go there …

12

The Physical The Physical SettingSetting

13

Protection of Human Protection of Human SubjectsSubjects

Applicant Applicant must must address:address:

1.1. Risks to Risks to human human subjectssubjects

2.2. AdequacyAdequacy

Summary reviewer Summary reviewer choices:choices: Human subjects NOT Human subjects NOT

involvedinvolved Human subjects involved, Human subjects involved,

ACCEPTABLEACCEPTABLE Human subjects involved, Human subjects involved,

UNACCEPTABLEUNACCEPTABLE Human subjects involved, Human subjects involved,

exemption claimedexemption claimed

14

Inclusion of Women and Inclusion of Women and MinoritiesMinorities

Gender CodeGender Code::

First Character = First Character = GG

1= Both Gender1= Both Gender

2= Only Women2= Only Women

3= Only Men3= Only Men

4= Gender Unknown4= Gender Unknown

Third Character:Third Character:A= A= Scientifically acceptableScientifically acceptable

U= U= Scientifically Scientifically unacceptableunacceptable

Minority CodeMinority Code::

First character = First character = MM

Second character:Second character:

1=Minority and 1=Minority and NonminorityNonminority

2= Only Minority2= Only Minority

3= Only Nonminority3= Only Nonminority

4= Minority unknown4= Minority unknown

Third Character:Third Character:A= A= Scientifically acceptableScientifically acceptable

U= U= Scientifically unacceptableScientifically unacceptable

15

Inclusion of AHRQ Priority Inclusion of AHRQ Priority Populations ChecklistPopulations Checklist

IncludeIncludedd

ExcludeExcludedd

Not Not addressedaddressed

ChildrenChildren

ElderlyElderly

RuralRural

Inner cityInner city

Low incomeLow income

DisabledDisabled

Chronically illChronically ill

End of lifeEnd of lifeAdequate numbers for sub-group analysis?For excluded, including rationale?

16

Inclusion of Women and Inclusion of Women and MinoritiesMinorities

Gender CodeGender Code::

First Character = First Character = GG

1= Both Gender1= Both Gender

2= Only Women2= Only Women

3= Only Men3= Only Men

4= Gender Unknown4= Gender Unknown

Third Character:Third Character:A= A= Scientifically acceptableScientifically acceptable

U= U= Scientifically Scientifically unacceptableunacceptable

Minority CodeMinority Code::

First character = First character = MM

Second character:Second character:

1=Minority and 1=Minority and NonminorityNonminority

2= Only Minority2= Only Minority

3= Only Nonminority3= Only Nonminority

4= Minority unknown4= Minority unknown

Third Character:Third Character:A= A= Scientifically acceptableScientifically acceptable

U= U= Scientifically unacceptableScientifically unacceptable

17

Adjectives Used in ReviewAdjectives Used in Review

1.0-1.51.0-1.5 OutstandingOutstanding

1.6-2.01.6-2.0 ExcellentExcellent

2.1-2.52.1-2.5 Very GoodVery Good

2.6-3.52.6-3.5 GoodGood

3.6-5.03.6-5.0 AcceptableAcceptable

18

Priority ScorePriority Score

How is the summary priority score How is the summary priority score calculated?calculated? Group average Group average Equal weightEqual weight

What is the fundable range?What is the fundable range? Study sections can have different normsStudy sections can have different norms When in doubt, ask the project officerWhen in doubt, ask the project officer

19

Critical Areas for Critical Areas for ImprovementImprovement

for for K0xK0x

WillWill

20

Critical Areas for Critical Areas for ImprovementImprovement

for for K0xK0x It Is Not AboutIt Is Not About

5 years of support5 years of support 75 percent buyout75 percent buyout $$$$$$

21

It Is AboutIt Is About

MMentoredentored Clinical ScientistClinical Scientist DDevelopmentevelopment

22

It Is About (cont’d)It Is About (cont’d)

It does require mentoringIt does require mentoring It is about career development It is about career development

for researchersfor researchers Not just about more educationNot just about more education Not just about doing preliminary Not just about doing preliminary

studiesstudies Don’t confuse K with series of Don’t confuse K with series of

R03’sR03’s

23

Disconnected MentorDisconnected Mentor

Mentor’s letter not closely tied to Mentor’s letter not closely tied to content of proposal.content of proposal.

Mentor’s letter written by Mentor’s letter written by proposer and it looks like it. proposer and it looks like it.

Mentor approached with proposal Mentor approached with proposal with only week left before due with only week left before due date. date. Little impact on proposal. Little impact on proposal. Worse if proposal is naïve.Worse if proposal is naïve.

24

Distant MentorDistant Mentor

Always very hard to sell. Always very hard to sell.

Study section distrusts supposed Study section distrusts supposed level of commitment by mentor.level of commitment by mentor.

Plans for linkage, visiting vague.Plans for linkage, visiting vague.

25

Who is in charge?Who is in charge?

Too many mentors Too many mentors

No strong primaryNo strong primary

Nobody with oversight Nobody with oversight responsibilityresponsibility

26

Career Development PlanCareer Development Plan

R-AvoidanceR-Avoidance Its thinly disguised research Its thinly disguised research

support for 5 years.support for 5 years. Little or no education component.Little or no education component.

A La Carte Education:A La Carte Education: Lacks coherent rationale for what’s Lacks coherent rationale for what’s

proposed.proposed. Need to lay out individual strengths Need to lay out individual strengths

and weaknesses.and weaknesses. It’s OK to say you’re imperfect!!!It’s OK to say you’re imperfect!!!

27

Career Development PlanCareer Development Plan

Educational elements too vagueEducational elements too vague Visiting Prof. Jones T times per Visiting Prof. Jones T times per

year. T small.year. T small. Lack of specificityLack of specificity Not clear depth of trainingNot clear depth of training Formal course work preferred Formal course work preferred

if a good match for needs. if a good match for needs. if level appropriateif level appropriate Avoid lower level MPH courses.Avoid lower level MPH courses.

28

Critical Areas for Critical Areas for Improvement in Improvement in RsRs

Design problemDesign problem MeasurementMeasurement Choice of variablesChoice of variables Intervention/comparisonIntervention/comparison

Analysis problemAnalysis problem Choice of approachChoice of approach TechniqueTechnique TestTest

29

Critical Areas for Critical Areas for Improvement in Improvement in RsRs

Weak justification for studyWeak justification for study Background and significance Background and significance

unconvincingunconvincing Literature review incompleteLiterature review incomplete

Investigator expertise deficientInvestigator expertise deficient Needs consultants or collaboratorsNeeds consultants or collaborators Theoretical or conceptual model or Theoretical or conceptual model or

framework framework Missing, deficient, or erroneous Missing, deficient, or erroneous

30

31

32

Mock ReviewMock Review

Chair: Willard Manning, PhDChair: Willard Manning, PhD Primary: Ming Tai-Seale, PhDPrimary: Ming Tai-Seale, PhD Secondary: Willard Manning, Secondary: Willard Manning,

PhDPhD

Usually there is a tertiary Usually there is a tertiary reviewerreviewer

K08 – Mentored Clinical Scientist K08 – Mentored Clinical Scientist Development AwardDevelopment Award

33

Review GuidelineReview Guideline CandidateCandidate Career development planCareer development plan Research planResearch plan Mentor/co-mentorMentor/co-mentor Environment and institutional commitmentEnvironment and institutional commitment BudgetBudget Human subjectsHuman subjects Women/minorities/childrenWomen/minorities/children SUMMARYSUMMARY

major strengths and weaknessesmajor strengths and weaknesses Recommendation for or against fundingRecommendation for or against funding

34

Summary Summary StatementStatement

MingMing

35

How to Read the Pink SheetHow to Read the Pink Sheet

Expect the language to be Expect the language to be Frank, and Frank, and Not overly enthusiastic Not overly enthusiastic

Be emotionally detached, after the Be emotionally detached, after the initial…initial…

Talk to an experienced grant-makerTalk to an experienced grant-maker Resubmit unless you see “Resubmit unless you see “fatally flawedfatally flawed”” Do NOT resubmit right awayDo NOT resubmit right away Recruit a “cold reviewer”Recruit a “cold reviewer”

36

Take a Vacation …Take a Vacation …

37

Role of Federal Role of Federal OfficialsOfficials

LindaLinda

38

What Can You Expect What Can You Expect FromFrom

Project officersProject officers Read your concept paper and draft Read your concept paper and draft

Send it in Send it in EARLY!EARLY! Interpret the fundability of your priority Interpret the fundability of your priority

score score

Scientific review administratorsScientific review administrators Assign reviewers who may have expertise to Assign reviewers who may have expertise to

review your proposal review your proposal

39

Questions & Questions & AnswersAnswers

PanelPanel

40

ResourcesResources

Video on Peer Review for Clinical ResearchVideo on Peer Review for Clinical Research http://www.csr.nih.gov/Video/Video_print.asphttp://www.csr.nih.gov/Video/Video_print.asp

Instructions on how to prepare your applicationInstructions on how to prepare your application http://grants.nih.gov/grants/funding/phs398/section_1.htmlhttp://grants.nih.gov/grants/funding/phs398/section_1.html

Our contact informationOur contact information Linda: Linda: linda.greenberg@ahrq.hhs.govlinda.greenberg@ahrq.hhs.gov Will: Will: w-manning@uchicao.eduw-manning@uchicao.edu Ming: Ming: mtaiseale@srph.tamhsc.edumtaiseale@srph.tamhsc.edu

Can’t comment on any current proposal in the review Can’t comment on any current proposal in the review processprocess