ahrq update francis d. chesley, jr., m.d. director, office of extramural research, education, and...
TRANSCRIPT
AHRQ UpdateAHRQ Update
Francis D. Chesley, Jr., M.D.Francis D. Chesley, Jr., M.D.Director, Office of Extramural Research, Director, Office of Extramural Research,
Education, and Priority PopulationsEducation, and Priority PopulationsJune 6, 2004June 6, 2004
OverviewOverview
““News You Can Use”News You Can Use” What’s New at AHRQWhat’s New at AHRQ TRIPP Now!TRIPP Now! Qs and AsQs and As
FY 2004 BudgetFY 2004 Budget
FY 2004 Budget = FY 2004 Budget = $304,000,000$304,000,000– Translating Research into Practice and PolicyTranslating Research into Practice and Policy– Prevention ResearchPrevention Research– Health Information TechnologyHealth Information Technology– Quality and Patient SafetyQuality and Patient Safety– Bioterrorism PreparednessBioterrorism Preparedness
FY 1995 - FY 2005FY 1995 - FY 2005Appropriation HistoryAppropriation History
Dollar in MillionsDollar in Millions
$204
$125$144 $147
$171
$204
$270 $299$309 $304 $304
$120
$170
$220
$270
$320
Fiscal Years
February 26, 2003J:/fms/FY2000-2004apphist.ppt
FY 2004 Patient SafetyFY 2004 Patient Safety
FY 2004 Patient Safety = $79.5 MillionFY 2004 Patient Safety = $79.5 Million– $50 M Patient Safety Health Care Information $50 M Patient Safety Health Care Information
Technology (IT) Initiative (Grants and Contracts)Technology (IT) Initiative (Grants and Contracts) $26 M for small and rural hospitals$26 M for small and rural hospitals Facilitate uptake of IT technologiesFacilitate uptake of IT technologies
– $10 M promoting and accelerating the $10 M promoting and accelerating the development, adoption, and diffusion of IT in development, adoption, and diffusion of IT in health care (All Contracts)health care (All Contracts)
– $2 M Patient Safety Improvement Corps (IAA)$2 M Patient Safety Improvement Corps (IAA)
– $17.5 M Patient Safety Commitments Contracts)$17.5 M Patient Safety Commitments Contracts)
FY04 HIT InvestmentFY04 HIT Investment
$62 million initiative:$62 million initiative:
– $26 million: earmarked for implementing proven $26 million: earmarked for implementing proven technologies in small and rural communities technologies in small and rural communities (where HIT penetration has been low)(where HIT penetration has been low)
– $24 million: targeted for developing, $24 million: targeted for developing, implementing, and evaluating the use of new and implementing, and evaluating the use of new and innovative technologies to improve patient safety innovative technologies to improve patient safety and quality of care in diverse health care settings. and quality of care in diverse health care settings.
– $12M: targeted for clinical data standards and $12M: targeted for clinical data standards and interoperabilityinteroperability
FY 2004 Non-Patient SafetyFY 2004 Non-Patient Safety Grants: Grants:
– $5 million in new funds to renew existing grant $5 million in new funds to renew existing grant programs including small, conference, programs including small, conference, dissertation, career, M-RISP, and BRIC awardsdissertation, career, M-RISP, and BRIC awards
Contracts:Contracts:– Overall decrease of $4 million. A total of $5 Overall decrease of $4 million. A total of $5
million is provided to support data collection million is provided to support data collection and dissemination efforts – based on the and dissemination efforts – based on the reviews by OMBreviews by OMB HCUP - $2 millionHCUP - $2 million CAHPS - $1 millionCAHPS - $1 million MEPS - $2 millionMEPS - $2 million
FY 2005 RequestFY 2005 Request
FY 2005 Request = FY 2005 Request = $303,695,000$303,695,000
– Maintains the FY 2004 Enacted LevelMaintains the FY 2004 Enacted Level
– Although there is no increase in funds, a Although there is no increase in funds, a number of grants and contracts end in FY number of grants and contracts end in FY 2004 allowing us to reinvest these funds in 2004 allowing us to reinvest these funds in new grants and contracts in FY 2005new grants and contracts in FY 2005
FY 2005 Patient SafetyFY 2005 Patient Safety
FY 2005 Patient Safety = $84 MillionFY 2005 Patient Safety = $84 Million
– An increase of $4.5 M over FY 2004An increase of $4.5 M over FY 2004
– Continues funding of the $50 M Patient Continues funding of the $50 M Patient Safety Health Care Information Technology Safety Health Care Information Technology (IT) Program(IT) Program
$7 M in planning grants end in FY 2004. Re-$7 M in planning grants end in FY 2004. Re-invest these funds in new implementation invest these funds in new implementation grants in FY 2005grants in FY 2005
FY 2005 Non-Patient SafetyFY 2005 Non-Patient Safety
GRANTS: Renewal of Existing AHRQ GRANTS: Renewal of Existing AHRQ Programs (+$14.039 Million)Programs (+$14.039 Million)
– Small, Conference and Small, Conference and DissertationDissertation Grants Grants– Career Development AwardsCareer Development Awards– BRIC and M-RISPBRIC and M-RISP– HIT ImplementationHIT Implementation– CERTsCERTs– PBRNs PBRNs
FY 2005 Non-Patient SafetyFY 2005 Non-Patient Safety
New Program (+$6.3 Million)New Program (+$6.3 Million)– Research Empowering America’s Research Empowering America’s
Changing Healthcare System (Changing Healthcare System (REACHESREACHES) ) will focus on adoption and assessment, will focus on adoption and assessment, and will fund demonstration projects for and will fund demonstration projects for translating existing research into clinical translating existing research into clinical practice and managing a changing practice and managing a changing environment in health care organizations environment in health care organizations (includes contract component)(includes contract component)
FY 2005 MEPSFY 2005 MEPS
FY 2005 MEPS - $55.3 millionFY 2005 MEPS - $55.3 million
– Maintains the FY 2004 enacted levelMaintains the FY 2004 enacted level
Ongoing OpportunitiesOngoing Opportunities
Renewed Program AnnouncementsRenewed Program Announcements– Translating Research Into Practice and PolicyTranslating Research Into Practice and Policy– Impact of Payment and Organization on Cost, Impact of Payment and Organization on Cost,
Quality, and EquityQuality, and Equity– Patient-Centered CarePatient-Centered Care
BT Program AnnouncementBT Program Announcement
Training OpportunitiesTraining Opportunities Pre and Postdoctoral TrainingPre and Postdoctoral Training
– National Research Service Awards (NRSA)National Research Service Awards (NRSA) Institutional Training Programs (T32)Institutional Training Programs (T32) Individual Predoctoral Fellowships (F31) and Individual Predoctoral Fellowships (F31) and Individual Postdoctoral Fellowships (F32)Individual Postdoctoral Fellowships (F32)
– Dissertation Grants (R36)Dissertation Grants (R36)
Career Development AwardsCareer Development Awards – Mentored Clinical Scientist Awards (K08)Mentored Clinical Scientist Awards (K08)– Independent Scientist Awards (K02)Independent Scientist Awards (K02)
Research Infrastructure Support ProgramsResearch Infrastructure Support Programs– Minority Research Infrastructure Support Program Minority Research Infrastructure Support Program – Building Research Infrastructure and Capacity Building Research Infrastructure and Capacity
Training: What Success Looks Training: What Success Looks LikeLike
Graduation
Publication
Visibility & Dissemination
Change Practices
Save Lives and Dollars
OverviewOverview
““News you can use”News you can use” What’s New at AHRQWhat’s New at AHRQ TRIPP Now!TRIPP Now! Qs and AsQs and As
Healthcare Research and Quality Healthcare Research and Quality Act (PL. 106-129)Act (PL. 106-129)
““Beginning in fiscal year 2003, the Beginning in fiscal year 2003, the Secretary, acting through the Director, Secretary, acting through the Director, AHRQ shall submit to CongressAHRQ shall submit to Congress
– an annual report on national trends in the an annual report on national trends in the quality of health carequality of health care provided to the provided to the American peopleAmerican people
– An annual report on “prevailing An annual report on “prevailing disparities disparities in health carein health care delivery as it relates to delivery as it relates to racial factors and socioeconomic factors in racial factors and socioeconomic factors in priority populations.” priority populations.”
HHS Reports: Quality and HHS Reports: Quality and Disparities in Health Care Disparities in Health Care
First national comprehensive efforts to measure the First national comprehensive efforts to measure the quality of health care in America and disparities in quality of health care in America and disparities in access to health care services for priority populationsaccess to health care services for priority populations
Presents data for clinical conditions, (cancer, diabetes, Presents data for clinical conditions, (cancer, diabetes, end-stage renal disease, heart disease, HIV and AIDS, end-stage renal disease, heart disease, HIV and AIDS, mental health, and respiratory disease)mental health, and respiratory disease)
Includes data on maternal and child health, nursing Includes data on maternal and child health, nursing home and home health care, and patient safetyhome and home health care, and patient safety
How Reports Are RelatedHow Reports Are Related
Disparities in health care presented as quality provided Disparities in health care presented as quality provided to different populationsto different populations
Improving quality can result in concurrent decreases in Improving quality can result in concurrent decreases in disparities associated with race and gender (e.g., ESRD)disparities associated with race and gender (e.g., ESRD)
NHDR focuses on NHDR focuses on – Equity dimension of qualityEquity dimension of quality– Access-related barriers to quality careAccess-related barriers to quality care
NHQR: Missed OpportunitiesNHQR: Missed Opportunities
Only Only 30%30% of patients with diabetes receive all of patients with diabetes receive all recommended testsrecommended tests
90% of adults are screened for high blood 90% of adults are screened for high blood pressure – but only pressure – but only 25%25% are controlled are controlled
Nearly Nearly 1/31/3 of adults and children with asthma of adults and children with asthma do do NOTNOT receive effective treatment receive effective treatment
Almost Almost 20%20% of persons with a usual source of of persons with a usual source of care report that they are care report that they are notnot asked about asked about medications to prevent interactionsmedications to prevent interactions
NHQR-DR Summary (1)NHQR-DR Summary (1)
37 of 57 areas with trend 37 of 57 areas with trend data presented in the NHQR data presented in the NHQR show no improvement or show no improvement or have deteriorated have deteriorated
Fewer than one in five Fewer than one in five people with hypertension people with hypertension have it under controlhave it under control
About one in five elderly About one in five elderly Americans prescribed Americans prescribed inappropriate/potentially inappropriate/potentially harmful medications harmful medications
High quality health care is not a given in High quality health care is not a given in the U.S. health care system.the U.S. health care system.
Patient Safety: Inappropriate drug prescription for community-dwelling elderly Americans
0
2
4
6
8
10
12
14
16
1996 1998
11 drugs that shouldalways be avoided
8 drugs that arerarely appropriate
14 drugs that havesome indications butare often misused
Percent of the elderly
NHQR-DR Summary (2)NHQR-DR Summary (2)
Blacks and Hispanics — score lower than Blacks and Hispanics — score lower than whites on about half of quality measureswhites on about half of quality measures
Hispanics and Asians — score lower than Hispanics and Asians — score lower than whites on about two-thirds of access whites on about two-thirds of access measuresmeasures
Poor people — score lower on about two-Poor people — score lower on about two-thirds of quality and access measuresthirds of quality and access measures
Gaps in health care quality are Gaps in health care quality are particularly acute for certain racial, particularly acute for certain racial, ethnic, and socioeconomic groups.ethnic, and socioeconomic groups.
NHQR-DR Summary (3)NHQR-DR Summary (3)
Smoking cessation counseling
60
40
0
20
40
60
80
100
% smokers receiving adviceto quit smoking
% AMI patients given smokingcessation counseling in hosp
Only 40% of people get smoking cessation counseling in Only 40% of people get smoking cessation counseling in the hospital. Only 60% get counseling during office visitsthe hospital. Only 60% get counseling during office visits
Black, Hispanic, poor adults—less likely to receive Black, Hispanic, poor adults—less likely to receive colorectal and breast cancer screening, influenza colorectal and breast cancer screening, influenza immunizationimmunization
Black, Hispanic,American Indian women—less likely to Black, Hispanic,American Indian women—less likely to receive prenatal carereceive prenatal care
Black, Hispanic, poor children— less likely to receive Black, Hispanic, poor children— less likely to receive dental caredental care
Black, Hispanic, poor elderly—less likely to receive Black, Hispanic, poor elderly—less likely to receive pneumococcal vaccination pneumococcal vaccination
Quality and disparity gaps are worse in preventive Quality and disparity gaps are worse in preventive services.services.
NHQR-DR Summary (4)NHQR-DR Summary (4)
Use of beta-blockers for heart attack Use of beta-blockers for heart attack patients rose from 21% of eligible patients rose from 21% of eligible patients in the early 1990s to 79%. patients in the early 1990s to 79%. 45 States are at or above 70% on 45 States are at or above 70% on this measurethis measure..
70% of women over 40 get 70% of women over 40 get mammograms for breast cancer. mammograms for breast cancer. This exceeds Healthy People 2010 This exceeds Healthy People 2010 objective. objective.
Black women have higher screening Black women have higher screening rates for cervical cancer. Death rates rates for cervical cancer. Death rates among black women are falling at among black women are falling at twice the rate as white women.twice the rate as white women.
Quality improvement efforts have Quality improvement efforts have resulted in reductions in black-white resulted in reductions in black-white differences in hemodialysis. differences in hemodialysis.
Percent of AMI patients prescribed a beta Percent of AMI patients prescribed a beta blocker at discharge by Stateblocker at discharge by State
Significantly below national avg.Significantly below national avg.
No different from national avg.No different from national avg.
Significantly above national avg.Significantly above national avg.
Improvement in quality and disparities is possible.Improvement in quality and disparities is possible.
Take Home PointsTake Home Points
The reports provide the most comprehensive The reports provide the most comprehensive picture of healthcare quality and disparities to picture of healthcare quality and disparities to datedate
Their value lies in the actions and Their value lies in the actions and improvements that they will stimulateimprovements that they will stimulate
They identify a core set of measures on which They identify a core set of measures on which assessments of quality and access can be assessments of quality and access can be based based
They will monitor progress towards They will monitor progress towards improvements in quality and accessimprovements in quality and access
Priority Areas for Priority Areas for ImplementationImplementation
DiabetesDiabetes: IOM priority area; measures in both reports: IOM priority area; measures in both reports
Respiratory DiseaseRespiratory Disease: IOM priority area (Asthma; : IOM priority area (Asthma; smoking); measures in both reports – priority smoking); measures in both reports – priority population (children)population (children)
Both areas are national priorities and also particularly Both areas are national priorities and also particularly important for priority populationsimportant for priority populations
DHHS Disparities CouncilDHHS Disparities Council
The Secretary has convened a DHHS Disparities The Secretary has convened a DHHS Disparities Council to coordinate disparities research across the Council to coordinate disparities research across the DepartmentDepartment
AHRQ is leading a Council workgroup:AHRQ is leading a Council workgroup:
– To develop strategies to address disparitiesTo develop strategies to address disparities
– To make recommendations for using the NHDR and NHQR to To make recommendations for using the NHDR and NHQR to address disparitiesaddress disparities
Recent Conference on MEPS Recent Conference on MEPS Informing Health Policy Informing Health Policy
MEPS: Informing Policy on Health MEPS: Informing Policy on Health Insurance Coverage and Health Care Insurance Coverage and Health Care Costs: May 13, 2004 – Capitol Hill, D.C.Costs: May 13, 2004 – Capitol Hill, D.C.
Highlighted recent research efforts from the Highlighted recent research efforts from the survey focused on healthcare costs and survey focused on healthcare costs and coverage that help inform consumer and coverage that help inform consumer and purchaser decisions. purchaser decisions.
Facilitated discussion of utility of MEPS to Facilitated discussion of utility of MEPS to inform policy and decisions by consumers and inform policy and decisions by consumers and purchaserspurchasers
Types of Analyses Supported by Types of Analyses Supported by MEPS Prescribed Medicine DataMEPS Prescribed Medicine Data
Trends in out of pocket burdens across all major Trends in out of pocket burdens across all major population subgroupspopulation subgroups
Examine burden on individuals and familiesExamine burden on individuals and families Prevalence of potentially inappropriate prescribing Prevalence of potentially inappropriate prescribing
patternspatterns Trends in use and expenditures by therapeutic Trends in use and expenditures by therapeutic
category: e.g. statins, anti-depressants, analgesics, category: e.g. statins, anti-depressants, analgesics, proton pump inhibitorsproton pump inhibitors
New WorkshopsNew Workshops
September 20-21 - Hands-on Workshop in Rockville- September 20-21 - Hands-on Workshop in Rockville- Using the MEPS Prescribed Drug and Condition FilesUsing the MEPS Prescribed Drug and Condition Files
November 30-Dec 1 - Hands-on Workshop in Rockville – November 30-Dec 1 - Hands-on Workshop in Rockville – MEPS Linking IssuesMEPS Linking Issues
Cyber Seminars- 2005Cyber Seminars- 2005
http://www.meps.ahrq.govhttp://www.meps.ahrq.gov
3 New States3 New StatesJoin HCUP PartnershipJoin HCUP Partnership
Recent AdditionsRecent Additions1.1. OhioOhio – greatly improves Midwest – greatly improves Midwest
representationrepresentation
2.2. NevadaNevada
3.3. South DakotaSouth Dakota
HCUP Now Includes:HCUP Now Includes:– 36 State Partners36 State Partners– 90% of all U.S. hospital stays90% of all U.S. hospital stays– over 31 million discharges over 31 million discharges
NewNew HCUPnet Helps Both HCUPnet Helps Both Researchers and Policy AnalystsResearchers and Policy Analysts
Includes cost data beginning 2000Includes cost data beginning 2000 Has separate paths for researchers and Has separate paths for researchers and
policymakers/non-researcherspolicymakers/non-researchers Is easier to printIs easier to print
To access the new HCUPnet or information To access the new HCUPnet or information about the HCUP databases, go to: about the HCUP databases, go to: http://www.hcup-us.ahrq.gov/home.jsphttp://www.hcup-us.ahrq.gov/home.jsp
What is Section 1013?What is Section 1013?
To improve the quality, effectiveness and To improve the quality, effectiveness and efficiency of health care delivered through efficiency of health care delivered through Medicare, Medicaid and the S-CHIP programsMedicare, Medicaid and the S-CHIP programs
$50 million is $50 million is authorizedauthorized in Fiscal Year 2004 in Fiscal Year 2004 for AHRQ to conduct and support research for AHRQ to conduct and support research with a focus on outcomes, comparative clinical with a focus on outcomes, comparative clinical effectiveness and appropriateness of health effectiveness and appropriateness of health care items and services (including care items and services (including pharmaceutical drugs), including strategies for pharmaceutical drugs), including strategies for how these items and services are organized, how these items and services are organized, managed and deliveredmanaged and delivered
What is Section 1013?What is Section 1013?
By June 2004, the Secretary shall By June 2004, the Secretary shall establish an establish an initialinitial list of research list of research priorities (including those related to priorities (including those related to prescription drugs)prescription drugs)
Priorities may include health care items Priorities may include health care items and services which impose a high cost and services which impose a high cost on Medicare, Medicaid or S-CHIP, on Medicare, Medicaid or S-CHIP, including those that may be including those that may be underutilized or over utilizedunderutilized or over utilized
What is Section 1013?What is Section 1013?
By June 2005, the Secretary shall identify By June 2005, the Secretary shall identify options to disseminate in a timely fashion options to disseminate in a timely fashion outcomes, quality of patient care, clinical data outcomes, quality of patient care, clinical data and patient-reported outcomes, which could and patient-reported outcomes, which could include voluntary collaboration with private include voluntary collaboration with private and public entitiesand public entities
No later than December 2005, AHRQ shall No later than December 2005, AHRQ shall complete its evaluation and synthesis of complete its evaluation and synthesis of available scientific evidence related to the available scientific evidence related to the initialinitial list developed by the Secretary, which list developed by the Secretary, which shall be made available to the Medicare shall be made available to the Medicare program, other health plans, and the publicprogram, other health plans, and the public
What is Not in Section 1013?What is Not in Section 1013?
AHRQ shall not mandate national AHRQ shall not mandate national standards of clinical practice or quality standards of clinical practice or quality health care standardshealth care standards
CMS may not use data obtained through CMS may not use data obtained through this provision to withhold coverage of a this provision to withhold coverage of a prescription drugprescription drug
No mandate to perform cost-No mandate to perform cost-effectiveness studieseffectiveness studies
No appropriation in FY 04No appropriation in FY 04
OverviewOverview
““News you can use”News you can use” What’s New at AHRQWhat’s New at AHRQ TRIPP Now!TRIPP Now! Qs and AsQs and As
New AHRQ Mission New AHRQ Mission StatementStatement
To improve the quality, safety, To improve the quality, safety, efficiency, and effectiveness of efficiency, and effectiveness of health care for all Americanshealth care for all Americans
Implications of ‘New’ Mission Implications of ‘New’ Mission
Emphasis on production and use of evidenceEmphasis on production and use of evidence
Increased emphasis on ‘value-added’ approach to Increased emphasis on ‘value-added’ approach to grant makinggrant making
Increased synergy between intramural and Increased synergy between intramural and extramural research: within portfoliosextramural research: within portfolios
Enhanced emphasis on problem-solving Enhanced emphasis on problem-solving user user input into the relevance questioninput into the relevance question
Safety/QualitySafety/Quality EfficiencyEfficiency EffectivenessEffectiveness
System Capacity and System Capacity and Emergency Emergency PreparednessPreparedness
Data DevelopmentData Development
Care ManagementCare Management
Cost, Organization Cost, Organization and Scoio-and Scoio-EconomicsEconomics
Health Information Health Information TechnologyTechnology
Long-Term CareLong-Term Care
Pharmaceutical Pharmaceutical OutcomesOutcomes
PreventionPrevention
TrainingTraining
Quality/Safety of Quality/Safety of Patient CarePatient Care
Portfolios of WorkPortfolios of Work
AHRQ – As a Science PartnerAHRQ – As a Science Partner
Fund and conduct research on issues Fund and conduct research on issues important to decisionmakers important to decisionmakers – ClinicalClinical– Health SystemHealth System– PolicyPolicy
Close the gap between evidence and practiceClose the gap between evidence and practice Nurture the next generation of health services Nurture the next generation of health services
researchersresearchers
AHRQ Core ActivitiesAHRQ Core Activities
Research: Discovering New
Knowledge
Implementation:Turning Evidence into
Action
Improvements in Quality & Outcomes
Supply-Side Research ParadigmSupply-Side Research Paradigm
Research world:
• Questions• Hypothesis
• StudyPublicationPublication
User world:
• Many needs• Beliefs & interests • Decision processes
The winding road The winding road to a receptor siteto a receptor site
Research world:
• Questions• Hypothesis
• Study
User world:
• Many needs• Beliefs & interests • Decision processes
Demand/Supply Side Model
What We Have LearnedWhat We Have Learned
Knowing the right thing to do is Knowing the right thing to do is NOTNOT = doing it! = doing it! Improvement must be based on scienceImprovement must be based on science Patients as participants are far more effective Patients as participants are far more effective
than patients as ‘recipients’than patients as ‘recipients’ Researchers may not be the best implementersResearchers may not be the best implementers Folks at all levels are engagedFolks at all levels are engaged
Translational ChallengesTranslational Challenges
Are researchers change agents?Are researchers change agents? What is the appropriate skill set?What is the appropriate skill set? In the research world are there incentives for In the research world are there incentives for
TRIPP (tenure, promotion, career track)?TRIPP (tenure, promotion, career track)? How do we train translational researchers?How do we train translational researchers? What are the incentives for TRIPP?What are the incentives for TRIPP?
Ten Roles of Government Ten Roles of Government in Health Care Qualityin Health Care Quality
Purchase health carePurchase health care Provide health careProvide health care Assure access for vulnerable Assure access for vulnerable
populationspopulations Monitor health care qualityMonitor health care quality Regulate health care marketsRegulate health care markets
Inform health care decision- makersInform health care decision- makers Support acquisition of new knowledgeSupport acquisition of new knowledge Support development of health Support development of health
technologies and practicestechnologies and practices Train the health care researchersTrain the health care researchers Convene stakeholdersConvene stakeholders
Getting a Grant: Before Getting a Grant: Before Develop and refine question and Develop and refine question and
develops testable hypotheses.develops testable hypotheses. Submit grant.Submit grant. Receive grant! -Receive grant! - do do immenseimmense
amounts of work; publish; get promoted.amounts of work; publish; get promoted. Start all over again.Start all over again. Receptor sites for findings assumedReceptor sites for findings assumed Contact with program staff optionalContact with program staff optional
Getting a Grant: FutureGetting a Grant: Future Develop idea – consult with usersDevelop idea – consult with users Before submission, identify who will use Before submission, identify who will use
findings and findings and howhow Include in application clear plan for Include in application clear plan for
ongoing consultation with usersongoing consultation with users Include users in peer reviewInclude users in peer review Share key findings* with users before Share key findings* with users before
publicationpublication Consultation with program staff essentialConsultation with program staff essential
Research Implementation: Research Implementation: Debunked AssumptionDebunked Assumption
QuestionQuestion
HypothesisHypothesis
StudyStudy
PublicationsPublications
Changes in practiceChanges in practice
““Opportunity is missed by most because it Opportunity is missed by most because it is dressed in overalls and looks like workis dressed in overalls and looks like work.”.”
Thomas EdisonThomas Edison