research in the real world: rebecca etz rebecca etz deborah cohen deborah cohen nicole isaacson...

Research in the Real Research in the Real World:World:

Rebecca EtzRebecca Etz Deborah CohenDeborah Cohen

Nicole IsaacsonNicole Isaacson Bijal Bijal

BalasubramanianBalasubramanian Benjamin CrabtreeBenjamin Crabtree Alfred TalliaAlfred Tallia

Lessons Learned from Prescription for Health

Let’s Get RealLet’s Get Real

TranslationTranslation• Realize the shift from efficacy to Realize the shift from efficacy to

effectivenesseffectiveness

DisseminationDissemination• Capture Capture context and strategycontext and strategy as well as well

as outcomesas outcomes

Public Health ImpactPublic Health Impact• Create Create partnershipspartnerships in the community in the community

ConceptualizationConceptualization• Studying implementation as way to Studying implementation as way to

evaluate what ‘works’ in real-life evaluate what ‘works’ in real-life setting (i.e. setting (i.e. effectivenesseffectiveness))

Methods UsedMethods Used• Practice SurveysPractice Surveys• Site Visits/InterviewsSite Visits/Interviews• Online diaryOnline diary

Evaluation of P4HEvaluation of P4HThe Analysis Team’s FocusThe Analysis Team’s Focus

Evaluation Evaluation The Analysis Team’s FocusThe Analysis Team’s Focus

Data Analysis – Qualitative Data Analysis – Qualitative DataData

11stst: Read the diary data : Read the diary data

22ndnd: Read it again: Read it again

33rdrd: Really read the data: Really read the data

Evaluation Evaluation The Analysis Team’s FocusThe Analysis Team’s Focus

Data Analysis – Quantitative Data Analysis – Quantitative DataData

11stst: Find missing data: Find missing data

22ndnd: Fix any inconsistencies: Fix any inconsistencies

33rdrd: Convert numbers into : Convert numbers into wordswords


TranslationTranslation

• Realize the shift from efficacy to Realize the shift from efficacy to effectivenesseffectiveness

• PBRNs are an ideal environmentPBRNs are an ideal environmentResearch oriented, but not unusualResearch oriented, but not unusual

85 practices (10 PBRNs)85 practices (10 PBRNs)

92% Family medicine practices92% Family medicine practices

Staff turnover of 8-12%Staff turnover of 8-12%

PBRNs and Getting PBRNs and Getting RealRealREAIM as our lensREAIM as our lens

• REAIM was proactively for the first REAIM was proactively for the first time in practice-based, primary time in practice-based, primary care researchcare research

• Across all 10 projects in round 2, Across all 10 projects in round 2, P4HP4H

RReach, each, EEffectiveness, ffectiveness, AAdoption, doption, IImplementation, mplementation, MMaintenanceaintenance

PBRNs and Getting PBRNs and Getting RealRealREAIM as our lensREAIM as our lensREAIM and our qualitative data allowed us to REAIM and our qualitative data allowed us to

ask ask

different kinds of questions:different kinds of questions:

• HowHow do we take something that we know do we take something that we know works, or that we think will work, and put works, or that we think will work, and put that thing into practice?that thing into practice?

• HowHow did you get it to work? did you get it to work?

• WhyWhy did it work and can we get it to work did it work and can we get it to work elsewhere?elsewhere?

PBRNs and Getting PBRNs and Getting RealRealREAIM as our lensREAIM as our lens

Two themes emerged from the diary Two themes emerged from the diary data – the tensions between…data – the tensions between…

Flexibility & FidelityFlexibility & Fidelity ……an issue of an issue of translationtranslation

Quality Improvement & Quality Improvement & ResearchResearch

… …an issue of an issue of disseminationdissemination

PBRNs and Getting PBRNs and Getting RealRealFidelity vs FlexibilityFidelity vs Flexibility

Fidelity and FlexibilityFidelity and Flexibility

• In the primary care settingIn the primary care setting, , interventions are not plug and playinterventions are not plug and play

• They are tailoredThey are tailored to the contours of to the contours of the practicethe practice

“… “… Dr made the comment that we need to come Dr made the comment that we need to come up with something that will work at a 1 nurse up with something that will work at a 1 nurse office like hers”office like hers”


Flexibility is a reflection of the Flexibility is a reflection of the inescapable realness of the real worldinescapable realness of the real world

‘‘The other striking thing is the varied ways that the The other striking thing is the varied ways that the practices are implementing the HRA:practices are implementing the HRA:

• Keep it Keep it at receptionat reception, given to patients at , given to patients at check check inin

• Give it to Give it to one staff memberone staff member, he , he administersadministers allall• Given to all patients of the in house Given to all patients of the in house diabetes diabetes

educatoreducator• Keep it at nurses station, used primarily by Keep it at nurses station, used primarily by

‘‘one care giving teamone care giving team’’


• Understanding how research is Understanding how research is integrated into real life practice is integrated into real life practice is essentialessential

• PBRNs facilitate that translation with PBRNs facilitate that translation with all the muckiness that it impliesall the muckiness that it implies


TranslationTranslation• Realize the shift from efficacy to effectivenessRealize the shift from efficacy to effectiveness

DisseminationDissemination

• Capture Capture context and strategycontext and strategy as as well as outcomeswell as outcomes

• Share Share solutions for navigatingsolutions for navigating the the tension between QI and researchtension between QI and research

PBRNs and Getting PBRNs and Getting RealRealQuality Improvement vs ResearchQuality Improvement vs Research


• Front line PCPs are overwhelmed – this is Front line PCPs are overwhelmed – this is realityreality

• Choices for recruitment are often about Choices for recruitment are often about choosing practices positioned for choosing practices positioned for improvementimprovement

• PBRNs allow us to see PBRNs allow us to see howhow those that those that actually take the change on are actually take the change on are able to able to make it happenmake it happen


On the ground, team members manage On the ground, team members manage patient expectationspatient expectations

““[Staff] report [this] is the first thing the patient [Staff] report [this] is the first thing the patient asks: asks: what do I get for participationwhat do I get for participation?”?”

Projects that were seamless – where Projects that were seamless – where increased service was in the foreground, increased service was in the foreground, research in the background – faired bestresearch in the background – faired best


QI is ‘institutional’ where research is finite QI is ‘institutional’ where research is finite in terms of time & resourcesin terms of time & resources

““We had practices We had practices take a two week breaktake a two week break from from referring patients … It has been more difficult than I referring patients … It has been more difficult than I had anticipated to get them back up and running ...”had anticipated to get them back up and running ...”

““We tried to fine tune our message for the staff ... That We tried to fine tune our message for the staff ... That is, is, one patient enrollment per [staff member] per dayone patient enrollment per [staff member] per day.”.”

PBRNs showed us institutionalization was PBRNs showed us institutionalization was manageable but involved sacrificesmanageable but involved sacrifices

Investigating with Investigating with REAIMREAIMWhere’s the Tension – QI and Where’s the Tension – QI and ResearchResearch• Front line practices don’t have a lot of Front line practices don’t have a lot of

slackslack

• Research teams bring resources and it’s Research teams bring resources and it’s hard to hold up when they are gonehard to hold up when they are gone

““This practice [does] prevention because it’s This practice [does] prevention because it’s important, not because it’s reimbursed. [We] offered important, not because it’s reimbursed. [We] offered to keep the intervention on at this practice (but to keep the intervention on at this practice (but without free services). They pulled the plug on the without free services). They pulled the plug on the project.project.””

Investigating with Investigating with REAIMREAIMWhere’s the Tension – QI and Where’s the Tension – QI and ResearchResearch

PBRN based research allows practices PBRN based research allows practices to build a line of inquiry that:to build a line of inquiry that:

• shows the value of what they did shows the value of what they did

• wins funding enabling translation of wins funding enabling translation of research into practice research into practice

PBRNs are an environment that makes PBRNs are an environment that makes translation happentranslation happen


TranslationTranslation• Realize the shift from efficacy to effectivenessRealize the shift from efficacy to effectiveness

DisseminationDissemination• Capture context and strategy, share solutionsCapture context and strategy, share solutions

Public Health ImpactPublic Health Impact

• Create Create partnershipspartnerships in the in the communitycommunity

• Fill in the missing elementsFill in the missing elements

In the Real World In the Real World … “linking … “linking to community resources is easier said to community resources is easier said

than done”than done”

Primary Care Primary Care ElementsElements

• Identify patients at Identify patients at riskrisk

• Capacity to refer Capacity to refer patientspatients

• Know how to access Know how to access community resourcescommunity resources

Community Community ResourceResource

ElementsElements• AvailabilityAvailability

• AffordabilityAffordability

• AccessibilityAccessibility

• Perceived as Perceived as valuablevaluable

Primary Care ElementsPrimary Care ElementsIdentifying Patients at RiskIdentifying Patients at Risk

Many practices did not have processes in place to identify patients at risk…

Use of HRA in P4H practices

56.4750.59

38.8232.94 34.12

0

20

40

60

80

100

tobacco alcohol diet physicalactivity

none

% p

ract

ices

Primary Care ElementsPrimary Care ElementsIdentifying Patients at RiskIdentifying Patients at Risk

…Project teams were able to help them develop what was needed

“[This] practice had, with the help of our facilitator, modified their vital signs template in the EHR (eClinical Works) to include smoking. His staff review smoking and update status at every visit. The inclusion of smoking as a vital sign plus verbal reminders by the nurses keeps the issue in front of [the doctor] and appears to have increased referrals to the Quit Line and his use of brief counseling strategies.”

Primary Care ElementsPrimary Care ElementsMaking the ReferralMaking the Referral

• Limited capacity for referrals was a barrier ... Limited capacity for referrals was a barrier ...

““If we need to print referrals for faxing…, If we need to print referrals for faxing…, maybe maybe someone in the [research team’s] office could do itsomeone in the [research team’s] office could do it (some concern about stressing the referral person...)”(some concern about stressing the referral person...)”

• Project teams offered different strategies for Project teams offered different strategies for simplifying referralssimplifying referrals

““Having the direct Having the direct real-time linkreal-time link to the designated to the designated outreach person…outreach person…makes it very easymakes it very easy for the for the

physician to physician to make the referral… quick and easy.”make the referral… quick and easy.”

Primary Care ElementsPrimary Care ElementsMaking the TimeMaking the Time

• Limited time was another barrier ... Limited time was another barrier ...

““Talked with [Doctor] 6:30 am... had read Talked with [Doctor] 6:30 am... had read material, material, [said] it sounded like a good study, [said] it sounded like a good study, as as long as it didn’t long as it didn’t take too much time would be take too much time would be happy to do ithappy to do it ( (like 30 like 30 secondsseconds).”).”

• And project teams helped practices met And project teams helped practices met these time constraintsthese time constraints

““The rooming staff person reported that she The rooming staff person reported that she did the did the P4H pop-up as part of the vital signs P4H pop-up as part of the vital signs process. process. She She reports it was not extra work/timereports it was not extra work/time.”.”

Primary Care ElementsPrimary Care ElementsKnowing How to Access/Use Knowing How to Access/Use ResourcesResources

• Community resources are fluid, many, and difficult to track

• Projects addressed this in a number of Projects addressed this in a number of waysways

Web-tools, HIT, resource cardsWeb-tools, HIT, resource cards

People as portals to community People as portals to community resourcesresources

Tapping into national, state, or Tapping into national, state, or health health systems systems

Building relationships with resourcesBuilding relationships with resources

Community Resource Community Resource ElementsElementsAvailability, Affordability…Availability, Affordability…• Many practices, particularly those in rural Many practices, particularly those in rural

and low income areas, reported a scarcity of and low income areas, reported a scarcity of community resourcescommunity resources

• When resources did exist in communities, When resources did exist in communities, they were often not affordablethey were often not affordable

““A barrier for counseling is patients affording it - A barrier for counseling is patients affording it - many would be interested in many would be interested in group visits if free, group visits if free, reason for interest in the studyreason for interest in the study……””

““Currently they will refer to some nutritionists, but Currently they will refer to some nutritionists, but sporadic and sporadic and dependant on what insurance will dependant on what insurance will covercover, more often for diabetic patients”, more often for diabetic patients”

Community Resource Community Resource ElementsElementsAccessibilityAccessibility

Grounded and informed by patient Grounded and informed by patient need, projects teams took many need, projects teams took many creative approachescreative approaches

• Offering in-house counselingOffering in-house counseling• Developing website and IVR systems Developing website and IVR systems • Offering and paying for branded Offering and paying for branded

servicesservices• Leveraging their relationships with Leveraging their relationships with

community resources community resources

PBRNs are RealPBRNs are Real

P4H projects, and the use of REAIM, show P4H projects, and the use of REAIM, show

PBRNs as an ideal setting in which to PBRNs as an ideal setting in which to innovateinnovate

• ProvidingProviding real-world effectiveness real-world effectiveness

Balancing internal and external validityBalancing internal and external validity

• FosteringFostering proactive intervention designs proactive intervention designs

Greater public appeal and rapid adoptionGreater public appeal and rapid adoption

• EnablingEnabling view of view of potentialpotential public impact public impact

Benefit of qualitative and quantitative in Benefit of qualitative and quantitative in unisonunison

PBRNs are RealPBRNs are Real

TranslationTranslation• IntegrateIntegrate: Full integration of RE-AIM model into : Full integration of RE-AIM model into

research and intervention design from project research and intervention design from project conceptionconception

DisseminationDissemination• ContextContext: qualitative and quantitative work : qualitative and quantitative work

together to suggest together to suggest potentialpotential

Public Health ImpactPublic Health Impact• AdaptAdapt: Interventions must be responsive to site : Interventions must be responsive to site

specific needs – functional, cultural, emotionalspecific needs – functional, cultural, emotional

research in the real world: rebecca etz rebecca etz deborah cohen deborah cohen nicole isaacson...

Documents