evidence farming and open architecture
DESCRIPTION
Presented at Mobile Health 2011, May 2011, Stanford.TRANSCRIPT
![Page 1: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/1.jpg)
Evidence Farming1: Implications forOpen Architecture
Ida Sim, MD, PhDDirector, Center for Clinical and Translational Informatics
University of California San FranciscoMay 5, 2011
1With thanks to Rich Kravitz MD, UC Davis and Naihua Duan, Columbia
![Page 2: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/2.jpg)
Rephrasing “Does it Work?”
(Complexes of)Exposures Outcome
strength of association?
individual
population
IncreasedbreastfeedingText4Baby
![Page 3: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/3.jpg)
Current Approaches: RCT
• Tests prespecified interventions and outcomes• To confirm a hypothesis at the population level• Strong internal validity• Problems: slow to set-‐up, expensive, short-‐term, lack
relevance to the real world
ER visits at 1 year50 people population
100 people
ER visits at 1 year50 people
Asthma App
Usual Care
![Page 4: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/4.jpg)
Exposures Outcomes?
population
Current Approaches: Data Mining
• Exposures and outcomes from care process systems• To generate hypotheses at the population level• Problems: limited to data collected, weak internal
validity (data not complete or systematic)
EHR
Apps
![Page 5: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/5.jpg)
Current Approaches:N-‐of-‐1 Studies
• Within-‐subject multiple crossover• Only formal method for determining individual
treatment effectiveness• Problems: complicated to set up, analysis is
difficult, little known, not widely used
individual
peak flowpeak flowUsual Care
Asthma app
Asthma app
Usual Care
Asthma app
Usual Care
![Page 6: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/6.jpg)
Evidence Extraction
• Evidence is something to be extractedfrom the care process– mining it from the data– directly manipulating the care process withrigid and pre-‐defined protocols
![Page 7: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/7.jpg)
Evidence Strip Mining
![Page 8: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/8.jpg)
Evidence Farming
Hay, et al. J Eval Clin Prac 14(2008):707-713.
![Page 9: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/9.jpg)
Rooting for Evidence
![Page 10: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/10.jpg)
Industrial Evidence Farming
ER visits at 1 year50 people population
100 people
ER visits at 1 year50 people
Asthma App
Usual Care
![Page 11: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/11.jpg)
Personal Evidence Gardens
individual
peak flowpeak flowUsual Care
Asthma app
Asthma app
Usual Care
Asthma app
Usual Care
![Page 12: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/12.jpg)
Personal Evidence Gardens
individual
dancingFlovent PRN
Flovent
Flovent
Flovent PRN
Flovent
Flovent PRN
dancing
![Page 13: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/13.jpg)
Crowdsourcing What Matters
• (Complexes of) Exposures– does chocolate trigger (my) asthma?– testing common regimens (ACEI, statin, b-‐blocker),
complementary medicines
• (Complexes of) Outcomes– what outcomes do patients care about?
![Page 14: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/14.jpg)
Evidence MacrosystemRooting forEvidence
Industrial EvidenceFarming
Personal EvidenceGardens
![Page 15: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/15.jpg)
How can we scale evaluation?
![Page 16: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/16.jpg)
StovepipedmHealth
• Health apps builtindependently– little data sharing and
interoperability
• Limits efficiency andimpact of qualitymHealth
![Page 17: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/17.jpg)
Internet Hourglass Model
• Standardize andmake open the“narrow waist”
• Reduces duplication,spurs communityinnovation, supportscommercial and non-‐profit uses
![Page 18: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/18.jpg)
OpenmHealth.org
Estrin DE, Sim I. Science; 330: 759-60. 2010.
![Page 19: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/19.jpg)
• The waist should supportthe evidence macrosystem
OpenmHealth.org
![Page 20: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/20.jpg)
Open Architecture for anEvidence Macrosystem
• Modules for usage analytics– # of text messages, # of sessions, etc.
• Rooting for (glocal) evidence– data sharing with shared syntax and semantics
• Industrial farming, e.g., with RCTs– modules for informed consent, randomization, adaptive
treatment strategy, mixed methods, etc.
• Personal evidence gardening, e.g., N-‐of-‐1– modules for scripting and analyzing individualized N-‐of-‐
1 protocols, etc.
![Page 21: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/21.jpg)
Open Architecture for anEvidence Macrosystem
• Social media for discovery of exposures andoutcomes that matter
• Shared libraries of validated measures andinstruments (e.g., PROMIS)– measures that get at finer-‐grained mechanisms based
on theoretical models of change, etc.
![Page 22: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/22.jpg)
![Page 23: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/23.jpg)
Goal for mHealth Evidence
• A learning community coupled with anopen architecture for broad, rapid, anditerative dissemination of evaluationmethods and findings that matter
![Page 24: Evidence Farming and Open Architecture](https://reader033.vdocuments.us/reader033/viewer/2022051817/5492f13ab479594c4d8b46fe/html5/thumbnails/24.jpg)
• Ida Sim [email protected]• Deborah Estrin [email protected]• http://openmhealth.org/