medication safety as a use case for argumentation mining, dagstuhl seminar 16161, 2016 04-19

Medication safety as a use case for argumentation

miningJodi Schneider and Richard D. Boyce

Dagstuhl Seminar 16161Natural Language Argumentation: Mining, Processing, and Reasoning over Textual Arguments2016-04-19 1

InformaticsThe management and processing of data, information and knowledge.

2[Fourman 2002]

InformaticsThe management and processing of data, information and knowledge.

Examples:o Biomedical informaticso Dental informaticso Legal informaticso Business informaticso Chemical informaticso Neurinformaticso ...

3[Fourman 2002]

Evidence InformaticsThe management and processing of data, information and knowledge ABOUT evidence.

4

Evidence InformaticsThe management and processing of data, information and knowledge ABOUT evidence.

Develop end-user applications.e.g. Information retrieval using arguments & evidence.o Kevin’s “legal argument roles”o Benno’s PageRank for argumentso Retrieve scientific articles by rhetorical or

argumentative features.

5

Evidence InformaticsThe management and processing of data, information and knowledge ABOUT evidence.

Develop end-user applications.e.g. Information retrieval using arguments & evidence.o Kevin’s “legal argument roles”o Benno’s PageRank for argumentso Retrieve scientific articles by rhetorical or

argumentative features.

Seek reusable underlying principles, shared between several fields. 6

My approach to evidence informaticso Understand user tasks and reasoning.o Identify domain-specific argumentation

schemes.o Fill a knowledge base with arguments.

• Use domain-specific argumentation schemes as templates.

• Fill “slots” in the scheme.• Hand-annotate to bootstrap information extraction.

o Search engine for arguments and evidence• Use rhetorical structures.• Use argumentative structures.

7

MEDICATION SAFETY

8

Prescribers check for known drug interactions.

9

Prescribers consult drug compendia which are maintained by expert pharmacists.

Medscape EpocratesMicromedex 2.0

10

Prescribers consult drug compendia which are maintained by expert pharmacists.

Medscape EpocratesMicromedex 2.0

Significant discrepancies on drug interactions!

11

Problem

o Thousands of preventable medication errors occur each year.

o Clinicians rely on information in drug compendia (Physician’s Desk Reference, Medscape, Micromedex, Epocrates, …).

o Compendia have information quality problems:• differ significantly in their coverage, accuracy, and

agreement• often fail to provide essential management

recommendations about prescription drugs

12

Problem

o Drug compendia synthesize drug interaction evidence into knowledge claims but:• Disagree on whether specific evidence items can

support or refute particular knowledge claims

13

Problem

o Drug compendia synthesize drug interaction evidence into knowledge claims but:• Disagree on whether specific evidence items can

support or refute particular knowledge claims• May fail to include important evidence

14

“Addressing gaps in clinically useful evidence on drug-drug interactions”

4-year project, U.S. National Library of Medicine R01 grant (PI, Richard Boyce; R01LM011838)o Evidence panel of domain experts: Carol

Collins, Amy Grizzle, Lisa Hines, John R Horn, Phil Empey, Dan Malone

o Informaticists: Jodi Schneider, Harry Hochheiser, Katrina Romagnoli, Samuel Rosko

o Ontologists: Mathias Brochhausen, Bill Hogano Programmers: Yifan Ning, Wen Zhang, Louisa

Zhang

15

Goals

o Long-term, provide drug compendia editors with better information and better tools, to create the information clinicians use.

o This talk focuses on how we might efficiently acquire and represent • knowledge claims about medication safety• and their supporting evidence

o In a standard computable format.

16

DOMAIN-SPECIFIC ARGUMENTATION

17

Drug Interaction Probability Score

1. Are there previous credible reports in humans?• If there are case reports or prospective studies that clearly provide

evidence supporting the interaction, answer YES. For case reports, at least one case should have a “possible” DIPS rating (score of 2 or higher).  

• If a study appropriately designed to test for the interaction shows no evidence of an interaction, answer NO.

…5. Did the interaction remit upon de-challenge of the precipitant drug with no change in the object drug?  (if no de-challenge, use Unknown or NA and skip Question 6)• Stopping the precipitant drug should bring about resolution of the

interaction, even if the object drug is continued without change. …• If dechallenge of the precipitant drug without a change in object

drug did not result in remission of the interaction, answer NO.• If no dechallenge occurred, the doses of both drugs were altered,

or no information on dechallenge is provided, answer NA.[Horn et al. 2007] 18

19

[Hu et al. 2011] 20

21[Hu et al. 2011]

22[Hu et al. 2011]

23[Hu et al. 2011]

[Boyce, DIKB, 2006-present] 24

[Boyce, DIKB, 2006-present] 25

DESIGNING AN EVIDENCE BASE

26

Multiple layers of evidence

Medication Safety Studies

Layer

Clinical Studies and Experiments

Scientific Evidence Layer

27

[Brochhausen, Schneider, Malone, Empey, Hogan and Boyce “Towards a foundational representation of potential drug-drug interaction knowledge.” First International Workshop on Drug Interaction Knowledge Representation (DIKR-2014) at ICBO.] 28

SCIENTIFIC EVIDENCE LAYER

29

Scientific Evidence Layer: Micropublications

[Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications.] 30

Scientific Evidence Layer: Micropublications

[Clark, Ciccarese, Goble (2014) Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications] 31

MODELING NARRATIVE DOCUMENTS AS EVIDENCE

32

33

34

MP:Claim

35

36

Building up an MP graph

37

Building up an MP graph

38

Building up an MP graph

39

Building up an MP graph

40

Building up an MP graph

41

Building up an MP graph

42

Building up an MP graph

43

Building up an MP graph

44

Building up an MP graph

45

HAND ANNOTATION TO CREATE THE EVIDENCE BASE

46

Hand-extracting claims and evidence

o Sources• Primary research literature• Case reports• FDA-approved drug labels

o Process• Spreadsheets• PDF annotation

47

48

49

Work to date

o 410 assertions and 519 evidence items transformed from prior work.

o 609 evidence items (pharmacokinetic potential drug-drug interactions) annotated by hand from 27 FDA-approved drug labels.

o 230 assertions of drug-drug interactions annotated by hand from 158 non-regulatory documents, including full text research articles.

50

DIRECTIONS & FUTURE WORK

51

We are developing a search/retrieval portal It will:o Integrate across multiple types of source

materials (FDA drug labels, scientific literature, …)

o Systematize search: Enable ALL drug compendium editors to access the same info

o Provide direct access to source materials• E.g. quotes in context

52

Quotes in context!

53

Evaluation plan for the search/retrieval portalo 20-person user studyo Measures of

• Completeness of information• Level of agreement• Time required• Perceived ease of use

54

Generate multiple KBs from the same EB

55

My approach to evidence informaticso Understand user tasks and reasoningo Identify domain-specific argumentation

schemes.o Create arguments

• Use domain-specific argumentation schemes as templates.

• Fill “slots” in the scheme.• Hand-annotate to bootstrap information extraction.• Automate.

o Provide argument and evidence-based information retrieval• Rhetorical functions• Argumentative structures

56

Evidence modeling & curation

o Analogous processes could be used in other fields: evidence modeling & curation is a general process.

o Biomedical curation is most mature: structured nature of the evidence interpretation, existing ontologies, trained curators, information extraction and natural language processing pipelines

o Curation pipelines need to be designed with stakeholders in mind.

57

Evidence InformaticsThe management and processing of data, information and knowledge ABOUT evidence.

Develop end-user applications such as using arguments & evidence for information retrieval.o Kevin’s “legal argument roles”o Benno’s PageRank for argumentso Retrieve scientific articles by rhetorical or

argumentative features

Seek reusable underlying principles, shared between several fields.

58

Thanks to collaborators & funderso Training grant T15LM007059 from the

National Library of Medicine and the National Institute of Dental and Craniofacial Research

o The entire “Addressing gaps in clinically useful evidence on drug-drug interactions” team from U.S. National Library of Medicine R01 grant (PI, Richard Boyce; R01LM011838) and other collaborators

59

Jodi Schneider, Mathias Brochhausen, Samuel Rosko, Paolo Ciccarese, William R. Hogan, Daniel Malone, Yifan Ning, Tim Clark and Richard D. Boyce. “Formalizing knowledge and evidence about potential drug-drug interactions.” International Workshop on Biomedical Data Mining, Modeling, and Semantic Integration: A Promising Approach to Solving Unmet Medical Needs (BDM2I 2015) at ISWC 2015 Bethlehem, Pennsylvania, USA.

Jodi Schneider, Paolo Ciccarese, Tim Clark and Richard D. Boyce. “Using the Micropublications ontology and the Open Annotation Data Model to represent evidence within a drug-drug interaction knowledge base .” 4th Workshop on Linked Science 2014—Making Sense Out of Data (LISC2014) at ISWC 2014 Riva de Garda, Italy.

Mathias Brochhausen, Jodi Schneider, Daniel Malone, Philip E. Empey, William R. Hogan and Richard D. Boyce “Towards a foundational representation of potential drug-drug interaction knowledge .” First International Workshop on Drug Interaction Knowledge Representation (DIKR-2014) at the International Conference on Biomedical Ontologies (ICBO 2014) Houston, Texas, USA.

Richard D. Boyce, John Horn, Oktie Hassanzadeh, Anita de Waard, Jodi Schneider, Joanne S. Luciano, Majid Rastegar-Mojarad, Maria Liakata, “Dynamic Enhancement of Drug Product Labels to Support Drug Safety, Efficacy, and Effectiveness.” Journal of Biomedical Semantics. 4(5), 2013. doi:10.1186/2041-1480-4-5

60

Medication Safety Studies Layer: DIDEO

Brochhausen et al, work in progress, example of Clinical Trial

62

DIDEO: Drug-drug Interaction and Drug-drug Interaction Evidence Ontology

63https://github.com/DIDEO

Definitions

o Drug-drug interaction• A biological process that results in a clinically

meaningful change to the response of at least one co-administrated drug.

o Potential drug-drug interaction• POSSIBILITY of a drug-drug interaction• Data from a clinical/physiological study OR

reasonable extrapolation about drug-drug interaction mechanisms

64

Other Implications

o Implications for ontology development.o Implications for improving medication safety.

65

MEDICATION SAFETY DOMAIN

66

Existing approaches: RepresentationBradford-Hill criteria (1965)

1. Strength2. Consistency3. Specificity4. Temporality5. Biological gradient6. Plausibility7. Coherence

Bradford-Hill A. The Environment and Disease: Association or Causation?. Proc R Soc Med. 1965;58:295-300.

67

Existing approaches: Representation

Horn, J. R., Hansten, P. D., & Chan, L. N. (2007). Proposal for a new tool to evaluate drug interaction cases. Annals of Pharmacotherapy, 41(4), 674-680.

68

Existing approaches: RepresentationRoyal Dutch Association for the Advancement of Pharmacy (2005)

1. Existence & quality of evidence on the interaction2. Clinical relevance of the potential adverse

reaction resulting from the interaction3. Risk factors identifying patient, medication or

disease characteristics for which the interaction is of special importance

4. The incidence of the adverse reaction

Van Roon, E.N. et al: Clinical relevance of drug-drug interactions: a structured assessment procedure. Drug Saf. 2005;28(12):1131-9.

69

Existing approaches: Representation

Boyce, DIKB, 2006-present 70

Existing approaches: Acquisition

o Evidence

71Boyce, DIKB, circa 2006

Silos: Multiple sources of information

Post-market studies

Reported in

Scientific literature

Pre-market studies Clinical experience

Drug product labels (US Food and Drug

Administration)

72

Reported in

o What arguments are used in medication safety?

o How can these arguments be mined/identified?

o What work needs to be done?

Why is a new data model needed?o Need computer integrationo Want a COMPUTABLE model that can make

inferences

74