1 question answering in biomedicine student: andreea tutos id: 41064739 supervisor: diego molla
Post on 19-Dec-2015
214 views
TRANSCRIPT
2
Project outline
Why medical question answering?
Current research
Project methodology
Project outcomes
3
Why Question Answering?
Thousands of new biological and medical research articles published daily world wide
66% of physicians report the volume of medical information as unmanageable (Craig et al, 2001).
The main impediment in maximizing the utility of research data: insufficient time
5
What is Question Answering?
The task of automatically finding an answer to a question
Relies on analyzing large collections of documents
Aims to provide short and concise answers rather than a list of relevant documents
6
Key steps to follow
Select the domain knowledge source
Construct the corpus of questions
Analyze the input question– Classify the question– Construct the search query
Extract the answer
7
Domain knowledge sources
Reliability of medical information is critical (NetScoring)
MEDLINE - medical repository maintained by the US National Library of Medicine (controlled vocabulary thesaurus MeSH)
8
Key steps to follow
Select the domain knowledge source
Construct the corpus of questions
Analyze the input question– Classify the question– Construct the search query
Extract the answer
9
Question corpus sources
Question sources we have reviewed in our research:
– The Parkhurst Exchange website
– The Clinical Questions Collection website
– The Journal of Family Practice website
10
Questions format
Natural language question:“In children with an acute febrile illness, what is the efficacy of single-medication therapy with acetaminophen or ibuprofen in reducing fever?”
PICO format question:“Problem/Population: acute febrile illness / in children Intervention: acetaminophen Comparison: ibuprofen Outcome: reducing fever “
(Demner - Fushman and Lin, 2007 )
11
Key steps to follow
Select the domain knowledge source
Construct the corpus of questions
Analyze the input question– Classify the question– Construct the search query
Extract the answer
13
Query analysis
Processes included:
– Keyword selection: extract keywords using parsers such as LTCHUNK identify named entities with the support of UMLS
– Answer pattern generation (different combinations of query terms)
(Molla and Vicedo, 2009)
14
Key steps to follow
Select the domain knowledge source
Construct the corpus of questions
Analyze the input question– Classify the question– Construct the search query
Extract the answer
15
Answer extraction
Identify relevant sentences that answer the question
Rank the answer candidates (popularity, similarity with the question, answer patterns, answer validation) (Molla and Vicedo, 2009)
Could use the IMRAD (Introduction, Methods, Results and Discussion) structure of biomedical articles (MedQA)
16
Search engines and question answering systems
Generic:
– Google– Answers.com– OneLook
Medical:
– PubMed– MedQA– Google on PubMed only
18
Project methodology - Question corpus
We have sourced 50 clinical questions and their answers from the Parkhurst Exchange web site
Question Category
Is watermelon allergenic No Intervention
When to introduce solids to infants Intervention
Should family doctors be immunized with Pneumovax and Menactra or Menjugate Intervention
Can cell phones cause cancer No InterventionHow much folic acid — 400 μg, 1 mg, 5 mg — is recommended before conception and during pregnancy Intervention
How to beat recurrent UTIs Intervention
How to recognize autism in adults No Intervention
Does skin colour affect vitamin D requirements No Intervention
19
Project methodology - Question processing
We have defined five levels of processing to be applied to improve search outcomes. Processing Level Description
Original Question/Term
Processed Question/Term
1
introduce synonyms/hypernyms infectious bacterial
2replace abbreviations BP blood pressure
3
Introduce general medical terms
What is shoulder frozen
What is shoulder frozen syndrome
4eliminate additional terms
Are there any contraindications to dental office visits in pregnancy
Dental office visits in pregnancy
5express medical context
What is the evidence that antibiotics change the course of the disease in infectious conjunctivitis
Are antibiotics recommended for bacterial conjunctivitis
20
Project methodology – Scoring system
We have used a scoring system first referred to in the Text Retrieval Conference (TREC), called Mean Reciprocal Rank (MRR)
(Voorhees, 2001)
A relevant link returned in nth position (n<= 10) received a score of 1/n
21
Results – No Intervention questions
No Intervention questions average scores
No Intervention questions
(strict evaluation)
0.27
0.04
0.38
0.04
0.80
0.41
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Average of
PubMed
Average of
OneLook
Average of
Answers.com
Average of
MedQA
Average of
Average of
Google On
Pubmed
22
Results – Intervention questions
Intervention questions average scores
INTERVENTION questions
(strict evaluation)
0.24
0.040.10
0.04
0.54
0.35
0.00
0.10
0.20
0.30
0.40
0.50
0.60
Average of
PubMed
Average of
OneLook
Average of
Answers.com
Average of
MedQA
Average of
Average of
Google On
Pubmed
23
Results – Answer location
Answer location in scientific articles
Answer location
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
Search Engine
Intervention
Non Intervention
Total
25
Medical search engines and QA systems
conclusions
Pubmed obtained similar scores for both categories (0.27 for No Intervention and 0.24 for Intervention questions)
Medical search engines perform relatively equal on Intervention and No Intervention questions
26
Generic search engines and QA systems
conclusions
Google recorded the best performance for both categories of questions
Both Google and Answers.com scored better results on No Intervention questions than on Intervention questions
Non-medical oriented search engines have more difficulties in producing answers for scenario-based, complex medical questions.
27
Conclusions
All selected questions are answerable with the current technology
50% of answers are located in the Abstract section of scientific articles; 25% in the Conclusions section
No Intervention questions are easier to answer than Intervention questions when it comes to generic search technology