kmed: a knowledge-based multimedia medical database system
DESCRIPTION
KMeD: A Knowledge-Based Multimedia Medical Database System. Wesley W. Chu Computer Science Department University of California, Los Angeles http://www.cobase.cs.ucla.edu. KMeD. October 1, 1991 to September 30, 1993. A Knowledge-Based Multimedia Medical Distributed Database System - PowerPoint PPT PresentationTRANSCRIPT
1
KMeD: A Knowledge-Based KMeD: A Knowledge-Based Multimedia Medical Database Multimedia Medical Database
SystemSystem
Wesley W. ChuComputer Science Department
University of California, Los Angeles
http://www.cobase.cs.ucla.edu
2
KMeD
A Knowledge-Based Multimedia Medical Distributed Database System
A Cooperative, Spatial, Evolutionary Medical Database System
Knowledge-Based Image Retrieval with Spatial and Temporal Constructs
Wesley W. Chu Computer Science DepartmentAlfonso F. Cardenas Computer Science DepartmentRicky K. Taira Department of Radiological Sciences
October 1, 1991 to September 30, 1993
July 1, 1993 to June 30, 1997
May 1, 1997 toApril 30, 2001
3
Research Team
StudentsJohn David N.
DionisioChih-Cheng HsuDavid JohnsonChristine Chih
CollaboratorsComputer Science
DepartmentAlfonso F. Cardenas
UCLA Medical SchoolDenise Aberle, MDRobert Lufkin, MDRicky K. Taira, MD
4
A NIH Grant at UCLA (2001-2005)
A Medical Digital library---A Digital File Room for Patient Care, Education, and Research
Wesley W. Chu, PhDWesley W. Chu, PhD
Hooshang Kangarloo, Hooshang Kangarloo, MDMD
Usha Sinha, PhDUsha Sinha, PhD
David B. Johnson, David B. Johnson, PhDPhD
Bernard Churchill, Bernard Churchill, MDMD
5
Significance
Query multimedia data based on image content and spatial predicatesUse domain knowledge to relax and interpret medical queriesPresent integrated view of multiple temporal and evolutionary data in a timeline metaphorRetrieve Scenario Specific Free-text documents in a Medical Digital Library
6
Overview
Image retrieval by feature and contentQuery relaxationSpatial query answeringSimilarity query answeringVisual query interfaceTimeline interfaceRetrieval of scenario specific free text medical documents
7
Image Retrieval by Content
Features size, shape, texture, density, histology
Spatial Relations angle of coverage, shortest distance,
overlapping ratio, contact ratio, relative direction
Evolution of Object Growth fusion, fission
8
9
10
11
12
Characteristics of Medical Queries
MultimediaTemporalEvolutionarySpatialImprecise
13
O O’
01
Om
O O
01
On
Evolution: Object O evolves into a new object O’
Fusion: Object 01, …, Om fuse into a new object
Fission: Object O splits into object 01, …, On
Representing of Temporal and Evolution Objects
14
Case a:
Case c:
The object exists with its supertype or aggregated type.
The life span of the object starts with and ends before its supertype or aggregated type.
Case b:
Case d:
The life span of the object starts after and ends with its supertype or aggregated type.
The life span of the object starts after and ends before its supertype or aggregated type.
Representing of Temporal and Evolution Objects (cont)
15
Lesion
Micro-Lesion
Micro-Lesion
An Example of Temporal and Evolution Object
16
17
18
Spatial Distance and Angle of Coverage of Two Objects
19
20
Query Modification Techniques
Relaxation Generalization Specialization
Association
21
Generalization and Specialization
More Conceptual Query
Specific Query
Conceptual Query Conceptual Query
Specific Query
Generalization
SpecializationGeneralization
Specialization
22
Type Abstraction Hierarchy
Presents abstract view of Types Attribute values Image features Temporal and evolutionary behavior Spatial relationships among objects
Provides multi-level knowledge representation
23
TAH Generation for Numerical Attribute Values
Relaxation Error Difference between the exact value and the
returned approximate value The expected error is weighted by the
probability of occurrence of each value
DISC (Distribution Sensitive Clustering) is based on the attribute values and frequency distribution of the data
24
TAH Generation for Numerical Attribute Values (cont.)
Computation Complexity: O(n2), where n is the number of distinct value in a cluster
DISC performs better than Biggest Cap (value only) or Max Entropy (frequency only) methods
MDISC is developed for multiple attribute TAHs. Computation Complexity: O(mn2), where m is the number of attributes
25
Query Relaxation
RelaxAttribute
Query
Yes
Display
QueryModification
AnswersDatabase
TAHs
No
26
An Cooperative Query Answering Example
Query Find the treatment used for the tumor similar-
to (loc, size) X1 on 12 year-old Korean males.
Relaxed Query Find the treatment used for the tumor Class X
on preteen Asians.
Association The success rate, side effects, and cost of
the treatment.
27
Type Abstraction Hierarchies for Medical Domain
Age
Preteens
910 1112
Teen Adult
Ethnic Group
Asian
Korean Chinese Japanese Filipino
African European
Tumor (location, size)
Class X
[loc1 loc3]
[s1 s3]
Class Y
[locY sY]
X1
[loc1 s1]
X2
[loc2 s2]
X3
[loc3 s3]
28
Knowledge-Based Image Model
Representation Level(features and contents)
Brain TumorLateral
Ventricle
TAHSR(t,b)
TAHTumor Size
TAHSR(t,l)
TAHLateral
Ventricle
SR: Spatial Relationb: Braint: Tumorl: Lateral Ventricle
Knowledge Level
Schema LevelSR(t,b) SR(t,l)
29
Queries
Query Analysis andFeature Selection
Knowledge-BasedContent Matching
Via TAHs
Query Relaxation
Query Answers
Knowledge-based Query Processing
30
User Model
To customize query conditions and knowledge-based query processing
User typeDefault Parameter ValuesFeature and Content Matching Policies Complete Match Partial Match
31
User Model (cont.)
Relaxation Control Policies Relaxation Order Unrelaxable Object Preference List
Measure for Ranking
33
34
Query Preprocessing
Segment and label contours for objects of interestDetermine relevant features and spatial relationships (e.g., location, containment, intersection) of the selected objectsOrganize the features and spatial relationships of objects into a feature databaseClassify the feature database into a Type Abstraction Hierarchy (TAH)
35
Similarity Query Answering
Determine relevant features based on query inputSelect TAH based on these featuresTraverse through the TAH nodes to match all the images with similar features in the databasePresent the images and rank their similarity (e.g., by mean square error)
40
41
Visual Query Language and Interface
Point-click-drag interfaceObjects may be represented iconicallySpatial relationships among objects are represented graphically
42
Visual Query Example
Retrieve brain tumor cases where a tumor is located in the
region as indicated in the picture
43
44
45
46
47
48
49
A Visual Query Example
50
A Visual Temporal Query Example
53
Implementation
Sun Sparc 20 workstations (128 MB RAM, 24-bit frame buffer)Oracle Database Management SystemX/Motif Development Environment, C++Mass Storage of Images (9 GB)
54
55
56
58
Summary I
Image retrieval by feature and contentMatching and relaxation images based on featuresProcessing of queries based on spatial relationships among objectsAnswering of imprecise queriesExpression of queries via visual query languageIntegrated view of temporal multimedia data in a timeline metaphor
59
A Knowledge-based Approach to Retrieve Scenario Specific Free-text in a Medical Digital Library
60
NIH Program Project Grant (2000-2005)
A 5 year $ 10M joint interdisciplinary project between Medical School & CS faculty
Project 1-- teleradaiology infrastructure
Project 2-- neuroradiology workstation
Project 3-- multimedia information architecture
Project 4-- natural language processing for medical reports
Project 5-- medical digital library
61
Project 5 Personnel
Graduate students:Victor Z. LiuWenlei MaoQinghua Zou
Consultants:Hooshang Kangaloo, M.D.Denies Aberle, M.D.
Project leader: Wesley W. Chu
62
Data in a Medical Digital Library
Structured data (patient lab data, demographic data,…)--CoBaseImages (X rays, MRI, CT scans)--KMeDFree-text Patient reports Teaching files Literature News articles
63
System Overview
Patient reports
Medical literature
Medical Digital Library(MDL)
Teaching materials
Query results
Ad-hoc query
Patient report for content correlation
News Articles
64
Treatment-related articles
??? How to treat the disease
Diagnosis-related articles
??? How to diagnose the disease
Scenario Specific Retrieval
…Tissue Source:LUNG (FINE NEEDLE
ASPIRATION) (LEFT LOWER LOBE)
…FINAL DIAGNOSIS:
- LUNG NODULE, LEFT LOWER LOBE (FINE NEEDLE ASPIRATION):- LUNG CANCER, SMALL CELL, STAGE II.
…
65
Challenge I: Indexing
Extracting domain-specific key concepts in the free text for indexing Free-text: Lung cancer, small cell, stage II
Concept terms in knowledge source: stage II small cell lung cancer
Conventional methods use NLP Not scalable Cannot adapt to various forms of word permutation
66
Challenge II: Terms used in the query are too general
Expanding the general terms in the query to specific terms that are used in the document
Query: lung cancer, diagnosis options
Document: … the effectiveness of chest x-ray and bronchography on patients with lung cancer …
?√
Query: lung cancer, chest x-ray, bronchography, …
67
Challenge III: Mismatching between terms used in query and documents
Example
Query: … lung cancer, …
Document 3: anti-cancerdrug combinations…
?? ?Document 1: … lung carcinoma …
Document 2: … lung neoplasm …
81
Application: Query Answering via Templates
Sample templates:“<disease>, treatment,”“<disease>, diagnosis ”
QueryExpansion
…Template:“<disease>, treatment”
lung cancer
lung cancerradiotherapychemotherapycisplatin
relevant documents
IndexFinder
lung cancer,treatment
Phrase-basedVSM
82
Application: Scenario Specific Content Correlation
Query Templates Scenario
Selection
e.g. treatment, diagnosis, etc.
PatientReport
QueryExpansion
…
relevant documents
Phrase-basedVSM
IndexFinder
83
Summary of MDL
Knowledge based (UMLS) approach provides scenario-specific medical free-text retrieval IndexFinder – use word permutation as well as syntactic and
semantic filtering to extract domain-specific key concepts in the free text for indexing
Knowledge-based query expansion – transform general terms in the query into the scenario specific terms used in the documents, giving the query a higher probability of matching with the relevant documents
Phrase based indexing – transform document indexing into phrase paradigm (concept and its word stems) to improve retrieve effectiveness
84
Acknowledgement
This research is supported in part by NIC/NIH Grant#4442511-33780
88
Demo http://fargo.cs.ucla.edu/umls/search.aspx
Test Texts
• Technically successful left lower lobe nodule biopsy.
• Preliminary localization CT images again demonstrate a left lower lobe nodule adjacent to the posterior segmental bronchus.
• CT scans obtained during biopsy demonstrate the coaxial cannula adjacent to the proximal aspect of the nodule.
• Surrounding pulmonary parenchymal hemorrhage as a result of the biopsy is also noted.
• There may be a tiny left apical air collection in the pleural space lateral to the apical bulla.
• Formal cytologic evaluation of the withdrawn specimen is pending at this time, although abnormal appearing "spindle" cells were identified during on-site cytopathologic evaluation of specimen adequacy.
102
107
108
109
110
111
112