![Page 1: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/1.jpg)
IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID
Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu
![Page 2: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/2.jpg)
Outline
Background The NoteAid System &
Evaluation Results Discussion Conclusion & Future Work
![Page 3: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/3.jpg)
Background
Patients reading their clinical notes has the potential to Enhance medical understanding Improve healthcare management and
outcome
![Page 4: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/4.jpg)
Background
“A patient with hx of active tobacco abuse, bronchitis, and psoriasis presented to ED earlier today with c/o SOB, mild wheezing, chest congestion and chills”
![Page 5: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/5.jpg)
The NoteAid System
NoteAid system - automatically Identifies clinically relevant concepts Links concepts to their definitions
![Page 6: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/6.jpg)
The NoteAid System
The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.
Knowledge Resources
(Medline Plus, UMLS, Wiki)
NLP Approaches
NoteAid
![Page 7: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/7.jpg)
The NoteAid System
The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.
EGD – Acronym for Esophagogastroduodenoscopy.It is a test to examine the lining of the esophagus (the tube that connects your throat to your stomach), stomach, and first part of the small intestine. It is done with a small camera (flexible endoscope) that is inserted down the throat. EGD may be done if you have symptoms that are new, cannot be explained, or are not responding to treatment, such as: Black or tarry stool, Feeling full sooner than normal or after eating less than usual, Swallowing problems or pain with swallowing, heart burns and others.
Knowledge Resources
(Medline Plus, UMLS, Wiki)
NLP Approaches
NoteAid
![Page 8: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/8.jpg)
The NoteAid System
A knowledge resource MedlinePlus UMLS – Unified Medical Language System Wikipedia
NLP approaches
![Page 9: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/9.jpg)
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
![Page 10: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/10.jpg)
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concepts
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
![Page 11: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/11.jpg)
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
![Page 12: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/12.jpg)
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
![Page 13: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/13.jpg)
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
![Page 14: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/14.jpg)
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
![Page 15: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/15.jpg)
NLP Approach
Two system components – Concept Identifier
Process input text and identify clinically relevant concept
Definition LocatorFetch definitions from MedlinePlus, UMLS and
Wikipedia
Input Split
Sentences
Mapped
Phrases
Mapped
Text
KnowledgeLinked Text
SentenceSplitter
ConceptMapper
ConceptFilter
DefinitionFetcher
COMPONENT 1 COMPONENT 2
CONCEPT IDENTIFIER DEFINITION LOCATOR
![Page 16: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/16.jpg)
Evaluation
Four NoteAid implementations MedlinePlus UMLS Wikipedia Hybrid (MedlinePlus+UMLS+Wikipedia)
![Page 17: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/17.jpg)
Evaluation Data
From the Pittsburgh NLP repository 20 Progress Note reports 20 Discharge Summary reports
![Page 18: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/18.jpg)
Subjects
The Amazon Mechanic Turk Has shown to be reliable for medical
annotations and evaluations Recruited 64 subjects
8 systems (4 implementations X 2 types of EHR notes)
8 subjects per system 59 subjects completed the evaluation
![Page 19: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/19.jpg)
Demographic Information
![Page 20: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/20.jpg)
Evaluation Process
Each subject read 20 EHR notes before and after NoteAid implementation Randomly assigned to an implementation Self-reported comprehension on a scale of
1 to 5
![Page 21: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/21.jpg)
Readability
Flesch-Kincaid grade level (Grade Level)
![Page 22: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/22.jpg)
Evaluation Data Statistics
Type Discharge Summaries
Progress Notes
No of Reports 20 20
Total (Avg) # of sentences
355 (17.8) 473 (23.7)
Total (Avg) # of words
2362 (118) 4862 (243)
Avg Flesch-Kincaid Grade
Level
8.8 9.8
![Page 23: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/23.jpg)
Readability and Self-Reported Comprehension
Grade Level correlated with self-reported comprehension score (ρ=-0.47, p<0.05, Spearman rank
correlation)
![Page 24: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/24.jpg)
Results
Average comprehension scores before and after each implementation
*p<0.05, Non-parametric Wilcoxon signed-rank test
System
Discharge Summaries Progress Notes
Before After diff (%) Before After diff (%)
MedlinePlus
3.52 3.49-0.03 (-
0.9)3.18 2.86
-0.32 (-10.1)
UMLS 3.80 3.81 0.01 (0.3) 3.75 4.01 0.26 (7.0)
Wiki 3.57 4.140.57
(16.0)3.45 4.53
1.08* (31.3)
Hybrid
3.86 4.02 0.16 (4.1) 3.40 4.54 1.14 (33.5)
![Page 25: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/25.jpg)
Number of Concepts Identified Total number of concepts that were
recognized by three different NoteAid implementationsSystem Discharge
SummariesProgress Notes
MedlinePlus 37 53
UMLS 171 362
Wiki 190 427
![Page 26: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/26.jpg)
Readability
UMLS Definition – Coagulopathy
“Hemorrhagic and thrombotic disorders that occur as a consequence of abnormalities in blood coagulation due to a variety of factors such as COAGULATION PROTEIN DISORDERS; BLOOD PLATELET DISORDERS; BLOOD
PROTEIN DISORDERS or nutritional conditions” Wiki Definition – Coagulopathy
“Coagulopathy is a condition in which the blood’s ability to clot is impaired. This condition can cause prolonged or excessive bleeding, which may occur spontaneously or following an injury or medical and dental procedures. The normal clotting process depends on the interplay of
various proteins in the blood”
![Page 27: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/27.jpg)
Discussion
Text readability correlated with the comprehension scores
Wiki performed the best Content coverage Readability
![Page 28: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/28.jpg)
Limitations
Limitation Lay people not patients performed evaluation Order-effect bias Score subjects’ EHR note comprehension but
not to what extent they accurately comprehended the EHR note content
![Page 29: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/29.jpg)
Conclusion and Future Work
NoteAid improved EHR note self-reported comprehension
Future Work Improve concept coverage and filtering Evaluate quality of the definitions Evaluate content comprehension Evaluate system in which patient read their
own EHR notes
![Page 30: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/30.jpg)
Acknowledgements
1R01GM095476 University of Massachusetts Medical
School National Center for Advancing
Translational Sciences of the National Institutes of Health under award number UL1TR000161
![Page 31: IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649ea45503460f94ba88a2/html5/thumbnails/31.jpg)
http://clinicalnotesaid.org
Thank You and Questions