BigMouth:Amulti-institutionaldental

datarepository

Muhammad F Walji, PhD

Outline

• DemonstrationofBigMouth• DataGovernance(howtocontributeandaccess)

• DataStandardizationinDentistry• ExamplesofResearchProjectsusingBigMouth• DentalQualityMeasurement

BigMouth• BigMouthcontainsdatafromsixdentalschools• BigMouthallowsnon-technicaluserstoquerytheaxiUm data• Dataavailable- Demographics,Diagnosis,Procedures,Forms,

Odontogram,Perio,Medication,Insurance,Practice

HarvardaxiUm

UCSFaxiUm

TuftsaxiUm

UPitt axiUm

UMichiganaxiUm

Extractscripts

Extractscripts

Extractscripts

Extractscripts

Extractscripts

SFTPserver

Loadscripts

UTNetwork

Extractedfiles

UTSDaxiUm Extractscripts

ElectronicHealthRecords(EHRs)

• Increasedadoptionindentistry

• Largeamountandvarietyofdata

• MajorityofUSDentalSchoolsusesameplatform

• Unstructureddata

• DatacollectedforTreatment,Payment,Operations

• LimitedstandardizationNodiagnosticterminologyuntilrecently

DatainaxiUm• PatientInformation– Demographics,vitals,height,weight,zipcodeetc.

• Dentalinformation– Treatments,Diagnoses,DentalHistoryetc.

• Medicalinformation– MedicalHistory• Medication– Currentmedication,Prescriptionetc.• Toothlevelinformation– Odontogram,DMFS/DMFT• PeriodontalChart• Educational– StudentsandResidents,Grades• Financial– Billing,Insurance

BenefitsofusingEHRdataforresearch

• Lowercostandquickerthantraditionalclinicalresearch1

• Patientsratherthanresearchsubjects1

• Datareadilyavailable• Potentialofusingtheseresource

to generate abstracts/proposals• Opportunitytodevelopscholarlyportfolio

1MeiSong,Kaihong Liu,RebeccaAbromitis,TitusL.Schleyer,Reusingelectronicpatientdatafordentalclinicalresearch:Areviewofcurrentstatus,JournalofDentistry,Volume41,Issue12,December2013,Pages1148-1163

ChallengesofusingEHRdataforresearch

• Clinicalnotes(freetext)arenotreadilyqueryable1

• Dataqualityissues1– Incomplete,Missing(e.g.Ethnicity)– Inaccurate(e.g.selfreporteddata)– Inconsistent(Codingissues)

• EthicalIssues(e.g.PatientPrivacy)• Datamaybeidiosyncraticandmaynotbegeneralizable1

1MeiSong,Kaihong Liu,RebeccaAbromitis,TitusL.Schleyer,Reusingelectronicpatientdatafordentalclinicalresearch:Areviewofcurrentstatus,JournalofDentistry,Volume41,Issue12,December2013,Pages1148-1163

8

https://bigmouth.uth.edu/

DatainBigMouth#ofpatients

#ofpatientsDemographics 2,178,816Diagnosis 301,970Procedures 1,223,806Forms 462,001Odontogram 1,903,625PeriodontalChart 227,369PrescribedMedication 141,613Insurance 444,727

Geographical distribution of Patients in BigMouth

LinkingMedicalandDentalRecordsUsinganeMPI

TheEnterprise-wideMasterPatientIndex:

• Adatabaseofallpatientsacrossanorganization,eachassignedauniqueID.

• Matchespatientsacrosssystemsusing“essentialdata”:– Name,Gender,DateofBirth,RaceandEthnicity,SocialSecurityNumber,CurrentaddressandContactinformation

• Necessaryforthemaintenanceofaccuratedemographicandclinicaldata

Joffe,E.,Byrne,M.J.,Reeder,P.,Herskovic,J.R.,Johnson,C.W.,McCoy,A.B.,...&Bernstam,E.V.(2014).Abenchmarkcomparisonofdeterministicandprobabilisticmethodsfordefiningmanualreviewdatasetsinduplicaterecordsreconciliation. JournaloftheAmericanMedicalInformaticsAssociation, 21(1),97-104.McCoy,A.B.,Wright,A.,Kahn,M.G.,Shapiro,J.S.,Bernstam,E.V.,&Sittig,D.F.(2013).Matchingidentifiersinelectronichealthrecords:implicationsforduplicaterecordsandpatientsafety. BMJquality&safety,22(3),219-224.

DataAvailability

UTP:AllScripts - 1,234,423

patients

SOD:axiUm-327,175

Patientoverlap44,188patients

Medical

Dental

Asof01/17/2017

DataGovernance

1. Adheretoprivacyandsecurityrequirements

2. Accessprovidedtothoseinstitutionsthatcontribute

3. Eachsourcesiteretainscontrol/ownershipofcontributeddata

4. Projectreviewcommitteeapprovalrequiredforuseofdataforspecificresearchprojects

5. Continuouslyassessandimprovethequalityofdata

15

DataGovernance2.0

1. Adheretoprivacyandsecurityrequirements

2. Accessprovidedtoresearchersfromnon-profitinstitutions

3. Eachsourcesiteretainscontrol/ownershipofcontributeddata

4. Projectreviewcommitteeapprovalrequiredforuseofdataforspecificresearchprojects

5. Continuouslyassessandimprovethequalityofdata

16

HowtoAccessBigMouth• ContributingInstitution

– Signupathttps://bigmouth.uth.edu/

• Level1Access– ExplorethedatausingtheBigMouthWorkbench

• Level2Access– SubmitproposaltoResearchReviewCommittee

• IRBApproval– Receivedatasettoconductanalysis

• Fillouttheresearchdatarequesttemplate

HowtobecomeaContributingInstitution

• COHRImemberinstitutionseligible

• Completeadatauseagreement

• IRBapproval

www.cohri.org

VISIONTobeleadingexpertsininformaticsfororalhealth

research,education,andpatientcare

MISSIONCOHRIwill:Ø Create,standardize,integrateandsharedatausingelectronic

healthrecords

Ø Improveinformaticsutilizationindentaleducation,healthcare,andresearch

Ø Workcollaborativelytodevelopresearchprojectstopromoteevidencebaseddentistry

Current Member Institutions (n = 40)AcademicCentreforDentistry(ACTA)BaylorUniversityColumbiaUniversityCreightonUniversityHarvardIndianaUniversityLECOM BradentonLomaLindaMedicalUniversityofSouthCarolinaMeharry MedicalCollegeMidwesternUniversityDownersGroveNewYorkUniversityNovaSoutheasternUniversityOregonHealth&ScienceUniversitySouthernIllinoisUniversityTempleUniversityTuftsUniversityUniversityofCalifornia,SanFranciscoUniversityofColoradoUniversityofFlorida

UniversityofIllinoisUniversityofKentuckyUniversityofLouisvilleUniversityofMichiganUniversityofMinnesotaUniversityofNewEnglandUniversityofNevadaLasVegasUniversityofOklahomaUniversityofPacificUniversityofPuertoRicoUniversityofPittsburghUniversityofSharjah,UAEUniversityofSouthernCaliforniaUniversityofTennesseeUniversityofTexas, HoustonUniversityofTexas, SanAntonioUniversityofWashingtonVirginiaCommonwealthUniversityWestVirginiaUniversityWillametteDentalGroup

COHRI Resources

1. Adult medical and dental histories

2. Pediatric medical and dental histories

3. Problem list

4. Dental Diagnostic System (DDS)

5. Caries risk assessment (long + short)

6. BigMouth Dental Data Repository

Interface Terminology

So many terms

SNOMED CT

SNODENTDental Diagnostic System

SNODDS

SNODDSGD

ANSIApprovedStandard

ExamplesofResearchProjects

TheValueofUsingaDentalDiagnosticTerminologyinanElectronicHealthRecord– Kalenderian etal.

• Goal:FindpatientsdiagnosedwithmoderatechronicperiodontitiswhoreceivedtreatmentaccordingtocurrentAmericanAcademyofPeriodontology(AAP)guidelines.

• Dataused:DiagnosisandProcedures

Kalenderian E,Tokede B,Ramoni R,KhanM,Kimmes N,WhiteJ,Vaderhobli R,YansaneA,Feilzer A,WaljiMF.Dentalclinicalresearch:anillustrationofthevalueofstandardizeddiagnosticterms.JournalofPublicHealthDentistry.2015.Oct30.doi:10.1111/jphd.12124.

Efficacyandaccuracyofpre-doctoralperiodontaleducationintheUS:Anevaluationofthegeneralistandspecialistteachingmodelsonperiodontaleducation– AliSajadi,DDS

• Goal:Tounderstandtheaccuracyofdiagnosisbyanalyzingclinicalchartingparametersanddiagnosticcodesbyfinalyearstudents

• Dataused:Diagnosis

Partial-mouthperiodontalexaminationprotocolforestimationofprevalence,severityandextentofperiodontitis– Tranetal.

• Goal:Thisprojectaimstoidentifyapartial-mouthperiodontal;examinationprotocolwhichcloselyapproximatesafull-mouthperiodontalexaminationprotocolinestimatingprevalence,severityandextentofperiodontitis inpopulation

• Dataused:Perio,Forms

Associationbetweenobesityandperiodontitis:Cross-sectionalstudyonpatientsvisitinguniversities'dentalclinics- Tranetal.

• Goal:Evaluateperio initialexamofobesepatients.• Dataused:axiUm (Perio,medicalanddentalhistoryand

Missingteeth)andAllScripts (Diagnosis,medicalhistoryetc.)

StatinUseisInverselyCorrelatedwithSymptomaticEndodonticDiagnoses– Kanwal etal.

• Goal:Thepurposeofthisretrospectivestudywastoverifyifcorrelationsexistbetweenendodonticdiagnosesandstatinuse.

• Dataused:Forms,Procedure

AssessmentofDiagnosedTemporomandibularDisordersandOrofacial PainConditionsbyPre-doctoralDentalStudents– Adibi etal.

• Goal:Thepurposeofthisstudywastoassessthedegreetowhich1)patientsreportsignsandsymptomsofTMD/OFP,2)ifpre-doctoraldentalstudentdiagnoseTMD/OFP,and3)ifpatientsreceiveanytreatmentforTMD/OFP.

• Data:Forms,Procedures,Diagnosis,Provider

Adibi SS,Kookal KK,Fishbeck NM,ThompsonCR,WaljiMF.AssessmentofDiagnosedTemporomandibularDisordersandOrofacialPainConditionsbyPredoctoral DentalStudents:APilotStudy.JDentEduc 201680:1450-1456.

ImplementingDentalQualityMeasuresinPractice

Ø R01,5-YearNIH/NIDCRGrantØ UTHouston,HSDM,UCSF,WDGØMarch012015– February282020

1. Assessdentalqualitymeasureimplementationfeasibilityandvalidityacross4coresites.

2. Implementvaliddentalqualitymeasuresacrossall11sitesandassessinter-sitevariation.

3. DevelopusefulandusableDentalQualityMeasuredashboardviewsandfunctionalitytosupporttheinterpretationofdentalqualitymeasures

WhatisQuality?

“Thedegreetowhichhealthservicesforindividualsandpopulationsincreasethelikelihoodofdesiredhealthoutcomesandareconsistentwithcurrentprofessionalknowledge.”

IOM

https://www.nationalacademies.org/hmd/Global/News%20Announcements/Crossing-the-Quality-Chasm-The-IOM-Health-Care-Quality-Initiative.aspx

WhatareClinicalQualityMeasures?

• Clinicalqualitymeasures,aretoolsthathelpmeasureandtrackthequalityofhealthcareservicesprovided

• CQMMeasure:– healthoutcomes– clinicalprocesses– patientsafety– efficientuseofhealthcareresources– carecoordination– patientengagements– populationandpublichealth– adherencetoclinicalguidelines

MeasuredomainsClinicalqualitymeasures:

ProcessPercentageofelevatedcariesriskpatientswhowereprescribedahighconcentrationfluoridetoothpaste

Access Percentageofelevatedcariesriskpatientswhovisitedadentalclinic

OutcomesPercentageofelevatedcariesriskpatientswhoremaineddiseasefreeinthepastyear

StructureDoesthehealthcarefacilityusesEHRs?

PatientSatisfactionPatientreportsthedentistcommunicatedwellwithhim/her

https://www.qualitymeasures.ahrq.gov/about/domain-definitions.aspx

TenDentalQualityMeasures1. DiabetesandAnnualOralEvaluation2. Sealants3. TobaccoScreeningandCessationCounseling

Intervention4. CariesRisk5. CariesTreatment6. CariesOutcomes7. PeriodontalRisk8. PeriodontalTreatment9. PeriodontalOutcomes10. PregnancyandOralEvaluation

Whatpercentageofdiabeticsreceiveanannualoralexamination?

NUM: Numberofdiabeticswhoreceivedthefollowingdentalexaminationprocedures:D0150,D0120&D0180inthemeasurementperiod

DEN: Dentalpatients,whoreportedhavingdiabetes,ofrecordinthedentalclinicinthereportingyear

MeasureScore

%

Claimsdataà EHR

ApproachforEvaluatingDQM1. DevelopmentofEHRquery

2. Calculatesamplesizesneededtoreview

3. 2Reviewersindependentlyreviewed50charts

4. Inter-raterreliability

5. Singlereviewofremainingcharts

6. Calculationofsensitivity,specificity,NPV,PPVandKappa

7. Identificationofwhypatientsdidnotreceiveoralevaluation

8. Developandevaluateimprovedmeasure

InitialEHR-Measureresults:

NUM: Howmanyreceivedthefollowingprocedures:D0150,D0120&D0180inthemeasurementperiod

DEN: Dentalpatientsofrecord,whoreportedhavingdiabetes,inthedentalclinicinthereportingyear

MeasureScore

60.3%*MeasureScore

71.7%**

*ManualReview**Automatedquery

Initial EHR measure

Overall Score

ManualReview(n= 1,242)

Measure rate:60.3%(95% CI; 58-63%)

P<0.0011

AutomatedQuery(n=12,960)

Measure rate:71.7%(95% CI; 71-72%)

Sensitivity 93% (95% CI; 90.7-94.5) of the time thatmanualreviewrevealedOralEvaluation,thequery

identifiedOralEvaluationSpecificity 100% (95% CI; 100-100) of the time that the

manual review did not reveal Oral Evaluation, the query did not identify Oral Evaluation

Positive Predictive Value

100% (95% CI; 100-100) of the time that the query identified Oral Evaluation, the manual

review revealed Oral EvaluationNegative Predictive Value

100% (95% CI; 100-100) of the time that the query did not identify oral evaluation, the manual

review did not reveal oral evaluation

WhydidDiabeticsNotReceiveanOralEvaluation?

• Group1:Reasonsthatjustifiedthepatientbeingexcludedfromthedenominator

• Theselectionofeligiblepatientsintheinitialmeasuredidnotfollowanexclusioncriteria

Group Category– Reasons forExclusion

1 Patientswerediscontinued2 Edentulous

3 Patientreportedforlimited/emergency/urgent./specialistcareonly

4 Lessthanayearofvisit5 Miscoded

WhydidDiabeticsNotReceiveanOralEvaluation?

• Group2:Reasonsthatjustifiedthepatientbeingincludedinthenumerator

• ToincludeonlycomprehensiveperiodontalevaluationORcomprehensiveoralevaluationORperiodicoralevaluationmayunderestimatethecarepatientsreceived

Category– ReasonsforInclusionOtheracceptablePeriodontalcare:Adult Prophylaxis(D1110)Periodontalmaintenance(D4910)Fullmouthdebridement(D4355)Scalingandrootplaningofonetothreeteeth(D4342)Scalingandrootplaningoffourormoreteeth(D4910)Periodontal ChartCompleted orInProgressDentalProceduresunderPlanned andinconjunctionwiththeDDSterminology

Figure 1. Numerator and Denominator workflow for Initial and Revised EHR-Measures:

RevisedEHR-qualityMeasurescores:

NUM

DEN

MeasureScore

82.3%*MeasureScore

83.9%**

*Manualreview**Automatedquery

Initial EHR measure Revised EHR measure

Overall Score

ManualReview(n= 1,242)

Measure rate:60.3%(95% CI; 58-63%)

P<0.0011 ManualReview(n= 710)

Measure rate:82.3% (95% CI; 79-85%)

P =0.2601

AutomatedQuery(n=12,960)

Measure rate:71.7%(95% CI; 71-72%)

AutomatedQuery(n= 13,221)

Measure rate:83.9% (95% CI; 83-85%)

Sensitivity 93% (95% CI; 90.7-94.5) 99% (95% CI; 98.7-100) P <0.001

Specificity 100 (95% CI; 100-100) 90% (95% CI; 85.0-95.2) P <0.001

Positive Predictive Value

100 (95% CI; 100-100) 98% (95% CI; 96.5-99.0) P <0.001

Negative Predictive Value

100 (95% CI; 100-100) 98% (95% CI; 94.8-100) P <0.001

Measure rates for the Initial and Revised EHR-Measures across four sitesfor both automated query and manual reviews.

1 – Relationshipbetweenmanualandautomatedqueriescalculatedwithtwosampletestofproportions2- Relationshipbetweenofthevalidationanalysisbetweentheadaptedandrevisedmeasurescalculatedwithtwosampletestofproportions

46

NextSteps

• Scalability

• Standardization

• Sustainability

• LinkingtoMedicalData

• LinkingtoGenomicData

BigMouthTeamUT-Houston• MuhammadWaljiPhD• KrishnaKookal M.S• JosephApplegate• SusanGuerrero

Tufts• BrittaMagnuson,DMD• MattJohnston

UCSF

Harvard

• Elsbeth Kalenderian DDS,MPHPhD• JoelWhiteDDSMS• BingEspiritu

• GermanGallucci DMD,PhD• ScottJason

Univ of Pittsburg• PatrickHetherington

Univ of Michigan• WillyWangsa

DQMTeamUT-HoustonMWalji(PI)ANeumannKKookalSKumarSBangarCVanStrienJModha

UCSFEKalenderian(PI)JWhiteEni ObadanAYansaneRVaderhobliBMertz

Harvard

BTokede

J Etolue

WillametteDentalGroup

KSimmonsJEvensJMullinsEBlagaTGuy

AcknowledgementsG08LM010075 (NLM):DevelopmentofanInter UniversityOralHealthResearchDatabase

R01DE021051(NIDCR):ACognitiveApproachToRefineAndEnhanceUseOfADentalDiagnosticTerminology

R01DE023061 (NIDCR):AWholeSystemsApproachToImplementingStandardizedDentalDiagnosticTerms

R01DE022628 (NIDCR):DevelopingaPatientSafetySystemforDentistry

R01DE024166 (NIDCR):ImplementingDentalQualityMeasuresinPractice

• Thankyoutoourresearchteammembers!

Muhammad.F.Walji@uth.tmc.edu

Questions?

Muhammad.F.Walji@uth.tmc.edu