![Page 1: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/1.jpg)
STS/IntermacsTop 10 List: Missing Data
Frequencies
Susan Myers, BBA/QMISSTS/IM Database Administrator and Statistical Analyst
How to manage missing data in a registry
![Page 2: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/2.jpg)
• Susan Myers, BBA/QMIS (STS/IM Database Administrator and Statistical Analyst)• Nothing to disclose
![Page 3: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/3.jpg)
Objectives
• Process of Managing Missing Data within a registry database system• Internal validity checks• External validity checks
• Impact of missing data on research/report results
![Page 4: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/4.jpg)
Process for managing an ongoing data collection registry
4
Site Enters Data WBDE
DCCC Raw Data(Data Manager
processes)
DCCC Research Datasets(Data Manager
generates)
DCCC Integrity Checks(Data Manager generates)
Nurse/Auditor: Reviews all inconsistencies – pt specific
DCCC- Reviews database related issues and any administrative inconsistencies
There may be times when the results from the integrity checks will require communication back to the site
Integrity Checks and Feedback Loop of Data
Quality Assurance Site Reports, Data Quality Site Reports, Research
![Page 5: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/5.jpg)
Examples of WBDE Internal Validity Checks
i. Range Checks (e.g. Labs)ii. Chronology Checks (are dates in
appropriate order)iii. Each Form submission requires ALL
variables to be answered
![Page 6: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/6.jpg)
Range checks
RANGE Checks (WBDE internal) Lab value out of range
Internal WBDE range check
Internal Validity Checks
![Page 7: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/7.jpg)
Chronology Checks Date out of range
Internal Validity Checks
DOB > Implant date
![Page 8: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/8.jpg)
Each Form submission requires ALL variables to be answered ‘Complete’ status
Internal Validity Checks
Patient Summary
![Page 9: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/9.jpg)
Additional views in WBDE checking completion of form submission
Internal Validity Checks
Subsections of Follow-up form Pending and in-progress forms on Site Dashboard
![Page 10: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/10.jpg)
Objectives
• Process of Managing Missing Data within a registry database system• Internal validity checks• External validity checks: Examples
• Impact of missing data on research/report results
![Page 11: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/11.jpg)
Process for managing an ongoing data collection registry
11
Site Enters Data WBDE
DCCC Raw Data(Data Manager
processes)
DCCC Research Datasets(Data Manager
generates)
DCCC Integrity Checks(Data Manager generates)
Nurse/Auditor: Reviews all inconsistencies – pt specific
DCCC- Reviews database related issues and any administrative inconsistencies
There may be times when the results from the integrity checks will require communication back to the site
Integrity Checks and Feedback Loop of Data
Data Quality Reports, Quality Assurance Reports, Research
![Page 12: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/12.jpg)
Missing vs Non-Missing valuesExternal Validity Checks
DCCC Research Datasets
![Page 13: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/13.jpg)
External Validity ChecksDCCC Research Datasets
• Missing Device Brand (informs pump flow type; axial, centrifugal etc)• Missing Device Strategy (informs intent; destination vs BTT)• Missing Patient Profile Level (informs how sick is the pt)• Primary COD is missing (if patient is deceased)• Explant reason Transplant but missing Transplant date (endpoint)
Missing vs Non-Missing valuesExamples from the External Checks (200+) run monthly
![Page 14: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/14.jpg)
External Validity Checks
![Page 15: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/15.jpg)
External Validity Checks
Missing STS/Intermacs ENDPOINTS?
When does STS/IM stop or censor a patient?
STS/IM Endpoints• Death• Transplant• Explant for recovery
And why is this important?
![Page 16: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/16.jpg)
External Validity Checks
Missing deaths / transplants / recoveries (endpoints)Site Data Quality Report example
Exhibit 6: Patients on a device greater than 6 yearsThe following table lists patients that have been followed at your hospital for more than 6years with no terminal or censoring events (death, transplant, or device removal for recovery)
Patient ID Implant Date Years Alive Confirmed by Site123456 01/01/2013 6.5098765 06/01/2010 9.0
![Page 17: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/17.jpg)
External Validity ChecksMissing deaths / transplants / recoveries (endpoints)
Comparing a Site to Overall STS/IM (EXAMPLE)How missing data impacts research/report results?
Hospital XOverall STS/IM
![Page 18: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/18.jpg)
External Validity ChecksMissing deaths / transplants / recoveries (endpoints)
Comparing a Site to Overall STS/IM (EXAMPLE)How missing data impacts research/report results?
Hospital Y
Overall STS/IM
0.01
Hospital X
0
![Page 19: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/19.jpg)
Missing vs Non-Missing Follow-up and Adverse Events
External Validity Checks
DCCC Research Datasets
![Page 20: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/20.jpg)
Follow-up Form Completion Rates(from STS/IM QA Report – Sample)
External Validity Checks
Hospital XHospital Y
![Page 21: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/21.jpg)
Follow-up Form Completion Rates: From 2007 to present
% F
ollo
w-u
p co
mpl
ianc
e ra
tes
Years 2007 2018
~60%
> 90%
STS/IM Nurse Monitors came on-board in 2007And implemented a plan to increase follow-up compliance rates
![Page 22: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/22.jpg)
Missing vs Non-Missing Lab values at 3 months Follow-upFor those patients alive with device in place at 3 months – Example of 10,000 pts(pt had opportunity for 3 month f/u)
n Variable N Missing% MissingƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒWhite Blood Cell Count (x10/uL) 9850 150 1.5%Creatinine (mg/dL) 9900 100 1.0%Hemoglobin (g/dL) 9900 100 1.0%Albumin (g/dL) 8640 1360 13.6%Total Bilirubin (mg/dL) 8900 1100 11.0%Mean Right Atrial Pressure > 20% missingSix Minute Walk > 20% missingƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
![Page 23: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/23.jpg)
23
March 2009
May 2012
June 2014June 2006March 2007 October 2016
Hospital Transfer= patient endpoint
March 2006
Evolution of STS/Intermacs Registry
Hospital Transfer ≠ patient endpoint(unless transfer site is non-STS/IM)
![Page 24: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/24.jpg)
Stroke scales (MRS/NIHSS) data capture at time of CVAAnd at follow-up post CVA from June 2, 2014 (v4.0)
Neurological Dysfunction Data Form
June 2014
October 2016
MRS
NIHSS
![Page 25: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/25.jpg)
![Page 26: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/26.jpg)
![Page 27: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/27.jpg)
50%
20%
30%Completed
20%
25%
55%
Time Point Completed 6MW Too Sick to Complete 6MW Missing 6MWPre-implant 20% 55% 25%Follow-up 50% 30% 20%
*% reflect approximate overall rates
6MW Pre-implant and Post implant*
Completed
Pt too Sick to complete
Pt too Sick to complete
Missing
Missing
![Page 28: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/28.jpg)
Possible Resolutions to Missing
![Page 29: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/29.jpg)
There are many reasons and possible resolutions for missing data in STS/IM Registry WBDEJust to name a few:
Reasons• Evolution of WBDE System (versions)• Patient too sick for test to be administered• Data entry Possible Resolutions• External validity checking• Mapping of elements or making agreed upon assignment of values (e.g. Pt ‘too sick’)• Research: time windows for specific studies (MRS: 2014 forward)• Implement processes to help with capture rate
![Page 30: STS/Intermacs Top 10 List: Missing Data Frequencies 1400... · 098765 06/01/2010 9.0 . External Validity Checks. Missing deaths / transplants / recoveries (endpoints) Comparing a](https://reader033.vdocuments.us/reader033/viewer/2022050313/5f754759aa5c19607d1d8dbb/html5/thumbnails/30.jpg)
John K. Pennington, STS/IM Database ManagerMaster of Science in Health Informatics (MSHI)