seer quality audit plan/ proactive quality audit plan pilot study ...€¦ · consolidated...

SEER Quality Audit Plan: Proactive Quality Audit Plan Pilot Study

June 12, 2018

Carmela Groves, RN, MS, CTR

Westat, Inc.

!2

Quality Audit for ER, PR, HER2

▪ Rationale for selection of ER, PR, HER2

▪ No known problems (proactive) ▪ Important indicators for breast cancer ▪ Treatment decisions

▪ Prognostic value

▪ Used in staging in 2018

!3

Methods

!4

Methods

▪ Conducted literature review ▪ Established benchmarks/expected values

▪ Captured historical information on variables ▪ Collected and analyzed data in SEER*Stat ▪ Counts, rates; trends by year, by registry

▪ Compared data in SEER*DMS ▪ Consolidated value/Abstract value

▪ HER2 value/interpretation

!5

Results

▪ Expected values based on literature review ▪ ER positive: 70% ▪ PR positive: 70%

▪ HER2 positive: 15-20%

▪ Factors affecting expected values: Age, Race

!6

Results

ER/PR: Timeline of data collection changes

1: ER collected as Tumor Marker 1, PR as Tumor Marker 2 (EOD) until 2004

2: ER collected as SSF1, PR as SSF2 in CSv1 in 2004 3: Coding notes added in 2007 4: SSF1 and SSF2 with CSv2; add’l notes added in 2010, 2012

1990 2004 2010 2018

1 2

2007

3 4

!7

Results

HER2: Timeline of data collection changes

▪ Introduced with CSv2 in 2010, in SSFs 8-16 ▪ SSF9 + SSF11 + SSF13 + SSF14 = SSF15

▪ SSF15 (HER2: Summary Result of Testing) was not required; derived based on an existing algorithm if not coded

▪ Derived HER2 Recode (2010+)

▪ Updated notes: CSv0204 (2012), CSv0205 (2014)

2010 201820142012

!8

SEER*Stat Results

!9

Female Breast Cancer Cases by Registry, 2014

ER 2014 CasesTotal

# of casesPositive Negative Unknown

Count Percent Count Percent Count PercentRegistry A 3,503 2,910 83% 509 15% 84 2%Registry B 3,248 2,696 83% 481 15% 71 2%Registry C 3,235 2,542 79% 625 19% 68 2%Registry D 1,166 987 85% 163 14% 16 1%Registry E 2,381 1,928 81% 402 17% 51 2%Registry F 1,364 1,057 77% 206 15% 101 7%Registry G 4,043 3,393 84% 556 14% 94 2%Registry H 1,528 1,306 85% 187 12% 35 2%Registry I 2,441 1,926 79% 434 18% 81 3%Registry J 1,723 1,413 82% 230 13% 80 5%Registry K 6,138 4,791 78% 1,036 17% 311 5%Registry L 73 56 77% 15 21% 2 3%Registry M 105 88 84% 13 12% 4 4%Registry N 14,296 11,423 80% 2,125 15% 748 5%Registry O 3,463 2,760 80% 592 17% 111 3%Registry P 3,362 2,517 75% 712 21% 133 4%Registry Q 7,488 5,994 80% 1,123 15% 371 5%Registry R 4,577 3,569 78% 837 18% 171 4%OVERALL 64,134 51,356 80% 10246 16% 2532 4%

Range: 75%-85%

!10


PR 2014 CasesTotal


Count Percent Count Percent Count PercentRegistry A 3,503 2,458 70% 955 27% 90 3%Registry B 3,248 2,339 72% 834 26% 75 2%Registry C 3,235 2,237 69% 925 29% 73 2%Registry D 1,166 863 74% 287 25% 16 1%Registry E 2,381 1,724 72% 604 25% 53 2%Registry F 1,364 985 72% 278 20% 101 7%Registry G 4,043 3,054 76% 882 22% 107 3%Registry H 1,528 1,121 73% 372 24% 35 2%Registry I 2,441 1,615 66% 738 30% 88 4%Registry J 1,723 1,206 70% 432 25% 85 5%Registry K 6,138 4,176 68% 1,631 27% 331 5%Registry L 73 54 74% 17 23% 2 3%Registry M 105 77 73% 24 23% 4 4%Registry N 14,296 9,955 70% 3,557 25% 784 5%Registry O 3,463 2,465 71% 886 26% 112 3%Registry P 3,362 2,193 65% 1,027 31% 142 4%Registry Q 7,488 5,388 72% 1,711 23% 389 5%Registry R 4,577 3,067 67% 1,329 29% 181 4%OVERALL 64,134 44,977 70% 16489 26% 2668 4%

Range: 65%-76%

!11


HER2 2014 CasesTotal


Count Percent Count Percent Count PercentRegistry A 3,503 526 15% 2840 81% 137 4%Registry B 3,248 503 15% 2611 80% 134 4%Registry C 3,235 533 16% 2581 80% 121 4%Registry D 1,166 156 13% 988 85% 22 2%Registry E 2,381 385 16% 1919 81% 77 3%Registry F 1,364 233 17% 999 73% 132 10%Registry G 4,043 617 15% 3270 81% 156 4%Registry H 1,528 269 18% 1197 78% 62 4%Registry I 2,441 456 19% 1860 76% 125 5%Registry J 1,723 268 16% 1352 78% 103 6%Registry K 6,138 1106 18% 4625 75% 407 7%Registry L 73 21 29% 50 68% 2 3%Registry M 105 18 17% 81 77% 6 6%Registry N 14,296 2381 17% 10934 76% 981 7%Registry O 3,463 594 17% 2674 77% 195 6%Registry P 3,362 601 18% 2561 76% 200 6%Registry Q 7,488 1288 17% 5664 76% 536 7%Registry R 4,577 849 19% 3487 76% 241 5%OVERALL 64,134 10804 17% 49693 77% 3637 6%

Range: 13%-29%

!12

Female Breast Cancer Cases by ER/PR/HER2 Status and Age, 2014

2014 Cases

# of casesPercent positive

Percent negative

Percent unknown

ER<50 12,235 76% 21% 3%50-64 23,919 80% 17% 3%65+ 27,980 82% 13% 5%PR<50 12,235 69% 27% 3%50-64 23,919 69% 28% 3%65+ 27,980 72% 23% 5%

HER2<50 12,235 22% 74% 5%50-64 23,919 19% 76% 5%65+ 27,980 13% 80% 7%

!13

Female Breast Cancer Cases by ER/PR/HER2 Status and Race, 20142014 Cases

# of casesPercent positive

Percent negative

Percent unknown

ERWhite 50,265 82% 14% 4%Black 7,241 69% 27% 4%American Indian/Alaska Native 411 79% 15% 6%Asian or Pacific Islander 5,491 81% 16% 3%PRWhite 50,265 72% 24% 4%Black 7,241 57% 38% 4%American Indian/Alaska Native 411 68% 25% 7%Asian or Pacific Islander 5,491 71% 26% 3%HER2White 50,265 16% 78% 5%Black 7,241 19% 75% 6%American Indian/Alaska Native 411 21% 70% 9%Asian or Pacific Islander 5,491 20% 76% 5%

!14

ER, Percent Positive by Registry, 2000-2014

Perc

ent P

ositi

ve

0%

25%

50%

75%

100%

Year2000 2004 2008 2011 2015

Registry ARegistry BRegistry CRegistry DRegistry ERegistry FRegistry GRegistry HRegistry IRegistry JRegistry KRegistry LRegistry MRegistry NRegistry ORegistry PRegistry QRegistry R

Benchmark

!15

ER Positive, Age Adjusted Rates per 100,0000 by Registry, 2000-2014R

ate

per 1

00,0

00

0

32.5

65

97.5

130

Year2000 2004 2008 2011 2015


!16

ER Negative, Age Adjusted Rates per 100,0000 by Registry, 2000-2014R

ate

per 1

00,0

00

0

32.5

65

97.5

130

Year2000 2004 2008 2011 2015


!17

ER Unknown, Age Adjusted Rates per 100,0000 by Registry, 2000-2014

Rat

e pe

r 100

,000

0

12.8

25.5

38.3

51

Year2000 2004 2008 2011 2015


!18

Female Breast Cancer Age-Adjusted Rates per 100,000 with Unknown ER, 2004-2014

Rat

e pe

r 100

,000

0

2

4

6

8

Year2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014

Test ordered, results not interpretable Test ordered, results not in chart Test not done (test not ordered and not performed)Unknown or no information

!19

PR, Percent Positive by Registry, 2000-2014

Perc

ent P

ositi

ve

0%

25%

50%

75%

100%

Year2000 2004 2008 2011 2015


Benchmark

!20

HER2, Percent Positive by Registry, 2010-2014

Perc

ent P

ositi

ve

0%

25%

50%

75%

100%

Year2010 2011 2012 2013 2015


Benchmark Range

!21

HER2 Positive Age Adjusted Rates per 100,0000 by Registry, 2010-2014

Rat

e pe

r 100

,000

0

35

70

105

140

Year2010 2011 2012 2013 2015


!22

SEER*DMS Results

!23

Comparison of Consolidated/Abstract Values, 2014

Match Status ER—Percent PR—Percent

CTC/Abstract Match 98.07 97.06

CTC +/Abstract - 0.09 0.13

CTC -/Abstract + 0.50 1.03

CTC not done/Abstract results 0.12 0.14

CTC not done/Abstract unknown 0.05 0.05

CTC unknown/Abstract results 0.30 0.32

CTC known/Abstract not done 0.09 0.11

CTC known/Abstract unknown 0.38 0.37

CTC unknown/Abstract not done 0.41 0.79

!24

IHC interpretation (SSF 9)

Positive Negative Borderline

IHC Lab Value (SSF8) 010 020 030 988-999 Percent

Discrepant

Score 1+ 010 104 21,357 210 25 2%

Score 2+ 020 285 552 10,543 43 8%

Score 3+ 030 5,943 82 109 12 3%

988-999 232 1,801 251 10,400 18%

Total Discrepant 7%

Comparison of SSF8 IHC Lab Values and SSF9 IHC Interpretation used in HER2 Analysis for Breast Cancer

!25

HER2: SSF15 Summary Result of Testing

▪ SSF9 used in deriving HER2 Summary ▪ Complicated algorithms/rules exist for HER2 ▪ CS v2, SSF15, Note 3: If the results of one test are

available, and it is known that a second test is performed but the results are not available, use code 997 ▪ Algorithm for Derived HER2 Summary, Rule 1:

If FCO_comb=negative, then derived HER2=negative (regardless of what IHC test results indicate)

!26

Summary/Conclusions

▪ No major inconsistencies or problems identified with ER, PR, HER2; minor variance with unknowns ▪Missing/unknown data has improved ▪ Increasing trend in percent ER positive ▪ Benchmarks: identification necessary in the process ▪ Considerations for database modifications: e.g., add data

entry warnings, re-evaluate algorithms; SSDIs; training

!27

Proactive Quality Audit Workgroup Members

▪ Kathy Cronin, content expert

▪ Linda Coyle

▪ Carmela Groves

▪ Serban Negoita

▪ Jennifer Stevens

▪ Marina Matatova

Thank YouAny questions?

seer quality audit plan/ proactive quality audit plan pilot study ...€¦ · consolidated...

Documents