funnel plots: visualizing small area estimates - sas group presentation… · funnel plots:...
TRANSCRIPT
Funnel Plots: Visualizing Small Area Estimates
eSUG May 2010
Doug Dover Surveillance and Assessment Branch
Community and Population Health Division Alberta Health and Wellness
Funnel Plots: Visualizing Small Area Estimates
eSUG May 2010
Doug Dover Surveillance and Assessment Branch
Community and Population Health Division Alberta Health and Wellness
Funnel Plots
The need to visualize small area rates. The need to visualize small area rates.
Small Area Rates
Issues:
•large variability in small populations
•ranking and rates •ranking and rates
•identification of anomalies
Small Area Rates
large variability in small populations
ranking and rates ranking and rates
identification of anomalies
Funnel Plots
Developed as alternatives to caterpillar or league plots.
History
Developed as alternatives to caterpillar or league plots.
Funnel Plots
Funnel plots show point estimates for small areas or domains against a background “expected” confidence interval. interval.
Funnel plots show point estimates for small areas or domains against a background “expected” confidence
Funnel Plots Expected Standard Errors
The “expected” standard error in the binomial case is given by
ˆ ˆ (1 ) / p p n − where is the overall rate
and n varies freely.
ˆ ˆ (1 ) / p p n − ˆ p
Expected Standard Errors
The “expected” standard error in the binomial case is given by
ˆ ˆ (1 ) / p p n − where is the overall rate
varies freely.
ˆ ˆ (1 ) / p p n −
Funnel Plots Expected Standard Errors Expected Standard Errors
Funnel Plots Expected Standard Errors
small n, large CI
Expected Standard Errors
, large CI
Funnel Plots Expected Standard Errors
small n, large CI
Expected Standard Errors
, large CI
large n, small CI
Funnel Plots Expected Standard Errors
small n, large CI
Expected Standard Errors
, large CI Limits centered at a fixed rate
large n, small CI
Funnel Plots Nice Example
Funnel Plots Nice Example
Same disease,
a few years later…
Funnel Plots in SAS
The funnel plots presented so far were created in SAS, but why not just use Excel for such simple graphs?
The funnel plots presented so far were created in SAS, but why not just use Excel for such simple graphs?
Funnel Plots in SAS
Because SAS is cool! Because SAS is cool!
Funnel Plots Which point?? Which point??
Funnel Plots Mouse over!
Funnel Plots SAS mouse over
*make funnel plot*;
goptions reset=symbol device=activex; ods html body="&htmlfile" path="&htmlpath" parameters=("ZoomControlEnabled"="false" "DataTipStyle"="stick_fixed");
Proc GPlot Data=_Funnel_; Plot &rate*&pop lfunnelLimit*&pop ufunnelLimit*&pop
/ overlay / overlay hminor = 0 vminor = 0 vaxis = axis1 vref=&ABvalue cvref=purple lvref=3 frame anno=anno
html=popup ;
format &pop comma12.; run; quit; ods html close;
SAS mouse over
device=activex; ods html body="&htmlfile" path="&htmlpath" parameters=("ZoomControlEnabled"="false" "DataTipStyle"="stick_fixed");
Plot &rate*&pop lfunnelLimit*&pop ufunnelLimit*&pop
vref=&ABvalue cvref=purple lvref=3
Funnel Plots SAS mouse over
*make popup text*; Data _Funnel_; Set _Funnel_; popup='ALT="'||compress(&PopupText)||'"';
run;
SAS mouse over
popup='ALT="'||compress(&PopupText)||'"';
Funnel Plots Are there commonalities? Are there commonalities?
Funnel Plots Text!
Funnel Plots Colour!
Funnel Plots SAS annotate
*make funnel plot*; goptions reset=symbol device=activex; ods html body="&htmlfile" path="&htmlpath" parameters=("ZoomControlEnabled"="false" "DataTipStyle"="stick_fixed");
Proc GPlot Data=_Funnel_; Plot &rate*&pop lfunnelLimit*&pop ufunnelLimit*&pop
/ overlay / overlay hminor = 0 vminor = 0 vaxis = axis1 vref=&ABvalue cvref=purple lvref=3 frame
anno=annods html=popup ;
format &pop comma12.; run; quit; ods html close;
goptions reset=symbol device=activex; ods html body="&htmlfile" path="&htmlpath" parameters=("ZoomControlEnabled"="false" "DataTipStyle"="stick_fixed");
Plot &rate*&pop lfunnelLimit*&pop ufunnelLimit*&pop
vref=&ABvalue cvref=purple lvref=3
Funnel Plots SAS annotate
*annotations*; Data anno_label; Set anno_label; format function $8. text $12. style $12.; retain xsys '2' ysys '2'; retain when 'A';
*move to point*; function='move'; x=&pop; y=&rate; output;
*decide where to draw line to*; offset_xpct = 0.03*(&maxpop ‐ offset_ypct = 0.02*(&maxrate ‐ *min() and max() keep x,y coords within plot*; if _OOL_=+1 then do; x=min(x+offset_xpct,&maxpop);
y=min(y+offset_ypct,&maxrate); end;
if _OOL_= 0 then do; x=min(x+offset_xpct,&maxpop); end;
if _OOL_=‐1 then do; x=min(x+offset_xpct,&maxpop); y=max(y‐offset_ypct,&minrate);
end; *draw line*; function='draw'; line=1; size=1; output;
format function $8. text $12. style $12.;
function='move'; x=&pop; y=&rate;
*decide where to draw line to*; ‐ &minpop ); ‐ &minrate);
*min() and max() keep x,y coords within plot*; if _OOL_=+1 then do; x=min(x+offset_xpct,&maxpop);
y=min(y+offset_ypct,&maxrate);
if _OOL_= 0 then do; x=min(x+offset_xpct,&maxpop);
1 then do; x=min(x+offset_xpct,&maxpop); offset_ypct,&minrate);
Funnel Plots SAS annotate
*add text*; function='label'; if _OOL_=+1 then position='3'; if _OOL_=0 then position='6'; if _OOL_=‐1 then position='9'; text=&AnnoVar; size=2; hsys='3'; style="Calibri"; %if &explore ne %then %do; *add explore category to label*; text=trim(compress(text))||" "||trim(compress(&explore)); %end; %end; output;
drop offset_xpct offset_ypct; run;
text=&AnnoVar; size=2; hsys='3'; style="Calibri"; %if &explore ne %then %do; *add explore category to label*; text=trim(compress(text))||" "||trim(compress(&explore));
Funnel Plots SAS colour
*draw symbol*; function='symbol'; hsys='3'; size=2 Select(&explore); When(0) do; color="blue"; text="star"; end; When(1) do; color="red"; text="dot"; end; When(2) do; color="yellow"; text="plus"; end; When(3) do; color="green"; text="diamond"; end; When(4) do; color="orange"; text="triangle"; end; When(4) do; color="orange"; text="triangle"; end; When(5) do; color="brown"; text="#"; end; When(6) do; color="blue"; text="hash"; end; When(7) do; color="red"; text=")"; end; When(8) do; color="yellow"; text=":"; end; When(9) do; color="green"; text="'"; end; Otherwise;
end; output;
function='symbol'; hsys='3'; size=2; x=&pop; y=&rate;
) do; color="blue"; text="star"; end; ) do; color="red"; text="dot"; end; ) do; color="yellow"; text="plus"; end; ) do; color="green"; text="diamond"; end; ) do; color="orange"; text="triangle"; end; ) do; color="orange"; text="triangle"; end; ) do; color="brown"; text="#"; end; ) do; color="blue"; text="hash"; end; ) do; color="red"; text=")"; end; ) do; color="yellow"; text=":"; end; ) do; color="green"; text="'"; end;
Funnel Plots Automation! %macro funnel
( Data=, Out=FunnelOut,
Rate=, Pop=,
ABvar=, ABrate=, RatePer=100000, RatePer=100000,
CIpct=95, CIpct2=, CItrans=NORMAL, OD=1, minpop=, maxpop=, minrate=, maxrate=,
htmlfile=FunnelPlot.html, htmlpath=,
AnnoWhen=OUTSIDE, AnnoVar=,
Explore=, League=NO );
( Data=, Out=FunnelOut,
Rate=, Pop=,
ABvar=, ABrate=, RatePer=100000, RatePer=100000,
CIpct=95, CIpct2=, CItrans=NORMAL, OD=1, minpop=, maxpop=, minrate=, maxrate=,
htmlfile=FunnelPlot.html, htmlpath=,
AnnoWhen=OUTSIDE, AnnoVar=,
Explore=, League=NO );
Funnel Plots in SAS
Data + Creation + Outputs + Automation Data + Creation + Outputs + Automation
Funnel Plots Analytics
Anomaly detection Anomaly detection
Over dispersion
Funnel Plots Modeling
adjusted rate p
ˆ i
where p is the overall rate, O is the observ and E s the expected count from a Poisson i using small area data and the covariates
adjusted rate p ˆ O i adjusted rate p i E = ⋅
where p is the overall rate, O is the observed count, i and E s the expected count from a Poisson regression using small area data and the covariates of interest.
ˆ i adjusted rate p i E i = ⋅
Funnel Plots Analytics
***adjustment time***;
Proc Genmod Data=rate_data;
Model count = risk_factor_1 risk_factor_2 Model count = risk_factor_1 risk_factor_2 / dist=Poisson offset=logpop;
Output out=adj predicted=expected; run;
**and then merge back expected counts**;
***adjustment time***;
Proc Genmod Data=rate_data;
Model count = risk_factor_1 risk_factor_2 Model count = risk_factor_1 risk_factor_2 / dist=Poisson offset=logpop;
Output out=adj predicted=expected;
**and then merge back expected counts**;
Funnel Plots Modeling
unadjusted adjusted
Funnel Plots Modeling
unadjusted adjusted
Funnel Plots
•Utility in analyzing small areas
•Regression diagnostics
•SAS macro for Automation
•Proc GPlot with Annotate and Pop html output
Utility in analyzing small areas
Regression diagnostics
SAS macro for Automation
Proc GPlot with Annotate and Popup in
Work on Funnel Plots is part of an ongoing collaboration between
Doug Dover, AHW Doug Dover, AHW
Don Schopflocher, UofA
Work on Funnel Plots is part of an going collaboration between
Doug Dover, AHW Doug Dover, AHW and
Don Schopflocher, UofA
Thank you Thank you
References:
Funnel Plot Theory “Funnel plots for comparing institutional performance” Spiegelhalter 2005 everything to do with funnel plots and more! quite accessible addresses many issues with use excellent, readable paper
“Bristol, Shipman, and Clinical Governance: Shewhart’s Forgotten Lesson” Mohammed, Cheng, Rouse, Marshall 2001 Mohammed, Cheng, Rouse, Marshall 2001 commentary on use of control plots related to funnel plots
“Defining poor and optimum performance in an IVF programme” Castilla et al 2008 illustrates league plots, control plots and funnel plots
“Funnel plots for comparing institutional performance”
everything to do with funnel plots and more!
addresses many issues with use
“Bristol, Shipman, and Clinical Governance: Shewhart’s Forgotten Lesson” Mohammed, Cheng, Rouse, Marshall 2001 Mohammed, Cheng, Rouse, Marshall 2001 commentary on use of control plots
“Defining poor and optimum performance in an IVF programme”
illustrates league plots, control plots and funnel plots
References:
Examples of Funnel Plots “Regional differences in population prefectures in Japan: Application of relative survival models with funnel plots” Ito et al 2009 excellent example of what can go wrong using funnel plots has large and small regions similar to Alberta use of funnel plots to show regional survival rates
“Using Funnel Plots to Explore Variation in Cancer Mortality across Primary Care Trusts in South Primary Care Trusts in South Davies et al 2008 cancer mortality rates in 76 PCTs
“An alternative approach to quantifying and addressing inequity in healthcare provision” Battersby, Flowers, Harvey 2004 they used control plots
Examples of Funnel Plots “Regional differences in populationbased cancer survival between six prefectures in Japan: Application of relative survival models with
excellent example of what can go wrong using funnel plots has large and small regions similar to Alberta use of funnel plots to show regional survival rates
“Using Funnel Plots to Explore Variation in Cancer Mortality across Primary Care Trusts in SouthEast England” Primary Care Trusts in SouthEast England”
cancer mortality rates in 76 PCTs
“An alternative approach to quantifying and addressing inequity in
Battersby, Flowers, Harvey 2004
Author: Doug Dover Surveillance and Assessment Branch
Visualizing Small Area Estimates
Surveillance and Assessment Branch Community and Population Health Division Alberta Health & Wellness
Version History: Based substantially on the work of Doug Dover and Don Schopflocher presented at AH&W on Tuesday, April 6, 2010, in the Statistics in Surveillance Series, under the same title. “Funnel Plots as a Vehicle for Policy Relevant Analysis: Gaining Focus” Presented at ICHPS, Washington, 2009.
Funnel Plots: Visualizing Small Area Estimates
Community and Population Health Division
Based substantially on the work of Doug Dover and Don Schopflocher presented at AH&W on Tuesday, April 6, 2010, in the Statistics in Surveillance Series, under the
“Funnel Plots as a Vehicle for Policy Relevant Analysis: Gaining Focus” Presented at ICHPS, Washington, 2009.