printed by

www.postersession.com

Implementation of a Pilot Program to Address Reliability of Environmental Sanitation Implementation of a Pilot Program to Address Reliability of Environmental Sanitation Inspections at Memphis and Shelby County Health DepartmentInspections at Memphis and Shelby County Health Department

Tyler Zerwekh, DrPHMemphis and Shelby County Health Department

Environmental Health Bureau

•With over 5,000 food establishments throughout Memphis and Shelby County, MSCHD is required by With over 5,000 food establishments throughout Memphis and Shelby County, MSCHD is required by both Tennessee Department of Health and Tennessee Department of Agriculture annotated law to both Tennessee Department of Health and Tennessee Department of Agriculture annotated law to inspect facilities twice per year, and more often if critical sanitation violations have been cited or if inspect facilities twice per year, and more often if critical sanitation violations have been cited or if there are food-borne outbreaks or food-borne complaint investigations.there are food-borne outbreaks or food-borne complaint investigations.

•The association between environmental sanitation inspections and food-borne outbreaks in The association between environmental sanitation inspections and food-borne outbreaks in communities has been documented extensively. With only seventeen environmentalists trained at communities has been documented extensively. With only seventeen environmentalists trained at MSCHD to perform sanitation inspections on the 5,000+ facilities, it is not uncommon for an MSCHD to perform sanitation inspections on the 5,000+ facilities, it is not uncommon for an environmentalist at MSCHD to perform over 700 inspections and follow-ups in one calendar year. The environmentalist at MSCHD to perform over 700 inspections and follow-ups in one calendar year. The sheer amount of inspections by environmentalists’ can lead to practices that ultimately compromise the sheer amount of inspections by environmentalists’ can lead to practices that ultimately compromise the integrity of an individual inspection to satisfy the overall arching goal of total inspection completion. integrity of an individual inspection to satisfy the overall arching goal of total inspection completion.

•During a recent audit investigation by the Tennessee Department of Health (TDH) on the MSCHD During a recent audit investigation by the Tennessee Department of Health (TDH) on the MSCHD Environmental Sanitation division, it was documented that unannounced inspections by TDH Environmental Sanitation division, it was documented that unannounced inspections by TDH immediately after a MSCHD inspection yielded an average difference score of 35 points with a range of immediately after a MSCHD inspection yielded an average difference score of 35 points with a range of one to five critical violations not documented by MSCHD environmentalists but observed and one to five critical violations not documented by MSCHD environmentalists but observed and documented by TDH audit staff.documented by TDH audit staff.

•A total of 36 facilities were audited for 16 Environmentalists by 3 Lead Environmentalists A total of 36 facilities were audited for 16 Environmentalists by 3 Lead Environmentalists during the month of December 2008.during the month of December 2008.

•Results demonstrated an average of 6.11 points lower score when the Lead Environmentalist Results demonstrated an average of 6.11 points lower score when the Lead Environmentalist performed an audit inspection after the environmentalist.performed an audit inspection after the environmentalist.

•The range of difference in sanitation inspection was from 20 points lower to 9 points higher The range of difference in sanitation inspection was from 20 points lower to 9 points higher than the Environmentalist when the Lead Environmentalist performed the audit inspection than the Environmentalist when the Lead Environmentalist performed the audit inspection immediately after the initialimmediately after the initial

Program GoalProgram GoalTo implement an environmental sanitation inspection pilot program to address reliability, validity, and repeatability results for To implement an environmental sanitation inspection pilot program to address reliability, validity, and repeatability results for a comprehensive environmental sanitation program.a comprehensive environmental sanitation program.Health ProblemHealth ProblemMore than 54 billion meals are served at 844,000 commercial food establishments in the United States each year and nearly More than 54 billion meals are served at 844,000 commercial food establishments in the United States each year and nearly half of all money spent on food is done so at food establishments. On a typical day, 44% of adults in the United States eat at a half of all money spent on food is done so at food establishments. On a typical day, 44% of adults in the United States eat at a restaurant. Approximately 40% of food-borne disease outbreaks reported to the Centers for Disease Control and Prevention are restaurant. Approximately 40% of food-borne disease outbreaks reported to the Centers for Disease Control and Prevention are attributed to commercial food establishments. Preventing restaurant-associated food-borne disease outbreaks is an important attributed to commercial food establishments. Preventing restaurant-associated food-borne disease outbreaks is an important task of public health departments. Many times, sanitation inspections performed by public health departments at these food task of public health departments. Many times, sanitation inspections performed by public health departments at these food establishments are not efficient or reliable when performed due to myriad reasons including, but not limited to, inspector establishments are not efficient or reliable when performed due to myriad reasons including, but not limited to, inspector training level, workload, and type of facility inspected.training level, workload, and type of facility inspected.Outcome ObjectiveOutcome ObjectiveThe Memphis and Shelby County Health Department Environmental Sanitation program aimed to work with program The Memphis and Shelby County Health Department Environmental Sanitation program aimed to work with program managers, lead environmentalists, and environmentalist to implement and execute a quality assurance pilot project that would managers, lead environmentalists, and environmentalist to implement and execute a quality assurance pilot project that would increase the repeatability and accuracy of environmental sanitation inspection scores at sanitation facilities.increase the repeatability and accuracy of environmental sanitation inspection scores at sanitation facilities.DeterminantDeterminantTennessee Department of Health program audit determined a difference in 35 points (5 violations, 1 critical violation) on Tennessee Department of Health program audit determined a difference in 35 points (5 violations, 1 critical violation) on average of facilities audited after a MSCHD Environmentalist inspectionaverage of facilities audited after a MSCHD Environmentalist inspection

METHODOLOGY:METHODOLOGY:  EventEvent: Quality Assurance Pilot Program Planning: Quality Assurance Pilot Program PlanningAActivities:ctivities:Meeting with Environmental Health Administrator and MSCHD Director of Health to present ideas for increasing repeatability Meeting with Environmental Health Administrator and MSCHD Director of Health to present ideas for increasing repeatability and accuracy of sanitation inspections (June 2008)and accuracy of sanitation inspections (June 2008)Meeting with MSCHD Environmental Sanitation program managers to unveil quality assurance pilot program sanitation Meeting with MSCHD Environmental Sanitation program managers to unveil quality assurance pilot program sanitation inspection audit process and enlist support for the program (August 2008)inspection audit process and enlist support for the program (August 2008)Develop a timeline for integration, implementation, and execution of the pilot program (August 2008)Develop a timeline for integration, implementation, and execution of the pilot program (August 2008)  Event:Event: Workforce Development and Enhancements – Addressing personnel issues and the increase in expertise through Workforce Development and Enhancements – Addressing personnel issues and the increase in expertise through workforce development and technologyworkforce development and technologyActivities:Activities:Assess TDH audit criteria into pilot program training to ensure standardization (August 2008)Assess TDH audit criteria into pilot program training to ensure standardization (August 2008)Conduct trainings for Lead Environmentalists to standardize audit inspections (September 2008)Conduct trainings for Lead Environmentalists to standardize audit inspections (September 2008)Conduct trainings/re-trainings for Environmentalist to standardize sanitation inspections (September 2008)Conduct trainings/re-trainings for Environmentalist to standardize sanitation inspections (September 2008)Reassign inspection “zones” to minimize inspection bias and confoundingReassign inspection “zones” to minimize inspection bias and confounding  Event:Event: Pilot Program Implementation Pilot Program ImplementationActivities:Activities:Environmentalists’ inspection of facilitiesEnvironmentalists’ inspection of facilitiesLead Environmentalists’ audit inspections of random facilities.Lead Environmentalists’ audit inspections of random facilities.Education to operators selected for audit inspections of pilot programEducation to operators selected for audit inspections of pilot programEnvironmental Sanitation program manager analysis of audit inspectionsEnvironmental Sanitation program manager analysis of audit inspections

For more information, contact:Tyler Zerwekh, DrPH

Memphis and Shelby County Health department 814 Jefferson Ave., 502H

Memphis, TN 38105Email: Tyler.Zerwekh@shelbycountytn.gov

CHART or PICTURE

CHART or PICTURE

The results demonstrate the pilot program was unsuccessful upon initial examination. The results demonstrate the pilot program was unsuccessful upon initial examination. However, it is worth noting that 20% (7 / 36) of the facilities audited did observe the However, it is worth noting that 20% (7 / 36) of the facilities audited did observe the environmentalist scoring the facility lower than the audit procedure done by the Lead environmentalist scoring the facility lower than the audit procedure done by the Lead Environmentalist after the initial inspection. These results extrapolated demonstrate Environmentalist after the initial inspection. These results extrapolated demonstrate there was value in implementation of the program, whether it was due to fear/concern there was value in implementation of the program, whether it was due to fear/concern the environmentalist would be exposed for poor inspection methodology or because the environmentalist would be exposed for poor inspection methodology or because there was an actual observed improvement in the sanitation inspection procedure.there was an actual observed improvement in the sanitation inspection procedure.  While the results were not favorable, it is worth noting this was the initial audit follow-While the results were not favorable, it is worth noting this was the initial audit follow-up analysis for this pilot program and hopefully future audits will demonstrate more up analysis for this pilot program and hopefully future audits will demonstrate more frequent and improved reliability and repeatability in environmentalists’ scores and frequent and improved reliability and repeatability in environmentalists’ scores and Lead Environmentalists audits. Future steps and next directions include: 1) continued Lead Environmentalists audits. Future steps and next directions include: 1) continued audit inspections to further underscore the importance of proper sanitation inspections, audit inspections to further underscore the importance of proper sanitation inspections, 2) refresher and additional trainings for environmentalists who continue score 2) refresher and additional trainings for environmentalists who continue score inspections higher than audit follow-ups, and 3) disciplinary action for inspections higher than audit follow-ups, and 3) disciplinary action for environmentalists who continue to underscore sanitation inspections.environmentalists who continue to underscore sanitation inspections.  

CHART or PICTURE

LOGO

•The significance of these findings listed in the Background has immediate The significance of these findings listed in the Background has immediate environmental public health ramifications on Shelby County, TN, and its stakeholders. environmental public health ramifications on Shelby County, TN, and its stakeholders. Inaccurate and underscored inspections increase the likelihood for a facility to continue Inaccurate and underscored inspections increase the likelihood for a facility to continue practices and controls deemed unsanitary and unsafe. This can lead to an increased risk practices and controls deemed unsanitary and unsafe. This can lead to an increased risk in unsafe food and food handlers, which ultimately leads to a higher probability of food-in unsafe food and food handlers, which ultimately leads to a higher probability of food-borne and infectious disease transmission to facility patrons. This project will focus on borne and infectious disease transmission to facility patrons. This project will focus on the development and implementation of a sanitation pilot program that aims to increase the development and implementation of a sanitation pilot program that aims to increase the accuracy and reliability of environmental sanitation inspections within the MSCHD the accuracy and reliability of environmental sanitation inspections within the MSCHD Environmental Sanitation Section.Environmental Sanitation Section.

•Problem Statement: Implementation of an environmental sanitation inspection pilot Problem Statement: Implementation of an environmental sanitation inspection pilot program to address reliability, validity, and repeatability results for a comprehensive program to address reliability, validity, and repeatability results for a comprehensive environmental sanitation program, which will provide accurate and representative environmental sanitation program, which will provide accurate and representative sanitation scores of operator facilities to Shelby County stakeholders.sanitation scores of operator facilities to Shelby County stakeholders.

BACKGROUND

PURPOSE AND HYPOTHESIS

MATERIALS AND METHODS RESULTS

CONCLUSIONS

BIBLIOGRAPHY

LOGO

Memphis and Shelby County Health Department (MSCHD) is a joint funded agency of County and City governments. Shelby County contains seven incorporated municipalities (e.g., Memphis, Arlington, Bartlett, Collierville, Germantown, Lakeland, and Millington) and several unincorporated areas. Shelby County is also the largest county in the State of Tennessee, both geographically (approximately 783 square miles) and in population (909,035). With over 5,000 food establishments throughout Memphis and Shelby County, MSCHD is required by both Tennessee Department of Health and Tennessee Department of Agriculture annotated law to inspect facilities twice per year, and more often if critical sanitation violations have been cited or if there are food-borne outbreaks or food-borne complaint investigations.

Changing Inspection Behavior

Not Changing Inspection Behavior

Benefits •Increased quality of food consumption

•Decreased incidence of food-borne illnesses and outbreaks

•Efficient inspection process

•Additional encouragement and acknowledgement by management and community

•Environmentalists continue with minimal workload

•Status quo

•Inspection quotas are met

Costs •More work on behalf of environmentalists

•More interaction (good or bad) and perception of accusations / unfair practices to restaurants

•More work for audits quality assurance inspections by lead staff

•Increased food-borne illnesses

•Vicious cycles on behalf of environmentalists performing inspections

•Environmentalists don’t have to change behavior

•No long term improvements in inspection process

•No opportunities for additional training that is meaningful

B Problem Symptom

Time

Shifting the Burden

Quick Fix: Time Spent and Resources

Poor quality of Sanitation

I nspections and Pressure to Meet

Quotas

B

Capacity development. For training

Betters I nspectors meet Quotas

Side Eff ects: Failing to meet monthly inspection

quotas and ‘cutting corners’ during

inspection process to meet their quota at the

last minute

R

Symptom Correcting

Process

Cause Correcting

Process

Quick Fixes

Long Term Solutions

Side Eff ects Undermining Eff orts to Address

Fundamental I ssue

www.appliedsystemsthinking.com

Side Effect —

B

R

B

Quick Fix -Time Spent and

Resources Allocated and used

Poor Quality of Sanitation

Inspections & FailureTo meet inspec.

quotas

Long Term SolutionCapacity development

for training. Better inspectors

meet quotas

B

Shifting the Burden: Interventions

X

X

Desired Vision

1

2

2

3

1. Quality inspections ultimately increase efficiency while decreasing time spent and resources allocated.

2. Better trained inspectors understand and implement quality inspections while maintaining quality assurance integrity and quota standards.

3. Cannot implement the quick fix to build…creates vicious cycles.

???

Accidental Adversaries

Elicit a reward program to encourage quantity over quality

inspections, pay raises, positive perf ormance

reviews

B

Deprives Sanitarians opportunity to diversif y workplace skills

while minimizes job marketability. Creates

redundancy in inspection process

Continue to rationalize no change f rom

management to justif y mediocre inspection. I ncreases inspection quantity which meets

manager goals

B Sanitarians meet inspection quotas. Receive protection f rom

manager f or adhering to her policies and stubbornness to

change

No f orum/ avenue to discuss or implement

change in inspection quality or effi ciency

Sanitation manager invokes processes to her liking that allows f or

sanitarian to inspect restaurants in an incomplete f ashion.

Sanitarians oppose antiquated and redundant inspections techniques yet keep manager happy by perf orming

them in the manner they are requested

R

R

Sanitation managers meets restaurant inspection quotas