04/18/23 1

Quality Assurance Overview

04/18/23 2

Quality Assurance System Overview FY 04/05- new Quality Assurance tools

implemented, taking into consideration CMS Quality Framework and expectations

FY 05/06- various revisions to Quality Assurance tools, including deletion / addition of some QA Indicators and reorganization of some Outcomes, scoring revision, modification of performance levels

FYs 06/07, 07/08, 08/09, 2010 & 2011- minimal changes to the Quality Assurance tools, contributing to the ability to further compare data / performance across extended periods of time

04/18/23 3

Inter-Rater Reliability Overview Reliability: measurement between 2 raters using the same tool, to

establish the extent of consensus on use of the tool

The expected outcomes of the Inter-Rater Reliability studies are: Provide wide-range sampling opportunities across regions and surveyors

Measure the extent of agreement in assessing compliance with QA Indicators

Identify Indicators where there are trends in scoring differences

Provide opportunities to increase consensus through: refinement of checklist guidance, modification of tools, education of those entities being surveyed

04/18/23 4

IRR- Summary of the Data

The table below shows Quality Assurance reliability data, agreement among Quality Assurance reviewers.

04/18/23 5

Impact of Quality Assurance Data Provides an overview of system performance

Viewed and utilized by a wide audience Data utilized by DIDD Central Office, Regional Offices, Quality Management Committees, court monitors

Facilitates change throughout the service delivery system and decision making Data is used in assessing progress and to

identify areas needing corrective intervention

Special Reporting Focused review of Domains, Outcomes and

Indicators Focused analysis with provider detail Review of provider performance by provider-

type and regionally Comparison of performance across years

04/18/23 6

Quality Assurance System Overview Quality Assurance Survey Process:

Consultative Reviews Annual Review except for some clinical providers and those

providers achieving 3 or 4 Star status Sample Selection Notification / document request On-site review Conciliation process Report of findings Reporting through Regional and Statewide Quality

Management Committees Quality Improvement Planning

04/18/23 7

Quality Assurance System Overview Quality Assurance Tools:

Organizational: Day-Residential / Personal Assistance /

Clinical Independent Support Coordination

(organizational practices & utilizing data from waiver Individual Record Reviews)

Individual: Day-Residential Personal Assistance Behavioral Nursing Therapy

04/18/23 8

Quality Assurance System Overview Quality Assurance Tool Structure:

Domains1. Access and Eligibility2. Individual Planning and Implementation3. Safety and Security4. Rights, Respect and Dignity5. Health6. Choice and Decision Making7. Relationships and Community Membership8. Opportunities for Work9. Provider Capabilities and Qualifications10. Administrative Authority and Financial Accountability

Outcomes Indicators

Guidance and Provider Manual References

04/18/23 9

Outcome 2B. Services and supports are provided according to the person’s plan.

Indicators Results Guidance Comments *2.B.5. Provider documents provision of services and supports in accordance with the plan.

Y N NA IJ

For all Providers: Provider documentation systems are developed to support the delivery of services. Practices include oversight to ensure staff understand responsibilities for documentation. Provider Manual Reference: 3.17; 6.11; 8.7.a.; 8.9.e.; Chapter 11; 12.9.; 12.10.; 13.14.; 13.15; 15.2; 15.3

Outcome 2D. The person’s plan and services are monitored for continued appropriateness and revised as needed.

Indicators Results Guidance Comments 2.D.5. The provider has a process for reviewing and monitoring the implementation of the plan and progress toward desired goals.

Y N NA IJ

For all Providers: The provider has developed an ongoing, systematic review process that promotes identification, tracking and coordination of activities related to implementation of each person’s plan and any needed follow-up activities. For Clinical Providers: The provider has a process to prepare a review of progress and an updated justification for services at the time of the annual ISP review. Day-Residential and PA Provider Manual Reference: 3.10.f.; 3.15.; 3.17.; 3.18.; 3.19.; 6.5. 13); 6.6.f.; 10.6.c.; Chapter 11 Clinical Provider Manual Reference: 3.10.f.; 3.15.; 3.17.; 3.18.a.,b.; 8.9.f.; 12.10.; 13.12.; 13.14.; 13.15.; 14.5.; 15.2.; 15.3.

*2.D.6. Provider documentation indicates appropriate monitoring of the plan's implementation.

Y N NA IJ

For all Providers: Providers utilize resolution processes if needed to ensure supports and services are provided in accordance with the ISP. The provider agency ensures that systems developed to verify service delivery are effective.

04/18/23 10

Quality Assurance System Overview Quality Assurance Scoring & Domain

Applicability: Domains Applicable by Provider Type:

Day-Residential: 2, 3, 4, 5, 6, 7, 8, 9, 10 Personal Assistance: 2, 3, 4, 5, 6, 9, 10 Support Coordination: 1, 2, 3, 9, 10 Behavioral: 2, 3, 4, 6, 9, 10 Nursing: 2, 3, 4, 5, 6, 9, 10 Therapy: 2, 3, 4, 6, 9, 10

On the web: QA and Waiver Review Tools Report Card Listing Star Lising Resource / Reference Documents

04/18/23 11

Quality Assurance System Overview Quality Assurance Tool Scoring:

Domains: 6- Substantial Compliance 4- Partial Compliance 2- Minimal Compliance 0- Noncompliance

Outcomes: SC- Substantial Compliance PC- Partial Compliance MC- Minimal Compliance NC- Noncompliance

Indicators: Yes- Substantial Compliance No- Noncompliance

Performance Levels: Exceptional Performance Proficient Fair Significant Concerns Serious Deficiencies

Special Scoring Criteria: Exceptional: A score of Substantial Compliance is required in Domains 2, 3, 5 and 9, if applicable.

Proficient: For each applicable Domain, the performance score must be at least Partial Compliance.

Fair: For each applicable Domain, the performance score must be at least Minimal Compliance.

04/18/23 12

Star Providers- 3 & 4 Star Status 4-Star Status:

96% or above compliance on QA surveys for 2 years; No Domain below Partial Compliance Must achieve Substantial Compliance in Domain 3 ISC agencies must achieve Substantial Compliance in Domain 2

No preventable egregious events resulting in death of individual for one year;

Annualized substantiated investigation rate of 10 substantiations per 100 persons supported (10:100) or less for one year;

Quality Tier designation from Court Monitor, if applicable.

Approval for four-star status is by the Regional Quality Management Committee, followed by submission to State-wide Quality Management Committee for final approval.

04/18/23 13

Star Providers- 3 & 4 Star Status 3-Star Status:

85% or above compliance on QA surveys for 2 years; No Domain below Partial Compliance Must achieve Substantial Compliance in Domain 3 ISC agencies must achieve Substantial Compliance in Domain 2

No preventable egregious events resulting in death of individual for one year;

Annualized substantiated investigation rate of 10 substantiations per 100 persons supported (10:100) or less for one year;

Quality Tier designation from Court Monitor, if applicable.

Approval for four-star status is by the Regional Quality Management Committee, followed by submission to State-wide Quality Management Committee for final approval.

04/18/23 14

Sample Data:

04/18/23 15

Sample Data:

04/18/23 16

Waiver Monitoring Overview DIDD implements three HCBS Waivers under the administrative oversight of

TennCare: Self-Determination Waiver State-wide Waiver Arlington Waiver

Each waiver entails annual monitoring as performed by Quality Assurance, with follow-up remediation and validation activities as coordinated by regional Operations and Case Management staff

Monitoring for each waiver consists of administration of two review tools Qualified Provider Individual Review (involves identification of a state-wide sample which is selected

at the beginning of each waiver-year) All findings / issues are expected to be remediated with provider and

systemic trends identified and addressed as data is analyzed. Findings / issues are reviewed and discussed by both the Regional Quality

Management Committees and the State-wide Quality Management Committee