electronic case reviews and consumer surveys september 16, 2013 soon to be known as: opportunities...

Electronic Case Reviews and Consumer SurveysSeptember 16, 2013

Soon to be known as: Opportunities for

Ohioans with Disabilities

Presented by:Janet Cool, MSSAAlana Evans, MS

2

Electronic Case Reviews & Consumer Surveys 9/16/2013

Case Review

s

Background for Case Reviews


3

Random Selection of VR Cases

IT generated list

Beginning of each month

Reflection of statewide patterns and trends

Background (continued)


4

Random Selection of VR Cases statewide that are either:

Eligible (with a Plan created)ServiceEmployedClosed-Rehabilitated or Closed-Other



5

Additional Criteria for Random Selection of VR Cases:

cases Closed-Other must have at least reached Eligible (with a Plan created) status

cases opened in Application status throughout the previous 24 months

either of the Closed statuses for cases closed within current FFY



6

# of random reviews completed by QA-RPS (PIE) determined by PIE Manager during first month of each new FFY

based upon previous years’ total number of plans written

statistically sound sampling (minimum of a 90 percent confidence level)

History of Ohio Case Reviews

Prior to FFY2012 – VR Supervisors = 10 case reviews per VR

Counselor each year PIE RPS = 10 case reviews per office, -> then

later, 1 case review for each VR Counselor statewide

FFY 2012 and 2013 approx. 375 random case reviews per year VRS = as needed for performance


7

Types of Case Reviews

Programmatic – (Program Specialists)

Performance - (Supervisors)

Targeted - (RPS & VRS)


8

Case Review Guide• Referenced throughout case review

process to:

verify the meaning of each item

determine how to rate item(s)

identify where information is typically documented in AWARE


9

VRQA System App.


10

VRQA System App. (continued)


11


Identifying information pre-populated from the AWARE system

Each item is rated by clicking the circle beneath the appropriate rating (P, PP, NP, N/A).


12


Each section considers pertinent quality indicator:

Case documentationVocational guidance &

counselingInformed choiceTimelinessFiscal responsibility


13

Case Review Form – (APPLICATION)

1. Application (CFR 361.37; 41; 47; 48; 52 / OAC 3304-2-51) Referral Date: Application Date: A. Referral/Intake Process P PP NP N/A

1. Participant Information/Application for Vocational Rehabilitation Services form signed and dated by applicant ..................................................................

2. Participant Information/Application for Vocational Rehabilitation Services form dated and initialed by authorized representative………………………

3. Intake Interview Form completed or information entered into AWARE Participant Module ............................................................................................

4. Health Assessment Questionnaire completed. ................................................... 5. Documentation of the identification of consumer’s impairments, needs,

and assets has begun. .......................................................................................

6. Evidence of vocational guidance and counseling and professional

relationship among the VRC, consumer, and stakeholders has begun. ..............

7. Informed choice documented. ............................................................................ 8. Referral/intake process completed in a timely manner with no unexplained

gaps. ..................................................................................................................

9. Documentation that only services authorized were those necessary to

participate in the intake/referral process, i.e. auxiliary services, transportation, etc. .............................................................................................

B. Confidentiality P PP NP N/A

(CFR 361.38 / OAC 3304-2-51; 63)

1. Signed releases for all individuals to whom staff released and obtained information .........................................................................................................

2. Release forms filled out completely and accurately ............................................

Comments: Application Score: ___/___ = ___% (depending on N/A responses)


14

VRQA System App. Complete all sections (or only those

items desired for targeted review)

If a new case review is commenced and the reviewer gets interrupted the “Save” button can be selected.

VRQA saves those completed portions as “In Process”.


15

Case Review Form – (ELIGIBILITY)

A. Eligibility/Order of Selection (OOS) Determination P PP NP N/A (CFR 361.5; 41; 42; 45; 47; 48; 52 / OAC 3304-2-51; 54; 65) 1. Documentation of physical/mental impairment ................................................... 2. Documentation of substantial impediment to employment .................................. 3. Documentation consumer can benefit from VR services .................................... 4. Documentation consumer requires VR services ................................................. 5. If eligibility not determined within 60 days, active time extension present… 6. Eligibility determination was accurate…………………………………………. 7. Documentation supports OOS functional capacity limitations……………… 8. Documentation supports the need for multiple services over an extended

period of time…………………………………………………………………….. 9. Order of Selection determination was accurate……………………………… 10. Certificate of Eligibility/Order Of Selection letter(s) present ............................... 11. Eligibility determined in a timely manner with no unexplained gaps……….. 12. OOS determined in a timely manner with no unexplained gaps……………. 13. DD Rights explained and signed ........................................................................ 14. Documentation of discussion and exploration of the impact of limitations on

employment and involving the consumer in a counseling relationship. ............... 15. Documentation that the consumer exercised informed choice in decisions

related to the provision of assessment services. ............................................... 16. If applicable, documentation that existing records, previous case

documentation, etc. were used to document the physical/mental impairment. ...


16

Case Review Form – (ELIGIBILITY)

B. Ineligibility P PP NP N/A

(CFR 361.5; 42; 43; 44; 47 / OAC 3304-2-54; 61)

1. Documentation of trial work experience(s)..........................................................

2. Closure-Ineligibility letter provided ......................................................................

Comments: Order of Selection Waiting List Date:

C. Order of Selection-Waiting List P PP NP N/A (CFR 361.5; 36; 37; 42; 47 / OAC 3304-2-54; 65) 1. Information and referral services was provided to consumer .............................. 2. Consumer on waiting list was contacted at least every 6 months ....................... Comments: Eligibility Score: ___/___ = ___% (depending on N/A responses)


17

VRQA System App.


18

Case Review Form – (SERVICE & EMPLOYMENT)

3. Service and Employment CA Date: Plan Signature/Start Date:

Service Begin Date:

Service Job Ready Date:

Post-Employment Services (most recent begin date):

A. Development of the Comprehensive Assessment P PP NP N/A (CFR 361.5; 37; 45; 46; 47; 48; 52 / OAC 3304-2-51; 56; 63) 1. The consumer’s unique strengths, resources, priorities, concerns,

abilities, capabilities, interests, informed choice, and need for supported employment were documented…………………………………………………

2. The consumer’s employment outcome and its appropriateness were documented ............................................................................................................................

3. The consumer’s vocational rehabilitation needs were documented……….. 4. Justification of services provided in non-integrated settings ...............................


19

Case Review Form – (SERVICE & EMPLOYMENT)

B. Individualized Plan for Employment (IPE)

(CFR 361.5; 22; 37; 45; 46; 47; 48; 52, 53 / OAC 3304-2-52; 55, 56) IPE Goal:

P PP NP N/A

1. Description of specific employment outcome chosen .........................................

2. Employment goal is supported by current Labor Market Information (LMI)

3. Specific rehabilitation services necessary to achieve employment outcome. .......................... 4. Providers chosen to provide VR services ................................................................................ 5. IPE includes estimated costs and time frames .................................................. 6. Consumer’s responsibilities individualized and relevant to his/her VR program

........................................................................................................................... 7. Consumer’s contribution clearly reflected on the IPE ........................................ 8. Comparable benefits clearly reflected on the IPE .............................................. 9. Supported employment section accurate and complete ....................................

10. All required signatures present ........................................................................... 11. Annual reviews completed ................................................................................. 12. Documentation was reflective of the decision-making process about the goal,

services, setting etc. ........................................................................................... 13. Documentation reflected VRC collaboration with stakeholders. .......................... 14. Documentation that the VRC continued to discuss and explore the impact of

limitations on employment and involve the consumer in a counseling relationship. ........................................................................................................

15. Documentation reflected evidence of consumer involvement throughout the decision-making process for all aspects of the IPE. ............................................

16. IPE developed in a timely manner with no unexplained gaps ............................. 17. Documentation showing consumer was making progress .................................. 18. Documentation that the VRC reviewed reports and took action based on

report content as needed .................................................................................. 19. Services were provided in a timely manner with no unexplained gaps ............... Comments: Service and Employment: ___ / ___ = ___% (depending on N/A responses)


20

Case Review Form – (CLOSURE)

(CFR 361.1; 5; 43; 44; 47; 48; 52; 56 / OAC 3304-2-54; 61) P PP NP N/A

A. Closure/Outcome 1. Use of correct letter with all relevant sections addressed .................................. 2. Employment outcome consistent with IPE employment goal, consumer’s

characteristics and informed choice. ................................................................ 3. Documentation employment outcome was satisfactory to consumer and VRC . 4. Documentation consumer and VRC agreed the consumer was performing well

on the job .......................................................................................................... 5. Documentation consumer’s wage was not less than the wages customarily

paid by the employer for the same work performed by non-disabled individuals .........................................................................................................

6. Documentation that services provided on the IPE assisted consumer in obtaining/maintaining employment ....................................................................

7. When appropriate, consumer was referred to other programs, and, if applicable, specific action steps for enhancing vocational readiness in the future were documented ....................................................................................

8. Documentation that VRC attempted to contact and provide the consumer with a full consultation prior to case closure ..............................................................

9. Case was closed in a timely manner ................................................................. Comments: Closure Score: ___ / ___ = ___% (depending on N/A responses)


21

Case Review Form – (FINANCIAL)

5. Financial (CFR 361.45; 46; 47; 48; 50 / OAC 3304-2-51; 56) P PP NP N/A A. Authorizations 1. Services authorized are on the IPE/amendments .............................................. 2. Signatures on IPE/amendments on or before authorization date 3. Authorization dates on or before authorized services/services provided............ 4. Authorization and billing processes reflective of ongoing fiscal

management ..................................................................................................... Comments: Service Delivery Score: ___ / ___ = ___% (depending on N/A responses)

Total Score: ____ / ____ = ____% (total of each section score)


22

VRQA System App.


23

VRQA System App. Once completed -> select

“Complete” button

VRQA saves the review and forwards notification to the appropriate counselor/coordinator for comment

Case is then in “Completed” status rather than “In Process”.


24

VRQA Case Review Reports


25

Tableau reports updated daily

Available online and accessed by DPI staff, VR Deputy Directors, VR Assistant Deputy Directors, Area Managers, VR Supervisors and VR/VRP3 Program Specialists.



26

Data can be drilled down separating:

PIE reviews from supervisor reviews

full case reviews from targeted case reviews

time periods (ie. Federal Fiscal Year, Quarters, etc.)

location (statewide, bureau, area, team, and counselor/coordinator levels)



27

Case Review Dashboard Report

updated quarterly and sent to VR administration

identifies areas of strengths, opportunities, and threats for specific case review items

highlights statewide trends



28

PIE Case Review Reportsscores for each item on the case review form as well as a total case review score

used to generate the quarterly Case Review Summary Report previously referenced

used to evaluate overall programmatic performance and to determine both programmatic and training decisions



29

PIE Case Review List Report

identifies case reviews currently in VRQA System for each counselor by:

review statusreview status datedivision/bureau, area, office/team,

supervisor, reviewer, type of review, and counselor/coordinator



30

PIE Case Review List Summary Reports

identifies case reviews by type:

NewIn processCompletedCommentedFinalized

Technical Assistance


31

In addition to PIE reports, several additional reports utilized to provide technical assistance, feedback and guidance related to program expectations and outcomes:

Master ListFront Door (aging)Area PerformanceCurrent Year ActivityStandards and Indicators

Training


32

PIE staff work with VR Training Manager and Area Managers to coordinate training topics based on patterns and trends from case reviews

Training addresses statewide issues, area issues, and PIE updates and may also include specialized populations (i.e. transition, TBI, substance abuse, etc.)

VR Action Plan


33

Action steps identified by VR management for prioritized areas that are below 90 percent compliance on a statewide level

Coordinated with the PIE Manager

who establishes methods to proactively address item(s) and reports to track progress on these priority area(s)

Administrative Review and Resolution


34

Case review finding is considered unjustified, discussed with VR Supervisor

If Supervisor disagrees with findings, contact reviewer within ten (10) business days of completion of the review and provide supporting written documentation contrary to the findings

Administrative Review and Resolution


35

Reviewer discusses with his/her manager to see if the documentation provided warrants having the finding rescinded

Manager contacts PIE Manager, if necessary, to discuss the issue in further detail prior to making a decision

36


Consumer

Surveys

Consumer Survey Types

• Consumer Satisfaction (CFR 361.52)

• Ineligibility (CFR 361.43)

• Extended Employment (CFR 361.55)


37

Consumer Satisfaction

• Completed for cases on the QA random list whose case was reviewed by PIE staff that assesses overall satisfaction with VR services including resources/information provided, staff, and outcome/results.


38

Consumer Satisfaction


39

Ineligibility• Completed within twelve (12) months

of closure, and annually thereafter if requested, to determine the individual’s current work status and need for VR services.


40

Ineligibility Survey


41

Extended Employment

• Completed annually for two (2) years, and then as requested for consumers closed in extended employment to determine interests, priorities, and needs with respect to competitive employment or training for competitive employment.


42

Extended Employment Survey


43

VRQA System App. The electronic system that stores

and manages information related to case reviews, consumer surveys, and program monitoring.

***VR staff attempts to secure an email address to promote more rapid feedback and to decrease the cost of time spent to contact consumers by telephone to obtain feedback.


44

Consumer Survey Procedure Basics

• Consumers can provide feedback anonymously (no name required)

• Sign in to QA application online to provide electronic feedback, w/ ability to “opt out”

• Surveys available in English, Spanish, and can be modified with web browser for large print

• Respond to survey online (within 90 days) or by telephone

• No longer sending hard copy surveys by mail


45

Procedure Basics (continued)

QA staff obtain lists of consumers for each of the 3 surveys every month.

QA RPS uses AWARE to email the survey cover letter as an attachment in a case note

If no response within the first month following notification, QA RPS sends a second email

If no response after second email notification, consumer’s name is added to the list to contact by phone

If no email address, consumer contacted by phone


46

Procedure Basics (continued


47

OR



48



49

Procedure Basics (continued)


50

Survey Follow Up Procedure• Each office/area designates staff responsible

to follow-up with consumers who indicated that they want to be contacted by RSC.

• Consumer Survey Follow-Up lists indicate the consumers requesting to be contacted and the type of survey completed.

• Area Management (or designee) determines distribution and management of the completion of follow-up contacts and documents this in AWARE case notes.


51

VRQA Consumer Survey Reports

• Consumer Survey Dashboard Report

• Survey Status Summaries

• Survey Status Details


52

Customer Satisfaction Survey Reports

• Customer Satisfaction Summary

• Customer Satisfaction Detail


53

Ineligibility Survey Reports

• Ineligibility Survey Summary

• Ineligibility Survey Detail


54

Extended Employment Survey Reports

• Extended Employment Summary

• Extended Employment Detail


55

Contact Information:Janet Cool, ManagerProgram Integrity and [email protected]

Alana Evans QA Rehabilitation Program [email protected]


56

Agency name changes October 1

QUESTIONS


57

electronic case reviews and consumer surveys september 16, 2013 soon to be known as: opportunities...

Documents