state monitoring 2010 nascsp wap orientation training bob scott director of wap services nascsp
TRANSCRIPT
![Page 1: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/1.jpg)
State Monitoring
2010 NASCSP WAP Orientation Training
Bob ScottDirector of WAP Services
NASCSP
![Page 2: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/2.jpg)
Grant Guidance Requirements
State must conduct comprehensive monitoring of each subgrantee at least once a year.
A minimum of 5% of completed units must be inspected by the Grantee Higher % strongly encouraged Additional inspections if significant problems are
found Grantees encouraged to also inspect jobs “in
progress” Safe work practices, LSW, and other relevant factors Can be very revealing
![Page 3: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/3.jpg)
Grant Guidance Requirements
Subgrantee should be briefed on the observations and findings generated by the visit.
Within 30 days, State will prepare a written report on its findings and send it to the subgrantee for corrective action
Noncompliance findings unresolved within 45 days should be reported to the PMC.
Sensitive or significant noncompliance findings should be reported to the PMC immediately.
![Page 4: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/4.jpg)
Grant Guidance Requirements
Major findings should be tracked by the State to final resolution.
DOE recommends tracking to include: findings, including success stories, recommended corrective actions, deliverables, due dates, responsible parties, actions taken, and final resolution.
State will summarize and review each subgrantee's audit, program assessment reports and findings. Results should be considered during annual planning and available for review during DOE visits.
![Page 5: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/5.jpg)
Monitoring Approach
Part of State Plan On-File Section Overview of procedures and
guidelines for Grantee monitoring of local agencies
![Page 6: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/6.jpg)
Monitoring Procedures Overview
Goals of Monitoring Guiding Principles Desktop Reviews Subgrantee Monitoring Visits Monitoring Procedures Local Agency Monitoring
Responsibilities Agency Discipline/At-Risk Policy
![Page 7: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/7.jpg)
Goals of Monitoring
Analyze whether best possible program services are being delivered to low-income population
Determine program compliance and accountability
Analyze program performance Analyze quality of work
![Page 8: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/8.jpg)
Goals of Monitoring
Identify problems, deficiencies, and areas that need improvement and advise on how to correct
Assist agencies in their program operations and compliance with DOE and State regulations
Assess T&TA needs Be a major tool for program
improvement
![Page 9: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/9.jpg)
Guiding Principles
Intended to be constructive process, conducted with consistency, fairness, respect, and timeliness
Foster positive, open, and constructive working relationships
Two-way process to promote interaction, feedback, and improvements to Grantee and Subgrantee
![Page 10: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/10.jpg)
Guiding Principles Understanding that Grantee and
Subgrantee share the same goals of program effectiveness and improvement
Promote improvement by providing technical assistance, reinforcing strengths, and sharing successes, innovations, and practices seen at other agencies
Reports will be based on established policies, procedures and standards
![Page 11: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/11.jpg)
Types of Monitoring
Desktop Monitoring
Subgrantee Visits
![Page 12: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/12.jpg)
Desktop Monitoring
Not an agency visit Based on reported information to the State by
subgrantees Can save time of actual subgrantee visits by
preparing and organizing information in advance
Performance analysis Review of expenditures Certain compliance issues Identify specific focus areas for agency visit,
including problematic jobs
![Page 13: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/13.jpg)
Subgrantee Visits
Fiscal Monitoring Program Specific Administrative and
Management Review Field Procedures Monitoring and Job
Inspections
![Page 14: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/14.jpg)
Fiscal Monitoring
Often part of overall agency fiscal monitoring conducted by specialized Grantee fiscal monitor
ARRA may necessitate more focused WAP Fiscal monitoring
Usually a distinction between fiscal/procurement monitoring and specific WAP issues like job costing and reporting
![Page 15: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/15.jpg)
Administrative and Management Review
Review production, expenditures, and related compliance issues with agency
Review Client Files Review material, tool, equipment, and
vehicle property and inventory records Review client flow charts and scheduling Review safety policies and practices Review insurance policies Inspect warehouse
![Page 16: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/16.jpg)
Field Procedures Monitoring and Job
inspections
Analysis of quality of field practices, including Diagnostics Overall effectiveness Compliance with audit protocols Compliance with State Field Standards Workmanship Appearance Thoroughness Health and Safety issues
![Page 17: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/17.jpg)
Field Inspections
Will probably have several more field inspection visits that fiscal and administrative visits ARRA production levels will necessitate Frequent visits recommended
Should be a random selection of jobs within reason
![Page 18: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/18.jpg)
Monitoring Procedures
Agency visits are typically scheduled in advance by notification from Grantee Representative
Upon arrival, Monitor(s) should conduct entrance interview
Local WAP Coordinator should be available at all times during visit
Exit conference at conclusion of visit
![Page 19: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/19.jpg)
Monitoring Procedures
Report should be sent to subgrantee within 30 days (preferably sooner)
Agency has to submit corrective actions, if necessary, within specified time frame
Grantee reviews agency response Grantee writes agency to note
acceptance of response or any unresolved issues
![Page 20: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/20.jpg)
Best Practices and Tips What We’ve Learned
Be constructive – conduct yourself professionally, be fair, treat agency personnel with respect, be on time
Never monitor with “I gotcha” intentions
Always act as a partner and team player – after all, you and the agency should share the same goal of program improvement
![Page 21: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/21.jpg)
Best Practices and Tips What We’ve Learned
Help find solutions to problems Understand the agency and local
program environment and set realistic expectations
Offer constructive criticism based on your knowledge, experience, and the fact that you have observed other agencies that have dealt with similar circumstances
![Page 22: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/22.jpg)
Best Practices and Tips What We’ve Learned
Remember – monitoring reports are often the only assessment by State an agency receives
Should be a thorough assessment that also recognizes good practices, not just deficiencies
Don’t be afraid to pat someone on the back, they are in the Weatherization Program and probably need and deserve it
![Page 23: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/23.jpg)
Best Practices and Tips What We’ve Learned
State needs to have clear, printed up-to-date administrative and field standards to base monitoring
States need to tell subgrantee in advance when you are coming, what you will be doing, and what documentation will be needed
![Page 24: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/24.jpg)
Best Practices and Tips What We’ve Learned
Monitor(s) should draft reports (or at least an outline of findings) and conduct exit interview so agency is not taken off guard by report
Talk to clients – assess their satisfaction, ask if they can tell difference in comfort and/or fuel bills, but be careful about discussing findings with them
![Page 25: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/25.jpg)
Best Practices and Tips What We’ve Learned
Define major and minor findings – Focus on major findings
Major finding – significant concern, such as misuse of funds, eligibility issue, major health and safety issue, consistently failing to follow rules or standards
Minor finding – Smaller concern, such as minor file omissions, quick fixes, doesn’t significantly impact overall results
![Page 26: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/26.jpg)
Best Practices and Tips What We’ve Learned
Make monitoring flow between visits – do follow-up of last visit Document progress from last visit Confirm whether cited deficiencies
were corrected Note areas of improvement
![Page 27: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/27.jpg)
Best Practices and Tips What We’ve Learned
Do frequent monitoring – agency tends to be more responsive to findings and suggestions
Consider team monitoring – can cover a lot more ground and can capitalize on individual strengths
Focus on issues – not people Be tough on issues, but (where possible)
sensitive to people
![Page 28: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/28.jpg)
Best Practices and Tips What We’ve Learned
Get your field monitors digital cameras – pictures can be a valuable way to document and illustrate field work Make digital pics part of the exit
conference and report DOE Reg. – Subgrantee must perform
final inspection. Grantee monitoring should analyze effectiveness of subgrantee inspection process.
![Page 29: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/29.jpg)
Best Practices and Tips What We’ve Learned
Make monitoring part of the Training Program and use it as an opportunity to provide T&TA Include agency field staff in field monitoring Consider asking agency to select their best jobs
and their most problematic ones and visit with them
Have them set up blower door, do diagnostic tests, heating system tests, etc.o Analyze their expertise and practiceso See if equipment is calibrated and maintained
![Page 30: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/30.jpg)
Best Practices and Tips What We’ve Learned
Strive to be consistent between monitoring staff Try to insure your monitoring forms/tools are as
objective as possible Have monitors do “practice” monitoring together.
Inspect same files individually and then compare notes. Go to same house, inspect individually and compare findings and discuss.
Consider developing library of “common findings” so there is similar wording and explanations between monitors in reports
Formulate criteria that calls for agency returns to job sites
![Page 31: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/31.jpg)
Best Practices and Tips What We’ve Learned
The report is the written final word – make sure monitors have time to carefully write report
Remember that some of the readers of the monitoring report may not have the same expertise as the monitor.
Make sure monitoring reports are clear and objective, and major findings are listed first and most prominently.
Findings should include specific references to Weatherization Standards
![Page 32: State Monitoring 2010 NASCSP WAP Orientation Training Bob Scott Director of WAP Services NASCSP](https://reader036.vdocuments.us/reader036/viewer/2022062408/56649eb35503460f94bba8ba/html5/thumbnails/32.jpg)
Best Practices and Tips What We’ve Learned
Either report or cover letter must make clear corrective actions needed and possible consequences.
State Program Managers must insure their monitors are thorough and correct and then must support them.