interim findings and recommendation from the whole house process evaluation – sce contractor...
TRANSCRIPT
Interim Findings and Recommendation from the Whole House Process Evaluation – SCE Contractor Training Assessment
May 1, 2012
Presented to: Staff from SCE and PG&E
Prepared by: ASW EngineeringSBW Consulting, Inc.
Part Two of Today’s Presentations
SCE Contractor Training Assessment In-depth examination of SCE’s training
designs, methods and materials. Do they prepare contractor’s to succeed?
Context Rapid change in training underway, so
assessment looked at a moving target. Focused on SCE but there are wider
implications. Recommendations from the overall process
evaluation need to be considered before training is changed.
2May 1, 2012
Training Big Picture
Contractors must attend training to qualify Performance gaps – a training problem?
3May 1, 2012
Assessment Objectives
Effectiveness and appropriateness Do the training objectives address EUC
tasks?
4May 1, 2012
Assessment Methods
Reviewed program materials (program design, training and reporting)
Interviewed key personnel (RHA, ICF) Mapped training objectives to program
requirements Assessed discrepancies
5May 1, 2012
Assessment Criteria
How well the training and materials: Supported behavior change Included motivation to action Applied adult learning principles
6May 1, 2012
Adult Learning Principles and Practices
Learner orientation, buy in, engagement Learner success engineering Practice, application, and interactivity Learning objectives
7May 1, 2012
Effect of Teaching Styles and Average Retention Rates
8May 1, 2012
Delivery Mode and Average Retention Rates
9May 1, 2012
Practice and application are crucial to training effectiveness
Relating Learning Levels to Performance Outcomes
10May 1, 2012 Bloom’s taxonomy
11
Required Training
Participation Workshop Basic Path Training Advanced Path Training Online Learning Center Mentoring
May 1, 2012 11
Participation Workshop
2-hour session required for at least one person from each contractor – webinar or classroom
Orients contractors to goals, scope of work, marketing
Describes contractor qualifications, benefits
12May 1, 2012
Participation Workshop Findings
Good pitch of program Lacks details on:
Job processing expectations (Job Reporting, EnergyPro, documentation)
Level of rigor for QA/QC
13May 1, 2012
Participation Workshop Recommendations
More content on: EnergyPro requirements Exposure to job reporting
documentation and process Job reporting may change based on other
recommendations Refer to:
“Adopt Common Statewide Job Reporting” (Whole House Process Evaluation – PG&E_SCE Interim Report Workbook, Caption Tab, ID=13)
14May 1, 2012
Basic Path Training
Mandatory for Basic Path program Exception if BPI Building Analyst on staff
15May 1, 2012
Basic Path Training Findings
California Building Performance Contractor’s Association (CBPCA) 3-day course a good match-up between training content and program measures
Lacks: Formal objectives Adult learning principles
16May 1, 2012
Basic Path Recommendations
Incorporate: Worksheets More hands-on practice Ways to demonstrate proficiency (e.g.,
asbestos identification and insulation measurement)
Formal learning objectives Adult learning principles
17May 1, 2012
Advanced Path Training
EUC requires BPI Building Analyst certification
Certification can be obtained with no formal training
Certification does not guarantee EUC skill set
Dozens of third-party trainings offered
18May 1, 2012
Third Party BPI Training
Quality and depth of training ranges widely
0 to 12 days Classroom, online, self-trained Insufficient practice for EUC
fieldwork (performance gaps) Loose alignment with some
measures (air sealing, weatherization)
19May 1, 2012
Advanced Path Training Findings
CBPCA 12-day course provides: Solid foundation for Advanced Path work Strong combustion safety curriculum Some hands-on practice with diagnostic tools
Lacks: Formal training objectives Adult learning principles Descriptive text for photos
20May 1, 2012
Advanced Path Recommendations
For CBPCA 12-day course: Add training objectives Apply adult learning principles Add descriptive text to photos
Supplement BPI training with: More hands-on practice with field testing procedures,
protocols Practice of program-specific desk work (forms, energy
modeling) Remedial training for specific performance gaps
(asbestos, air sealing, etc.)
21May 1, 2012
Concerning HVAC Quality Installation (QI Program) and EUC
For minor HVAC repair work, BPI training is adequate
For major HVAC repair or replacement, then a permit is required and contractor must meet Title 24 quality standards
HVAC QI training addresses Title 24 code
22May 1, 2012
Online Learning Center – A New Resource
On-demand training on: JRT Administration Energy Modeling Job Processing Combustion Safety Testing
Mandatory participation; must pass exam
23May 1, 2012
Online Learning Center Findings
Well designed training Addresses some performance gap issues Incorporates adult learning features Useful for BPI certified contractors who did
not attend CBPCA 12-day course
24May 1, 2012
Online Learning Center Recommendations
Record exam scores for multiple employees
Expand topics: BPI and program standards Field inspection / testing Asbestos identification Insulation and weatherization
25May 1, 2012
26
Mandatory Mentoring
Required of one person from each contractor for both program levels
Three topics: Field Work Job Reporting/Job Processing Energy Pro
May 1, 2012
Mentoring Findings
Not formal training per se Mentoring includes:
Observation of contractor performance Evaluation and scoring Feedback
27May 1, 2012
Mentoring Recommendations
Standardize mentoring and evaluation protocols
QC firm should provide input Provide remedial training adapted to
individual needs, based on QA/QC findings Remote mentoring for Energy Pro and Job
Reporting
28May 1, 2012
29
Top 10 Performance Gaps
Performance Gaps Inaccurate insulation R-value Inability to identify asbestos Failure to achieve minimum air-sealing Inaccurate wall, window, and floor measurements Improper combustion safety testing (CO readings) Failure to identify combustion appliance hazardous
conditions (e.g., gas leaks) Failure to complete Basic Path measures for
Advanced projects
May 1, 2012
Top 10 Performance Gaps
Performance Gaps Inaccurate insulation R-value Inability to identify asbestos Failure to achieve minimum air-sealing Inaccurate wall, window, and floor measurements Improper combustion safety testing (CO readings) Failure to identify combustion appliance hazardous
conditions (e.g., gas leaks) Failure to complete Basic Path measures for Advanced
projects
30May 1, 2012
Top 10 Performance Gaps (continued)
Performance Gaps (continued) Information does not agree across documents (JRT,
Energy Pro, forms) Inaccurate duct leakage tests
Inaccurate assessment of insulation coverage and quality
Refer to: “Performance Gaps Observed in SCE Whole House
Training” (Whole House Process Evaluation Interim Report Workbook, Caption Tab, ID=74)
31May 1, 2012
Proposed Inspector-Trainer Role
Establish an internal inspector-trainer within each contractor’s staff—will be responsible for: Attending all required training Providing in-house training Ensuring quality of work
Suggestions: Collaborative effort between QA/QC (RHA) and mentors
(ICF) for training inspector-trainers Develop and provide train-the-trainer materials for
inspector-trainers
32May 1, 2012
Proposed Marketing and Sales Training
Finding Little marketing and sales content in current
training materials Recommendation
Suggest formal marketing and sales training focused on contractor sales force
Classroom opportunity to practice EUC program sales skills
Opportunity to apply adult learning principles
33May 1, 2012
From Interim Report - Ongoing Safety Training Findings
Safety issues are noted frequently and may be missed by contractors
34May 1, 2012
Safety Issue SCE/SCG (n=16) PG&E (n=51)Possible asbestos in ducts - noted by inspector 7 NAPossible asbestos in ducts - noted by contractor 3 2CAZ failure in Test-in 4 12CAZ failure in Test-out 6 3CAZ failure not corrected after Test-in failure 2 0Mechanical ventilation recommended or required - noted in pre-application
1 4
Mechanical ventilation recommended or required - noted in post-application
7 7
From Interim Report - Ongoing Safety Training (continued)
Recommendations Provide ongoing safety training to contractors. Based on
feedback from inspectors and QC Issues to emphasize, include:
Low house air leakage Possible asbestos in ducts CAZ testing Correct installation of insulation
Refer to: “Ongoing safety training” (Whole House Process Evaluation – PG&E_SCE
Interim Report Workbook, Recommendations Tab, ID=17) Relevant To
PG&E and SCE, BIG and ICFI / RHA
35May 1, 2012