state of wisconsin department of administration division...

State of Wisconsin Department of Administration Division of Energy Focus on Energy Statewide Evaluation

Business Programs: Lighting and Motor/Drive Channel Market Effects Contract Metrics Assessment

Final: November 10, 2006

Evaluation Contractor: PA Government Services Inc.

Prepared by: Ken Agnew, Paula Ham-Su, and Miriam Goldberg, KEMA Inc.

This report is the property of the State of Wisconsin, Wisconsin Department of Administration, Division of Energy, and was funded through the Wisconsin Focus on Energy program.

State of Wisconsin Department of Administration Division of Energy Focus on Energy Statewide Evaluation

Business Programs: Lighting and Motor/Drive Channel Market Effects Contract Metrics Assessment

Final: November 10, 2006 © PA Knowledge Limited 2006

Liaison Contact: Dr. David Sumi PA Government Services Inc. 6410 Enterprise Lane, Suite 300 Madison, WI 53719 Tel: +1 608 443 2700 Fax: +1 608 661 5181 E-mail: [email protected]

Prepared by: Ken Agnew, Miriam Goldberg, and Paula Ham-Su, KEMA Inc.

Acknowledgment: Ralph Prahl, Prahl & Associates, contributed critical review and analysis.

iii

Business Programs: Lighting and Motor/Drive Channel Market Effects Contract Metrics Assessment. Final: 11/10/06

TABLE OF CONTENTS

1. Executive Summary 1-1 1.1 Metrics Achievement Summary 1-1

2. Introduction 2-1 2.1 Market Effects Contract Metrics 2-1 2.2 Approach 2-1 2.3 Organization 2-2

3. Motor/Drive Channel Metrics 3-1 3.1 Summary of Metrics and Accomplishment 3-1 3.2 Metric Specifications 3-1 3.3 Data Collection 3-2 3.4 Motor/Drive Channel Metric #1: NEMA Premium-Efficiency

Shipments 3-3 3.5 Motor/Drive Channel Metric #2: Participating Dealer Premium-

Efficiency Share 3-3 3.6 Motor/Drive Channel Metric #3: Market Presence 3-6 3.7 Motor/Drive Channel Metric #4: Large Participating Dealers

Premium-Efficiency Stock 3-11 3.8 Motor/Drive Conclusions 3-12

4. Lighting 4-1 4.1 Summary of Metrics and Accomplishment 4-1 4.2 Metric Specifications 4-1 4.3 Data Collection 4-2 4.4 Lighting Channel Metric #1 4-2 4.5 Lighting Channel Metric #2 4-5 4.6 Lighting Conclusions 4-7

5. Pulp and Paper Industrial Cluster 5-1 5.1 Summary of Metrics and Accomplishment 5-1 5.2 Metric Specifications 5-1 5.3 Approach 5-2 5.4 Detailed Findings 5-3

6. Conclusions 6-1 6.1 Motor/Drive and Lighting Channels 6-1 6.2 Pulp and Paper Industrial Cluster 6-1

TABLE OF CONTENTS…

iv


Appendices

APPENDIX A: Motor/Drive Dealers/Distributors Questionnaire A-1

APPENDIX B: Lighting Contractor Survey Instrument B-1

APPENDIX C: List of Pulp and Paper Mills in the State of Wisconsin C-1

APPENDIX D: Pulp and Paper Mill Survey Instrument D-1

1-1


1. EXECUTIVE SUMMARY

This report provides the Focus Business Program market effects contract metrics assessment for fiscal year 2006 (FY06) for:

• The Motor/Drive Channel

• The Lighting Channel

• The Agriculture Program

• The Pulp and Paper Industry Cluster.

The metric goals are contractual requirements for the program administrator. The goals were included in Business Program’s strategy matrices in the Business Program administrator’s contract amendment for the FY06 funding cycle.

1.1 METRICS ACHIEVEMENT SUMMARY

1.1.1 FY06 Metrics Goal Achievement—Motor/Drive Channel

Four motor/drive channel contract metrics were proposed for FY06. Three are measured in this report and met their goals. All three metrics are based on change from FY05, the year prior to the official start of channel activities. Evaluation of a fourth market effects contract metric for the motor/drive channel for FY06 will not be possible until the next round of data become available later in the fall of 2006. Only one of the contract metrics includes a numerical metric goal. The evaluation team has used this numerical goal as the critical level for all four metrics.

Table 1-1 lists the market effects contract metrics for FY06 for the motors/drive channel. The table also indicates the status with respect to meeting the metric goals.

Table 1-1. Motor/Drive Channel Metrics

Metric # Critical Goals Critical MetricsCritical Level1

Metric Goal Achieved?

1Sales growth of NEMA premium motors and drives sold

Market sales growth index based on state-level NEMA Premium motors shipment data in relation to national sales growth index.

1%Data available,

fall 2006


Premium motor market share increase of 1% among participating dealers based on participant self-reported sales data or ratios.

1% Yes

3

Increase the market presence of Focus within dealer network

Market presence indicator of dealers working with Focus, based on participating dealer employment. Compare to FY05 participating dealer data and/or evaluation results.

1% Yes

4

Increase stocking and use of NEMA premium motor in a range of sizes at time of failure

Increase in the stock of NEMA premium motors, in a range of sizes, by large participating motor distributors w/>=15 employees or >1 branch. 1% Yes

1 Where level is not specified explicitly in Critial Goals or Critical Metrics, the Critical Level is an extrapolation by the evaluation team.

1. Executive Summary…

1-2


1.1.2 FY06 Metrics Goal Achievement—Lighting Channel

Two lighting channel contract metrics were proposed for FY06. Both metrics are measured in this report. Metric two, the market share metric, reached its critical level. Both metrics are based on change from FY05, the year prior to the official start of channel activities.

Table 1-2 lists the market effects contract metrics for FY06 for the motors/drive channel. The tables also indicate the status with respect to meeting the metric goals.

Table 1-2. Lighting Channel Contract Metrics

Metric # Critical Goals Critical MetricsCritical Level


1Market presence indicator of lighting contractors working w/Focus, based on participating contractor employment. 5% No

2

Increase ratio of Focus prescriptive technology sales to standard efficiency alternative among participating contractors, based on participant self-reported sales data or ratios.

5% Yes

Increased program participation by low and mid-sized contractors as categorized by the market potential study by 5% from FY05

1.1.3 FY06 Metrics Goal Achievement—Agriculture Sector

The proposed Agriculture sector contract metrics were to be based on questions included in third-party surveys. Evaluation’s role was limited to working with the program to develop the specific questions. The program has been working with Wisconsin Agricultural Statistics Service and Hoard’s Dairyman Magazine to get the questions included in their standing surveys. This work is still in process, thus, this metric was not measured.

Table 1-3. Agriculture Sector Contract Metrics



1

Increase the priority for EE by AG customers as determined by evaluation survey

Establish baseline of AG customer capital investment priorities from previous research and measure change in priority.

Not Measured

1.1.4 FY06 Metrics Goal Achievement—Pulp and Paper Industrial Cluster

A single contract metric was proposed for FY06 for the pulp and paper industrial cluster. The pulp and paper contract metric does not include a numerical metric goal. However, this metric is measured and passed as a positive effect.

1. Executive Summary…

1-3


Table 1-4. Industrial Cluster—Pulp & Paper Contract Metrics



1

Measure rate of preliminary Adoption of the Best Practices by the Wisconsin Paper Council, the Lake States Technical Association of Pulp & Paper Industry and Focus participating Companies

Number of groups, experts, and customers utilizing or endorsing the Best Practices in Pulp & Paper Industry.

Yes

2 No critical level is provided or implied.

1.1.5 Ongoing Metrics Tracking

A. MOTOR/DRIVE AND LIGHTING CHANNELS

Two of the motor/drive channel metrics and one of the lighting channel metrics are share-oriented metrics. This assessment of FY06 metrics used retrospective data collected from survey respondents. The evaluation of the share-oriented contract metrics for FY07 will provide a more rigorous challenge for the channels. They will be based on longitudinal analysis as well as retrospective questions. Under the present plans, a next round evaluation of these metrics will take place for the lighting channel after the end of FY07. We recommend continuing evaluation of the share-oriented contract metrics for the motor/drive channel as well.

Each channel also had a market presence index. The market presence metrics compared participating dealers or contractors to the population of potential dealers or contractors. The market presence index is problematic and needs further development for FY07. Under the present plans, a next round evaluation of a market presence metric will take place for the lighting channel after the end of FY07. We recommend that a market presence indicator also be evaluated for FY07 for the motor/drive channel.

The channel market effect contract metrics should include explicit numerical metric goals by which to evaluate metric achievement.

B. AGRICULTURE SECTOR

This metric is still in the development stages. Ongoing evaluation will depend on the initial survey results when it is completed.

C. INDUSTRIAL CLUSTER—PULP & PAPER

The pulp and paper industry survey establishes baseline values across a range of indicators developed using the Pulp and Paper Energy Best Practice Guidebook. This data can be updated with the fielding of a similar survey. The baseline values will facilitate the measurement of Guidebook-related market change. Further evaluation of this cluster for the FY07 program year is slated for the beginning of FY08. Explicit, numerical metric goals should be included by which to evaluate metric achievement.

2-1


2. INTRODUCTION

This report provides the Focus Business Program market effect contract metrics assessment for fiscal year 2006 (FY06). The metric goals are contractual requirements for the program administrator. The goals were included in Business Program’s strategy matrices in the Business Program Administrator’s contract amendment for the FY06 funding cycle.

2.1 MARKET EFFECTS CONTRACT METRICS

Contract metrics include both operational metrics that can be tracked by the program and verified by evaluation, and market effects metrics. On the operational side, evaluation’s primary role is to verify gross savings tracked by the program. Other operational metrics are reported by the programs based on their own tracking and are not verified by evaluation. Market effects metrics relate to program effects that are not directly tracked by the program, and generally require data collection and analysis by evaluation. This report provides the evaluator’s assessment of the market effects contract metrics for FY06 for the Business Programs motors and lighting channel strategies and the pulp and paper industrial cluster.

The Focus on Energy Business Program channels was initially conceived as a market transformation effort. This is why the motor/drive channel and lighting channel have market effects contract metrics. In practice, for FY06, the channels had tracked savings and savings-related goals. In particular, the channels were identified with prescriptive rebates. At the same time, the programs continue to view these efforts as part of a strategy to change the broader market beyond the effects measured in terms of program rebate activity.

Similarly, the pulp and paper industrial cluster’s development and marketing of the Pulp and Paper Energy Best Practice Guidebook is viewed as part of a strategy to change the broader market.

The purpose of this report is the analysis of these market effects contract metrics. The goal of this analysis is the design and implementation of quantitative processes that will allow the measurement of these contract metrics now and in future evaluations.

2.2 APPROACH

The scope of the FY06 contract metrics evaluation was discussed in Section 2.3 of the Business Programs: Contract FY06—Detailed Evaluation Plan finalized on November 22, 2005. The verbatim metrics addressed below are included there as well as further guidance regarding the evaluation of these metrics.

The first step in the analysis of the contract metrics is the explicit interpretation of each metric on terms that facilitate the evaluation. Metric interpretation is particularly important in three respects:

• The specific relevant population to target

• The technologies included in the measurement

• The specific measurement to make.

2. Introduction…

2-2


This interpretation process is prior to and distinct from the analysis methodologies that are used to implement the assessment. It is essential to understand the metric interpretation to be able to understand the results that are produced from the analysis. That is, the reader needs to know the specific “question” before the reader will understand the “answer.”

The analysis methodologies used for the metrics assessment for channels are twofold.

1. We fielded surveys of members of both the motor/drive and lighting relevant populations to learn about changes in shares of energy-efficient equipment. On these surveys, we asked respondents about sales volume and, in the case of motor dealers, stock shares.

2. We compared participating dealers and contractors to market-wide lists to measure the program’s market “presence.” That is, what proportion of the total motors and lighting markets in Wisconsin are being “touched” by Focus.

In addition, for the motors/drive channel a metric has been evaluated based on market sales data provided by the motor manufacturers. This metric was evaluated and reported on previously; findings are summarized here.

The pulp and paper metrics primary assessment methodology is a survey fielded of active pulp and paper mills in Wisconsin designed to quantify the use and impact of the Guidebook after one year.

2.3 ORGANIZATION

This report is organized by channel/cluster.

2.3.1 Motor/Drive and Lighting Channels

For the two channels, there is a discussion of the targeted market players and the included technologies. These two aspects of the metric interpretation are consistent across the multiple metrics for each channel.

For each metric, an introductory section clarifies the specific measurement to be made. This section is designed to give the reader the necessary information to understand the results. This is followed by a summary of the findings for that metric.

Each metric section is then rounded out with a full discussion of the analysis methodology used to produce the results for that metric. A more extensive display of the findings for that metric is also included. For each channel, there is a section summarizing the overall channel findings.

2.3.2 Pulp and Paper Metrics

The pulp and paper section of the report presents the results of the survey of pulp and paper mills regarding the adoption of the Pulp and Paper Energy Best Practice Guidebook.

3-1


3. MOTOR/DRIVE CHANNEL METRICS

3.1 SUMMARY OF METRICS AND ACCOMPLISHMENT

Table 3-1 provides a list of the motor/drive channel metrics for FY06. The table also indicates whether the metric was achieved, failed, or postponed.

Table 3-1. Motor/Drive Channel Metrics




Market sales growth index based on state-level NEMA Premium motors shipment data in relation to national sales growth index.

1%Data available,

fall 2006


Premium motor market share increase of 1% among participating dealers based on participant self-reported sales data or ratios.

1% Yes

3

Increase the market presence of Focus within dealer network

Market presence indicator of dealers working with Focus, based on participating dealer employment. Compare to FY05 participating dealer data and/or evaluation results.

1% Yes

4

Increase stocking and use of NEMA premium motor in a range of sizes at time of failure

Increase in the stock of NEMA premium motors, in a range of sizes, by large participating motor distributors w/>=15 employees or >1 branch. 1% Yes

1 Where level is not specified explicitly in Critial Goals or Critical Metrics, the Critical Level is an extrapolation by the evaluation team.

Metric #1 is based on shipment data that will not be available until FY07. The evaluation methodology has been developed and tested on previous year data. This work was provided in an earlier memo and will only be briefly summarized here. This report directly addresses metrics #2 through #4. Only one of the contract metrics includes an explicit numerical metric goal. The evaluation team has used this goal as the critical level for all four metrics.

Metric #3 is not included in the FY07 contract metrics. The specific metric developed here has methodological problems for ongoing tracking. However, we recommend further work to develop and track a measure of market presence if there is substantial market transformation activity in the motor channel.

Metric #4 is also not currently included in the FY07 contract metrics. We recommend continuing to track the metric developed here if there is substantial market transformation activity in the motor channel.

3.2 METRIC SPECIFICATIONS

3.2.1 Target Population

The three motor/drive metrics addressed in this report (#2–#4) target either participating dealers or distributors. There is overlap between the definitions of dealer and distributor. We identify a single set of motor and drive market players for all three metrics. We targeted distributors that deal directly with contractors, installers, or end users. This approach offered a balance between maximizing the coverage of the motor and drive markets while avoiding possible double counting of the same stock at multiple points in the supply chain. For simplicity’s sake, we will use the term dealer when referring to the group we targeted.

3. Motor/Drive Channel Metrics…

3-2


Participating dealers were identified in two ways. The motor/drive channel provided KEMA with a list of motor/drive dealers with whom they have worked on channel activities. This represented a group of direct channel participants.

A second group of motor/drive participants was identified in WATTS, the Focus tracking database. We identified dealers that sold a motor or drive that received a prescriptive rebate. Dealers that provided units to end users receiving a prescriptive rebate may not have been “working with” the channel. They might not even have realized a rebate was received on a motor they sold. Regardless, they did participate in an important aspect of channel activities, the prescriptive rebate process. To this end, we considered it necessary to include these dealers in the market assessment as well. These dealers were identified in WATTS with measures implemented during FY05 and FY06 through April 21, 2006.

3.2.2 The Technologies

The motor/drive channel works in both the motor and drive markets. Only two of the metrics, #2 and #4, are technology-specific and they clearly target NEMA premium motors. Metric #3, the market presence metric, is not technology-specific. In this case, because dealers of drives are included in the sample, the market presence metric does cover the combined motor and drive market.

All four metrics focus on motors and drives ultimately sold directly to the end user. There is a large market for motors that is included as a component in a larger piece of equipment. These original equipment manufacturers (OEMs) are a distinct market from motor end users. Focus on Energy does not rebate motors going to OEMs, so they are not part of the motor/drive channel efforts. Where possible we gathered data only about the non-OEM sales of the motor dealers with whom we spoke. OEMs usually buy from the biggest distributors as they buy in bulk and price is a high priority. The larger distributors were generally able to provide responses based on non-OEM sales alone.

For metrics #2 and #4, we divided motors into three size groups, 1 to 5 horsepower (HP), 7.5 to 20 HP, and greater than 20 HP.

3.3 DATA COLLECTION

Data used in this analysis included:

• Shipments of NEMA premium-efficiency motors by state and year from NEMA (Metric #1 only)

• Lists of participating suppliers provided by the Channels and extracted from WATTS

• Survey data from participating suppliers

• Dun & Bradstreet data on businesses and employment in industries that supply motors in Wisconsin.

Information collected on the supplier surveys that was used for this analysis included:

• Percentage of sales volume that was NEMA premium in FY05 and FY06

• Percentage of NEMA premium sales in each size category


3-3


• Percentage of stock that was NEMA premium in FY05 and FY06.

Data was collected by phone interview in May and June 2006. The number of dealers for which we obtained data in the interviews and the total number in the combined channel and WATTS lists are summarized in Table 3-2. For the motor channel, data were obtained for the large majority of participating dealers.

Table 3-2. Motor/Drive Channel Survey Sample Size and Completes

Participating Locations 26

Participating Companies 22

Surveyed Companies 15

Percent Surveyed 68%

3.4 MOTOR/DRIVE CHANNEL METRIC #1: NEMA PREMIUM-EFFICIENCY SHIPMENTS

Market sales growth index based on state-level NEMA premium motors shipment data in relation to national sales growth index

The market sales growth index uses NEMA premium motors shipment data for Wisconsin and the rest of the country to provide a growth index that controls for national sales trends and economic conditions. The percentage increase in Wisconsin is compared to the percentage increase across the remainder of the country. The resulting ratio indicates the percentage change in Wisconsin net of change at the national level.

NEMA provides the data annually. The data for a comparison of FY06 with FY05 will not be available until FY07. This will be the first year that both years (prior and current) of NEMA data will have OEM motor sales removed. The methodology for estimating this metric has been developed and tested, as described in the memorandum “Focus on Energy Public Benefits Evaluation BP motors metric preliminary assessment” (M. Goldberg to O. Bloch, December 30, 2005).

3.5 MOTOR/DRIVE CHANNEL METRIC #2: PARTICIPATING DEALER PREMIUM-EFFICIENCY SHARE

Premium motor market share increase of 1 percent among participating dealers based on participant self-reported sales data or ratios

3.5.1 Measurement

The motor/drive channel metric #2 measures market share. Market share was defined as the sales volume of NEMA premium motors as a percentage of the sales volume of all motors. Market share increase was defined as the difference between the two market share percentages from consecutive years. The increase is expressed as a percentage of the market.

Survey respondents were asked what share of sales within each size category were NEMA premium motors. These questions were asked for both FY06 and FY05. In addition,


3-4


respondents were asked to break out their total NEMA premium sales into the three size categories. The market share for the three motor groups was estimated as:

Equation 3-1

1 1

( 06 05 )n n

g mgi i i i ii i

mgi d w dMS P P w= =

−= ∑ ∑

where:

MSg = market share for size group g

P06mgi and P05mgi = market share percentage of NEMA motors in size group g in FY06 and FY05, respectively

wi = sampling weight (essentially, the number of units in the population represented by each unit in the completed sample)

di = measure of company size, in this case employment.

A single, overall motor metric was created by combining the three size groupings. The group-level market shares were combined on a horsepower-weighted basis.

Equation 3-2

1 1

( 06 05 )n n

mi i i i ii i

mi d w dMS P P w= =

−= ∑ ∑

Given:

Equation 3-3

3 3

1 1

06 05 ( 06 05 )mi mgi g gi g gig g

mi mgi f H fP P P P H= =

− = −∑ ∑

where:

Hg is the midpoint of the horsepower range of the size category and

fgi is the share of the respondent’s NEMA sales in that size category.

3.5.2 Summary of Findings

Table 3-3 below shows the average reported change of market share of NEMA premium motors from FY05 to FY06. On average, dealers reported an increase of 2 percent in sales of premium motors across all size categories. This increase was statistically significantly greater than zero. (That is, the lower bound of a one-sided 95 percent confidence interval is greater than zero.) We therefore conclude that this metric was achieved.


3-5


Table 3-3. Change in Non-OEM NEMA Premium Motor Market Share FY05 to FY06 for Participating Dealers

Motor Size CategoryNumber of

Respondents1

Market Share % Change,

FY05 to FY06Standard Error of % Change

95% Confidence

Lower Bound

All motors 14 2.0 0.8 0.61 One respondent did not provide FY05 Market share for calculation of % change

3.5.3 Approach

This section provides further background on the survey of motor dealers from which the data is drawn for this metric.

MOTORS/DRIVES SURVEY

Population. The combined list of participating motor dealers from the motor channel and from WATTS included 22 companies with 26 locations. It proved easiest for companies to respond to the survey for all of the relevant Wisconsin, Focus-related locations at one time. Thus, the effective population for the survey questions was 22 companies. We expected to talk to a sufficiently large number of the dealers that stratification was unnecessary.

Instrument. KEMA developed a survey instrument with the primary goal of measuring sales and stocking levels related to these metrics. Metrics-related questions were asked separately for the three different size categories of NEMA premium motors. The full motors/drives survey instrument is attached in Appendix A.

Analysis of results. We completed surveys with 15 out of 22 of the companies selected for this analysis. The market share results were calculated using Equations 1 through 3 above. These equations employ a ratio estimator approach with employment as an indicator of size. That is, we weighted each company’s response by the company’s relative size in the market. This calculation approximates an average market share over all motors sold by participating dealers. While we would have preferred to weight by total motor sales at each company, the majority of companies were unwilling to provide this information.

3.5.4 Detailed Findings

Table 3-4 provides the metric results by size group. For all three size groups, the percentage point changes are statistically significantly greater than zero. The results indicate that the large motor group has the highest percentage point increase and, consistent with the HP weighting, drives the “all motors” results.


3-6


Table 3-4. Change in Non-OEM NEMA Premium Motor Market Share FY05 to FY06 for Participating Dealers, by Size Group

Motor Size Category

Number of Respondents, FY06 Market

Share FY06 Market

Share %

Number of Respondents,

Market Share % Change1


FY05 to FY06

Standard Error of % Change

95% Confidence

Lower Bound

1 to 5 Hp 15 45.2 14 1.2 0.5 0.4 7.5 to 20 Hp 15 47.4 14 1.2 0.5 0.4

25 Hp or more 15 51.1 14 2.3 0.9 0.6All motors n/a 14 2.0 0.8 0.6

1 One respondent did not provide FY05 Market share for calculation of % change

This table also includes estimates of the FY06 NEMA premium market share for the three size groups. For participating motor dealers, almost 50 percent of non-OEM motor sales are NEMA premium. This number is consistent with results reported in the 2004 Supply Chain Characterization and Baseline Study produced for Focus on Energy Business Programs. In that study, estimated market shares for NEMA premium motors in Wisconsin for all dealers (not restricted to program participants) were 21, 41, and 51 percent, respectively, for the three size groups small to large. The participating dealers would be expected to have a higher market share of NEMA premium motors than the market as a whole.

3.6 MOTOR/DRIVE CHANNEL METRIC #3: MARKET PRESENCE

Market presence indicator of dealers working with Focus based on participating dealer employment. Compare to FY05 participating dealer data and/or evaluation results.

3.6.1 Measurement

The motor/drive channel metric #3 describes an index that can be compared year to year to measure changes in channel program presence in the market. Specifically, the market presence index is defined as the number of employees at businesses “working with Focus” as a percentage of all employees working for businesses that could potentially work with Focus through this channel. The metric is interpreted as stipulating an increase from year to year, in this case, from FY05 to FY06.

The intent of this metric is to indicate how much of the motor sales market Focus is reaching. In principle, the programs could do well in terms of the efficient shares among participating dealers (metric #2) by concentrating efforts on a very limited number of enthusiastic dealers. The market presence indicator balances the participant share metric by measuring the breadth of activity.

An ideal market presence indicator would be total motor sales by participating dealers as a fraction of all motor sales in the state. However, neither participating nor total motor sales data are available. As a proxy, we use total employment by participating dealers and total motor-sales-related employment in the state. However, even this proxy is problematic because we do not have a good measure of motor-sales-related employment.

The methodology we employ defines a population of potential participating dealers and then matches the actual participants with the list of potential participants. The potential participant population and thus, the matching are limited to motor-related SIC categories from Dun & Bradstreet (D&B).


3-7


The indices are simply the ratio between the matched participants and the full population from D&B. In addition to employment, we look at number of sites. The indices are not statistical estimates based on samples, and thus have no associated sampling error. However, there are limitations to these indices due to the nature of the data available for their computation, as discussed above.

The market change indices are intended to represent a measure of activity that can be repeated consistently year to year. While the resulting indices are suited to measuring changes in market presence, the index levels themselves are not necessarily representative of true market presence and are only reported here for comparison of index level across subcategories.

For this metric, the population of participating dealers is the same combined list of motor dealers used in metrics #2 and #4. This is the combination of channel contacts and dealers that sold motors rebated by Focus. For this metric, we used the channel sample at the site level. The population of potential participating businesses includes all Wisconsin businesses that sell or install motors and VFDs.

For FY05, there were no contacts provided by the channel. The program database, WATTS, indicates that 13 motors or VFD measures were implemented in FY05 (compared to the 52 measures implemented in FY06). Of these prescriptive projects, only one included the distributor ID, the dealer ID we used to identify motor dealers for FY06. The FY05 results are based on the single dealer ID available from the FY05 data.

We provide the observed levels in FY06 and FY05 as well as the change from FY05. This change is some combination of the change in tracking procedures and changes in the program’s approach to the market. The change from FY06 to FY07 will be the first true measure of the effect of that approach in terms of this index.


Table 3-5 shows the market presence indices of the FY05 and FY06 motor/drive channel participants in terms of percentage of businesses and percentage of employment. The table also provides the difference in indices between FY05 and FY06.

Of 26 participating sites, 14 were matched to businesses in motor and drive-related 8-digit SIC categories. The overall market presence based on these matches is 2.2 percent in terms of employment and 1.7 percent in term of numbers of sites. Taking the metric literally, we must consider this metric to have been achieved. However, the relatively small fraction of sites that could be matched in the population and the inability to define the relevant population tightly make this index problematic for ongoing tracking of program breadth.

Table 3-5. Increase in Market Presence of Participating Motor and Drive Dealers, by Number of Locations and Site Employment, FY05 to FY06

D&B Population

Matched Participants Percent

D&B Population


Total/Avg FY05 776 1 0% 7418 60 1%Total/Avg FY06 776 14 2% 7418 223 3%Change in Percent (Difference) 2% 2%

Motor Dealer Category by 8-Digit SIC Number

Number of Locations Employment


3-8


3.6.3 Approach

A market-based index is only as good as its definition of the market. We measured market presence by comparing the list of participating channel dealers with a full list of motor dealers in Wisconsin as identified in the D&B database. D&B maintains a database of businesses in the U.S. categorized with eight-digit SIC codes. We defined the population for this analysis using the eight-digit SIC codes identified by the 2004 Supply Chain Characterization and Baseline Study (see Table 3-6) as well as an additional SIC code representing motor repair shops. We also limited the population to businesses sited in Wisconsin. The advantage of this approach is that D&B provides an independent identification of the motor-related population, and that population changes over time to reflect the actual market. For this first year metric evaluation, we must use this year’s D&B population for FY05 as well as FY06.

We match the list of participating dealers with the list of dealers generated from the D&B database. The matched set of participants represents a percentage of the whole motor-related population. This percentage is the basic market presence index. Along with the number of sites matched, the accompanying employment data provides an additional index. The indices are calculated as follows:

Equation 3-4

MPb = np/Ndb

where:

MPb is a location-based market presence measure,

np is the number of participating dealers, and

Ndb is the number of dealers in the relevant SIC codes in D&B.

D&B provides a count of employees. Hence, the ratio can be done as a function of employment as well as number of locations. For example, the equation for the employment-based market presence measure is:

Equation 3-5

MPe = ep/Edb

where participant (ep) and population (Edb) employment numbers come from D&B.

Future measurements of both ratios representing new lists of participating dealers will be directly comparable to this year’s measures but using current D&B data. The D&B population is updated regularly but represents a consistent definition of the population of potential motor and drive dealers.

This approach has some issues that must be taken into consideration when considering the results.

Just over half of the participating dealers were found in the motor dealer population identified in D&B. This reflects our choice to limit both the population and the matching process to


3-9


certain criteria. With regard to the identification of the population, we focused on motor dealers to avoid a population (the denominator of our ratio) that would swamp the participants. We only included businesses in the population that indicated the motor related eight-digit SIC codes as their primary SIC codes. Many of the participating motor dealers do business in other areas and are thus categorized elsewhere. If we had expanded the population to include all the categories into which participants fell, it is likely the market presence indices would have been smaller as the SIC codes with the highest density of participants were already included. The same explains the exclusion of dealers from outside of Wisconsin. Any site located outside of Wisconsin was not matched.

With regard to the matching process, the criteria we used also limited the number of matches. When matching participants to the D&B population, both the name and the location had to be the same to make the match. We were flexible to reasonable name variations if the location was correct. For multiple branches, however, we only made the match if the locations matched. The reason for this is that branches may operate quite independently so that Focus involvement at one location has little to do with the others.

An additional smaller scale limitation inherent in the D&B database is missing data. D&B stakes its reputation on providing accurate data. When it cannot get an accurate measure of employment, it will leave that figure blank. Thus, the employment data with which we create the employment indices may reflect some missing data. However, because the market presence measure is a ratio of two numbers from D&B, we expect, on average, a similar amount of missing data in the denominator (D&B population) as the numerator (participants matched with D&B). Essentially, the missing data increases the variation around the estimated market presence, but it should not bias the result. The location-based index does not suffer from missing data.


The FY05 and FY06 market presence indices for individual eight-digit SIC sub-categories are provided below in Table 3-6 and Table 3-7. The eight-digit categories show where the motor dealer population was located. The “Matched Participants” field indicates in which categories the participating motor dealers were matched. The FY05 results reflect the activity of only one motor dealer. The results are reported here primarily as an example of how the indices work. Table 3-8 shows the change in the market presence indices. For the one motor dealer category with activity in FY05, “Motors, Electric,” the employment index increase nine percentage points.

The table shows that many of the SIC codes that include motor sales activity are broad and have substantial other activity as well. This difficulty identifying “relevant” motors employment from D&B with the small proportion and absolute number of participants matched to this database makes this index problematic. We recommend exploring further refinements before relying on the market presence indicator.


3-10


Table 3-6. Market Presence of Participating Motor and Drive Dealers, by Number of Businesses and Site Employment FY05

D&B Population


D&B Population


5063-0000 Electrical apparatus and equipment 137 0 0% 1,414 0 0%5063-9905 Motors, electric 36 1 3% 655 60 9%5084-0000 Industrial machinery and equipment 329 0 0% 3,585 0 0%5085-0000 Industrial supplies 185 0 0% 1,367 0 0%5085-0501 Bearings 20 0 0% 108 0 0%7694-0000 Armature rewinding shops 6 0 0% 23 0 0%7694-0100 Rewinding services 1 0 0% 1 0 0%7694-0200 Motor repair services 8 0 0% 24 0 0%7694-0201 Electric motor repair 52 0 0% 238 0 0%7694-0202 Rebuilding motors, except automotive 2 0 0% 3 0 0%All Categories 776 1 0% 7418 60 1%


EmploymentNumber of Locations

Table 3-7. Market Presence of Participating Motor and Drive Dealers, by Number of Businesses and Site Employment FY06

D&B Population


D&B Population


5063-0000 Electrical apparatus and equipment 137 0 0% 1,414 0 0%5063-9905 Motors, electric 36 6 17% 655 120 18%5084-0000 Industrial machinery and equipment 329 0 0% 3,585 0 0%5085-0000 Industrial supplies 185 4 2% 1,367 54 4%5085-0501 Bearings 20 1 5% 108 10 9%7694-0000 Armature rewinding shops 6 0 0% 23 0 0%7694-0100 Rewinding services 1 0 0% 1 0 0%7694-0200 Motor repair services 8 0 0% 24 0 0%7694-0201 Electric motor repair 52 3 6% 238 39 16%7694-0202 Rebuilding motors, except automotive 2 0 0% 3 0 0%All Categories 776 14 2% 7418 223 3%


EmploymentNumber of Locations

Table 3-8. Change in Market Presence Percentage of Participating Motor and Drive Dealers FY05 to FY06, by Number of Businesses and Site Employment

Motor Dealer Category by 8-Digit SIC Number5063-0000 Electrical apparatus and equipment 0% 0%5063-9905 Motors, electric 14% 9%5084-0000 Industrial machinery and equipment 0% 0%5085-0000 Industrial supplies 2% 4%5085-0501 Bearings 5% 9%7694-0000 Armature rewinding shops 0% 0%7694-0100 Rewinding services 0% 0%7694-0200 Motor repair services 0% 0%7694-0201 Electric motor repair 6% 16%Total/Avg 2% 2%



3-11


3.7 MOTOR/DRIVE CHANNEL METRIC #4: LARGE PARTICIPATING DEALERS PREMIUM-EFFICIENCY STOCK

Increase in the stock of NEMA premium motors in a range of sizes by large participating motor distributors with 15 employees or more or more than one branch

3.7.1 Measurement

Motor/drive channel metric #4 focuses on the premium motor stocking practices of larger distributors. Larger distributors are those with 15 employees or more, or that have more than one branch. The stock share of NEMA premium motors is defined as the stock of the NEMA motors as a percentage of stock of all motors. The process for determining stocking share follows that used for the market share used in Metric #2.


Table 3-9 below shows the average reported change of stocking share of NEMA premium motors from FY05 to FY06 for the companies that fit the criteria to be larger distributors. On average, dealers reported a 2.9 percent increase in the stock of premium motors across all size categories. This increase was statistically greater than zero using a one-side T-test at a 95 percent confidence level. The results indicate an increase in the share of NEMA premium motors, thus we conclude that this metric was achieved.

Table 3-9. Change NEMA Premium Motor Stocking Share at Large Motor Dealers, FY05 to FY06

Motor Size Category

Number of Respondents,

Market Share % Change

Stocking Share % Change,

FY05 to FY06Standard Error of % Change

95% Confidence

Lower Bound

All motors 9 2.9 1.5 0.1

3.7.3 Approach

For details on the survey used to gather stocking data and the calculation of the stocking share results, see Motor and Drives Metric #2.

The results were calculated in the same manner as the market share measure.

Equation 3-6

1 1

( 06 05 )n n

g sgi i i i ii i

sgi d w dSS P P w= =

−= ∑ ∑

where:

SSg is stocking share of NEMA premium motors for size group g and

05sgiP and 06sgiP are the stocking share percentages provided by respondents in the

large distributor category for size group g.


3-12


All of the equations are identical to the market share equation except that the targeted group was the subset of larger distributors. The percentages used to combine the three size groups are the same percentages used for the market share calculation.


Table 3-10 provides the stocking share results by size group. For all three size groups, the percentage point change is positive. For the two greater HP categories, the change was statistically significantly greater than zero at a 95 percent confidence level. The metric specifies increases in a range of sizes. With positive results across all three groups and statistical significance for two of them, this aspect of the metric is achieved.

Table 3-10. Change NEMA Premium Motor Stocking Share at Large Motor Dealers, FY05 to FY06, by size group

Motor Size Category

Number of Respondents, FY06 Market

ShareFY06 Stocking

Share %

Number of Respondents, Market Share

% Change

Stocking Share

% Change, FY05 to FY061

Standard Error of %

Change

95% Confidence

Lower Bound

1 to 5 Hp 8 14.2 8 4.1 2.3 -0.2 7.5 to 20 Hp 7 18.5 7 5.1 2.5 0.4

25 Hp or more 8 57.2 8 3.0 1.6 0.0All motors n/a 9 2.9 1.5 0.1

1 The All Motors results is lower than all three group estimates due to inclusion of respondents active in onlyone size category

This table also includes the FY06 market share for the three size groups. The stocking level of NEMA premium motors in the large motor group is three times higher than the other two groups. This difference is statistically significant. This result indicates that dealers are more likely to keep NEMA premium motors in stock at higher HP.

3.8 MOTOR/DRIVE CONCLUSIONS

Three of the four motor/drive metrics were evaluated in this analysis. For all three of these metrics, the goal was achieved. The remaining metric, metric #1, will be evaluated in the fall of 2006.

3.8.1 Metric #1—Market Sales Growth Index

The FY06 market sales growth index will measure the increase in NEMA premium motor shipments between calendar years 2004 and 2005. This metric will be evaluated in the fall of 2006 when the 2005 data is available.

As a test of the methodology, the market sales growth index was defined and evaluated for calendar year 2004 relative to 2003. For the motor/drive channel, calendar year 2004 is considered the base year for the index. It is the last full year prior to channel activity. It also represents the first year for which the shipment data does not include OEM shipments. The 2005 calendar year used in the FY06 index will reflect shipments from Q3 and Q4 from FY05 and Q1 and Q2 from FY06.

This metric is included as an FY07 metric (to be evaluated fall 2007). The FY07 metric describes the index as an indicator of stocking, not sales, since the data provided are


3-13


shipments to Wisconsin, not sales to final users. Shipments are, in fact, a combination of order fulfillment, replacement stock for units already sold, and stock increases. As a result, they cannot be regarded as solely representing final sales or stock increase. Despite the hybrid nature of the shipping data, we suggest continuing the index as direct measure of the motor market activity in Wisconsin.

3.8.2 Metric #2—Premium Motor Market Share Increase of 1 Percent

Motor/drive metric #2 has been defined and evaluated. The goal of an increase in market share of at least 1 percent was achieved for FY06.

The FY06 evaluation of percentage increase was based on retrospective survey questions. Respondents were asked to provide their sales share of NEMA motors for the current year, FY06, and the previous year, FY05. A second round of evaluating this metric will give the opportunity to do longitudinal analysis of responses to current year questions from this year’s survey and next year’s survey. This metric has not been included in the contract metrics for FY07. We recommend keeping it.

3.8.3 Metric #3—Market Presence Index

A market presence index has been defined and evaluated. The index provides an estimate of market presence based on employment that is designed to measure change from year to year. Results for FY06 indicate an increase of 2.2 percentage points. This metric was achieved.

The particular metric developed is problematic methodologically. In addition, the program and DOA have chosen not to include a market presence index in the FY07 contract. However, we recommend that some type of market presence indicator continue to be explored and tracked over time.

3.8.4 Metric #4—Premium Motor Stocking Share Increase

Motor/drive metric #4 was defined and evaluated in parallel with metric #2. The metric follows the same process as metric #2, but measures the share of NEMA premium motor in the stock of the larger motor dealers in the sample. The stocking share increase was positive for all HP categories and was statistically significant for two of the three categories and the aggregate measure. The metric goal was met.

This metric was not included in the draft contract metrics for FY07. We recommend keeping it.

4-1


4. LIGHTING


Table 4-1 provides the lighting channel contract metrics. The table also indicates whether the metric was achieved or failed.

Table 4-1. Lighting Channel Contract Metrics



1Market presence indicator of lighting contractors working w/Focus, based on participating contractor employment. 5% No

2

Increase ratio of Focus prescriptive technology sales to standard efficiency alternative among participating contractors, based on participant self-reported sales data or ratios.

5% Yes

Increased program participation by low and mid-sized contractors as categorized by the market potential study by 5% from FY05

There are no changes in the proposed lighting channel metrics for FY07.


4.2.1 The Target Population

Both lighting metrics target lighting contractors who are “working with Focus.” The lighting channel does not work directly with lighting contractors and there is no list of direct participants in the channel. For the purpose of this analysis, we defined participating contractors as contractors identified with end users that received prescriptive lighting rebates. FY06 contractors were identified in WATTS with measures implement during FY06 through April 21, 2006. FY05 contractors, needed for Metric #2, were identified in WATTS with measures implemented during FY05.

For the lighting metrics, the population is limited to contractors associated with the three channel technologies discussed below.

4.2.2 The Technologies

There are a large number of lighting technologies within the purview of the lighting channel. We focused on the three energy-efficient lighting technologies that produced the largest savings for Focus in FY06 (both lighting channel savings and general prescriptive lighting savings). These include highbay T8s, CFLs, and high-efficiency T8s. These energy-efficient lighting technologies all had clear standard efficiency alternatives, which was essential to addressing the second metric related to market share. Highbay T8s were compared to high intensity discharge lighting. CFLs were compared to incandescent bulbs. High-efficiency T8s were grouped with both standard T8s and low-wattage T8s. Standard T8s are clearly the standard-efficiency alternative to high-efficiency T8s. Low wattage T8s are another energy efficient option that is available for similar applications. We included the low wattage T8s both because they will be available for rebate through focus in FY07 and because they are a potential substitute technology for high-efficiency T8s.

4. Lighting…

4-2


Just as important, the three different technology groupings were reasonably distinct in their applications. There is little overlap where more than one of these energy-efficient technologies could be used. Combining the two efficient T8 technologies allowed us to do a three-way market share and avoid overlapping market share calculations relative to standard T8s.

4.3 DATA COLLECTION

Data used in this analysis included:

• Survey data from participating lighting contractors

• D&B data on businesses and employment in industries that supply lighting equipment in Wisconsin.

Information collected on the lighting contractor surveys used for this analysis included:

• Percentage of sales volume that was the energy-efficient alternative for the three technology groups in FY06

• Percentage of sales volume that was the energy-efficient alternative for the three technology groups in FY05.

Data were collected by phone interview in May and June 2006. The number of lighting contractors for which we obtained data in the interviews and the total number in WATTS are summarized below. We stratified the sample to assure responses reflecting a reasonable mix of the target technologies. For the Lighting Channel, all strata targets were met.

Table 4-2. Lighting Channel Survey Sample Size and Completes

Participating Locations 152

Surveyed Locations 41

Percent Surveyed 27%

4.4 LIGHTING CHANNEL METRIC #1

Market presence indicator of lighting contractors working with Focus based on participating contractor employment

4.4.1 Measurement

Lighting channel metric #1 is parallel to the motor/drive channel metric #3. Market presence is defined as the number of employees at businesses “working with Focus” as a percentage of all employees working for businesses selling or installing lighting. The market presence index represents the ratio of matched lighting contractors to the lighting-related contractor population. As with the motor market presence metric, the difference in the indices from FY05 to FY06 is proposed as the primary measure of interest.

An ideal market presence indicator would be total lighting contractor sales by participating contractors as a fraction of all lighting contractor sales in the state. However, neither participating nor total lighting contractor sales data are available. As a proxy, we use total employment by participating dealers and total motor-sales-related employment in the state.

4. Lighting…

4-3


Even this proxy is problematic, however, because we don’t have a good measure of lighting contractor-sales-related employment.

In the case of lighting contractors, there was FY05 market activity for the FY05 index. Under normal conditions, we would use a D&B population from the relevant period. As this is the first metrics evaluation, we are doing this analysis retrospectively and use the present D&B population for both FY05 and FY06 indices.


Table 4-3 shows the market presence indices of the FY05 and FY06 lighting channel participants in terms of percentage of businesses and percentage of employment. The table also provides the difference in indices between FY05 and FY06. The results show an increase in market presence for both indices. In terms of employment, the market presence increased 0.8 percentage points.

Table 4-3 Increase in Market Presence Index of Participating Lighting Contractors, by Number of Locations, Annual Sales, and Site Employment, FY05 to FY06

D&B Population


D&B Population


FY05 886 23 2.6% 9584 577 6.0%FY06 886 37 4.2% 9584 655 6.8%Change in Percent (Difference) 1.6% 0.8%

Program Year


These results reflect the market presence of contractors selling CFLs, high-performance T8s, and highbay fluorescents. There were no prescriptive highbay fluorescent measures in FY05. Thus, the FY05 participants do not include contractors that would only do highbay installations. On the other hand, the index includes all of FY05 while the FY06 index only includes dealers associated with measures implemented through mid-April. Neither of these limitations would be expected to substantially distort the index.

The index shows an increase in market presence that is less than the critical level of 5 percent. As a result, the metric goal was not achieved.

4.4.3 Approach

See the motor and drive metric #3 for a discussion of the market presence approach.


The changes in the market presence index for the eight-digit SIC sub-categories are provided below in Table 4-4 and Table 4-5. Table 4-6 shows the change in the market presence indices. The categories indicate where the participating lighting contractors were located in the D&B population. In FY05 and FY06, 23 and 25 percent, respectively, of participants were matched in the lighting-related eight-digit SIC categories. We targeted SIC codes that related to businesses that do lighting contractor work. We did not try to match retail outlets that provided the rebated lighting technologies. This meant that general retailers (Menards, Fastenal, Ace Hardware) were not matched despite being present in D&B. These kinds of businesses were greater than 20 percent of the population.

4. Lighting…

4-4


Table 4-4. Market Presence of Participating Lighting Contractors, by Number of Businesses and Site Employment FY05

D&B Population


D&B Population


1731-9903 General electrical contractor 601 10 2% 6,642 412 6%1731-9904 Lighting contractor 17 1 6% 66 3 5%5063-0000 Electrical apparatus and equipment 137 4 3% 1,414 90 6%5063-0206 Electrical supplies, nec 75 4 5% 1,084 65 6%5063-0400 Lighting fixtures 35 0 0% 295 0 0%5063-0401 Light bulbs and related supplies 8 2 25% 23 2 9%5063-0402 Lighting fittings and accessories 4 1 25% 16 1 6%5063-0403 Lighting fixtures, commercial and industrial 9 1 11% 44 4 9%All Categories 886 23 3% 9,584 577 6%

Industry/8-digit SIC Code

Number of Businesses Employment

Table 4-5. Market Presence of Participating Lighting Contractors, by Number of Businesses and Site Employment FY06

D&B Population


D&B Population


1731-9903 General electrical contractor 601 17 3% 6,642 408 6%1731-9904 Lighting contractor 17 2 12% 66 8 12%5063-0000 Electrical apparatus and equipment 137 4 3% 1,414 95 7%5063-0206 Electrical supplies, nec 75 5 7% 1,084 60 6%5063-0400 Lighting fixtures 35 4 11% 295 58 20%5063-0401 Light bulbs and related supplies 8 4 50% 23 16 70%5063-0402 Lighting fittings and accessories 4 1 25% 16 10 63%5063-0403 Lighting fixtures, commercial and industrial 9 0 0% 44 0 0%All Categories 886 37 4% 9,584 655 7%



Table 4-6. Change in Market Presence Percentage of Participating Lighting Contractors, by Number of Businesses and Site Employment FY05 to FY06

Industry/8-digit SIC CodeNumber of Locations Employment

1731-9903 General electrical contractor 1% 0%1731-9904 Lighting contractor 6% 8%5063-0000 Electrical apparatus and equipment 0% 0%5063-0206 Electrical supplies, nec 1% 0%5063-0400 Lighting fixtures 11% 20%5063-0401 Light bulbs and related supplies 25% 61%5063-0402 Lighting fittings and accessories 0% 56%5063-0403 Lighting fixtures, commercial and industrial -11% -9%Overall 2% 1%1For this year's analysis D&B population is the same for both FY05 and FY06

4. Lighting…

4-5


The difficulty identifying “relevant” lighting contractor employment from D&B with the small proportion and absolute number of participants matched to this database make this index problematic. We recommend further refinements be explored before relying on the market presence indicator.

4.5 LIGHTING CHANNEL METRIC #2

Increase ratio of Focus prescriptive technology sales to standard-efficiency alternatives among participating contractors based on participant self-reported sales data or ratios

4.5.1 Measurement

Similar to the motor/drive channel metric #2, the lighting channel metric #2 measures the change in market share. Market share is defined as the sales volume of the energy-efficient lighting technology as a percentage of overall sales volume of lighting in that particular application. For this year’s evaluation, there is no baseline data so we asked retrospective questions to determine if an increase occurred. For future evaluations, this year’s results will provide a baseline.

The lighting market share results were calculated using essentially the same equations (3-1 through 3-3), as above for the motor metric #2. Equation 3-3 is restated to provide the different definitions of the weights and subscripts. The combined share difference is:

Equation 4-1

4 4

1 1

06 05 ( 06 05 )mi mgi g gi g gig g

mi mgi f H fP P P P H= =

− = −∑ ∑

where:

Hg is the mean savings for technology g in WATTS and

fgi is the share of the respondent’s high-efficiency lighting in technology category g.


Table 4-7 provides the combined result for the lighting channel metric #2. The increase in efficient lighting share was 7.3 percentage points. This result is statistically significantly greater than zero at a 95 percent confidence level. The critical level for this metric, as indicated in the lighting channel’s critical goal, is 5 percent. This metric goal was clearly achieved.

Table 4-7. Change in Energy-Efficient Lighting Market Share, FY05 to FY06

Lighting Technology Group

Number of Respondents


FY05 to FY06

Standard Error of %

Change

95% Confidence

Lower Bound

All Technologies 39 7.3 1.7 4.4

4. Lighting…

4-6


4.5.3 Approach

We addressed metric #2 with a survey of lighting contractors who have sold energy-efficient lighting products rebated by Focus Business Programs.

LIGHTING SURVEY

Population. The group of participating lighting contractors provided the sample for the lighting survey. We knew a contractor sold or installed a technology group if an end user ultimately received a rebate for the energy-efficient alternative in that group. We did not know if they sold or installed other technology groups if the end user did not get rebates for those technologies. We stratified the sample based on the technology groups we knew the contractors sold or installed to maximize coverage across the three technology groups of interest. That is, we guaranteed that we spoke with a minimum number of contractors we knew carried each technology group. If they carried other the technology groups, which we did not know about from WATTS, they were asked about those technology groups as well.

A substantial number of contractors carried only CFLs and highbay fluorescents. These two groups were put in their own strata. Any contractor that carried high-efficiency T8s, regardless of other technologies, filled out a third stratum. The remainder was put in a fourth stratum. Table 4-8 provides the sample design along with the target and actual completes.

Table 4-8. Lighting Contractor Sample Design

Strata

Technologies Sold and Rebated by

Focus on EnergyBusiness

CountTarget

Completes Completes1 CFLs only 32 10 10

2Highbay fluorescent only 45 10 10

3High-efficiency T8s, with any other tech 20 10 10

4 Remainder 55 10 11Totals 152 40 41

Instrument. KEMA developed a survey instrument with the primary goal of measuring sales of energy-efficient lighting technologies as a percentage of all sales for that application. In addition, we sought information on the relative size of sales of the four different energy-efficient technologies.

The full lighting contractor survey instrument is attached in Appendix A.

Analysis of results. The market share results were calculated using Equation 4-1 above. This equation shows a ratio estimator approach with employment as a proxy for company size. This approach provides results where individual lighting contractor’s responses are weighted according that contractor’s approximate activity in the market.

4. Lighting…

4-7



Table 4-9 reports the market share results at the lighting technology group level. All of the changes in market share results are statistically significantly greater than zero at the technology group level.

Table 4-9 also reports the average level of market share for FY06. These results indicate that highbay T8s and CFLs represent more than half the sales in their technology groupings. High-efficiency T8s and low-wattage T8s combined represent approximately just over 30 percent of the T8 market. While these are participating lighting contractors, these market shares are unexpectedly high.

Table 4-9. Change in Energy-Efficient Lighting Market Share, FY05 to FY06 by Technology Group

Lighting Technology Group

Number of Respondents

FY06 Market Share %


FY05 to FY06

Standard Error of %

Change

95% Confidence

Lower Bound

CFL 35 58.9 3.8 1.2 1.7High Bay Fluorescent Fixtures

36 55.2 8.2 2.1 4.7High Performance T8 39 20.7 7.3 2.8 2.5Low Watt T8 40 11.2 6.1 2.9 1.2All Technologies 39 n/a 7.3 1.7 4.4

4.6 LIGHTING CONCLUSIONS

Both lighting channel metrics were evaluated in this analysis. For metric #2, the market share metric, the metric goal was achieved.

4.6.1 Metric #1—Market Presence Index

Lighting channel metric #1 compares market activity between FY05 and FY06. The employment indices show an increase in market activity in FY06 of less than 1 percent. The metric goal stipulates a 5 percent increase to pass this metric. This metric shows an increase in market presence, but fails to achieve its critical level of increase.

This lighting channel metric, with identical wording, is proposed for FY07. The FY07 evaluation will allow full comparison of channel activities between the two years. The particular metric developed for FY06 is problematic methodologically. However, we recommend that some type of market presence indicator continue to be explored and tracked over time.

4.6.2 Metric #2—Market Share Index

A market share index has been defined and evaluated for the lighting channel. The index provides an estimate of increase in market share based on surveys of lighting contractors. Results for FY06 indicate an increase in market share of 7 percentage points. The metric goal stipulates a 5 percent increase to pass this metric. This metric goal was achieved.

4. Lighting…

4-8


This metric will be evaluated again for FY07. The second round of evaluation of this metric will give the opportunity to do longitudinal analysis of lighting contractors who participated in both years.

5-1


5. PULP AND PAPER INDUSTRIAL CLUSTER


Table 5-1 provides the pulp and paper industrial cluster contract metrics. The table also indicates whether the metric was achieved or failed.

Table 5-1. Industrial Cluster—Pulp & Paper Contract Metrics



1

Measure rate of preliminary Adoption of the Best Practices by the Wisconsin Paper Council, the Lake States Technical Association of Pulp & Paper Industry and Focus participating Companies

Number of groups, experts, and customers utilizing or endorsing the Best Practices in Pulp & Paper Industry.

Yes

2 No critical level is provided or implied.

This metric is continued for FY07.


The pulp and paper industry cluster metric targets a broad range of potential market actors. The majority of our evaluation effort for FY06 was focused on pulp and paper mills that are the target market for the Pulp and Paper Energy Best Practice Guidebook. This survey had the dual purpose to establish a baseline that will allow further measurement of this metric beyond this the first year of its publication, while using retrospective questions to measure an effect in the first year.

There is evidence that mills in Wisconsin are utilizing the Guidebook for industry-specific practices. Seven percent of the industry indicated the Guidebook was influential in their energy related decision making.

The metric explicitly includes “groups” and “experts” as well as “customers” in the metric. Endorsement of the Guidebook can be seen as a precursor to wider endorsement in the marketplace. Within a year of publication of the Guidebook a major conference was held that highlighted energy issues in general and the Guidebook in particular.1 This conference was co-sponsored by major industry groups including the Wisconsin Paper Council and Lake States TAPPI. This conference is evidence of endorsement of the Guidebook by industry groups and experts.

1 Forum on Energy: Immediate Solutions, Emerging Technologies - 2006 TAPPI. Appleton, WI. May 15–17, 2006.

5. Pulp and Paper Industrial Cluster…

5-2


No numerical goal or critical level was set for this metric in the contract. We therefore evaluate the goal based on whether there was demonstrably a positive effect. This standard, effectively setting a target of exceeding zero effect, is a weak goal. If this metric continues to be tracked we recommend setting explicit numeric targets.

The combination of industry group endorsement and target market awareness indicate the stated metric is satisfied for FY06. The Guidebook has influenced a small percent of the industry to improve industry-specific energy-related practices.

5.3 APPROACH

This section discusses the survey of Wisconsin pulp and paper mills. KEMA utilized the pulp and paper mills list published by the Center for Paper Business and Industry Studies (CPBIS)2 as a frame. As of the update published in May 2006, this list included 54 pulp and paper mills in Wisconsin. Of these, eight3 were identified as closed or idle and were excluded from the frame. This list is presented in Appendix A.

The CPBIS list of mills includes data on the installed capacity of the mills, which was used to stratify the frame, establish sampling targets, and produce sampling weights. The strata descriptions, sample targets, and final sample are detailed in Table 5-2.

Table 5-2 Pulp and Paper Industry

Strata Strata description

Population (Active Pulp and Paper Mills as of

May, 2006)

Total annual production capacity in

thousands of short tons

Percent of Annual

Production Capacity

Sample Target

Sample Count

Sample annual production capacity in

thousands of short tons

Percent of Annual

Production Capacity

1 400 tons or more 7 3,731 48% 7 5 2,900 37%2 100 to 399 tons 16 2,939 38% 9 8 1,665 21%3 40 to 99 tons 12 806 10% 3 4 244 3%4 39 or less, unknown 11 292 4% 1 2 85 1%

Total 46 7,768 100% 20 19 4,894 63%

The sample utilized for this metric assessment covers 63 percent of the active installed capacity of the pulp and paper mills in the state.

The survey instrument designed for this metric is presented in Appendix B. It contains specific questions about eight of the best practices, an open-ended question regarding the other practices, and ratings on the influence of the Guidebook in the mills’ energy practices.

The survey was fielded by calling the mills in the sample and speaking to a person that participates directly in decisions concerning the specification, purchase, operation, or maintenance of energy using equipment at the location. When available, WATTS contacts

2 The Center for Paper Business and Industry Studies is a joint venture of the Georgia Institute of Technology (“Georgia Tech”), the Alfred P. Sloan Foundation (“the Sloan Foundation”), and the paper industry. http://www.cpbis.gatech.edu. May 2006. 3 Six of the closed paper mills were listed as such on the CPBIS list and two were found by KEMA while conducting the industry survey. These two resulted in minor adjustments to the original sampling frame.


5-3


were used as the first attempted contact. Three of the 19 respondents preferred to respond to the survey by e-mail.

5.4 DETAILED FINDINGS

Core Best Practices

Respondents were asked if their facilities employed any of the following eight core best practices. None of the respondents indicated that they had implemented any of these Best Practices as a result of consulting the Guidebook. These results are illustrated in Figure 5-1.

• Energy cost reduction goals. It is estimated that 97 percent of the active installed capacity in Wisconsin is in facilities that have energy cost reduction goals.

• Energy management budget. Only 36 percent of the active installed capacity in the State is in facilities that have formal energy management budgets.

• Energy benchmarking. Energy benchmarking occurs in facilities that account for about 91 percent of the active installed capacity.

• Assessment of individual pump systems. Facilities that account for 85 percent of the installed capacity periodically assess the operation of individual pump systems.

• It is estimated that virtually all facilities:

− Have persons responsible for identifying facility improvements to reduce energy us.

− Collect and reuse hot mill water.

− Use variable frequency drives (VFD)

− Use high efficiency motors.

Our understanding is that the Guidebook was not intended primarily to promote adoption of the core best practices. Rather, these practices are included as foundation and context. The practices to be promoted are the specific ones described below. Nonetheless, it is helpful to establish that the core practices are indeed very well established in this industry. The Guidebook should not be expected to provide much assistance in these areas.


5-4


Figure 5-1. Current Adoption of Core Best Energy Practices without Influence of the Guidebook

(Percent of Wisconsin’s Active Installed Capacity)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Has energy cost reduction goals

Has an annual budget assigned toenergy management

There are persons responsible foridentifying facility improvements to

reduce energy use

Benchmarks energy consumption

Collects and reuses mill hot water

Uses VFDs

Uses high efficiency motors

Assesses operation of individualpump systems

YES NO

Effect of the Pulp and Paper Best Energy Practice Guidebook

Awareness of the Guidebook. Question 14 asked whether the respondent was aware of the Pulp and Paper Best Energy Practice Guidebook developed by Wisconsin’s Focus on Energy Program. It is estimated that 36 mills, which account for 67 percent of the active installed capacity in Wisconsin, are aware of the Guidebook. This result is illustrated in Figure 5-2.


5-5


Figure 5-2. Question 14: Awareness of the Guidebook (Percent of Wisconsin’s Active Installed Capacity)

Yes67%

No33%

Use of the Guidebook as a resource. Question 15 asked if the respondent has used the Guidebook as a resource. Of the 67 percent of installed capacity that is associated with awareness of the Guidebook, 50 percent of total installed capacity is in facilities that have used the Guidebook, and the remainder (17 percent) is in facilities that have not.


5-6


Figure 5-3. Question 15: Consulted the Guidebook as a Resource (Percent of Wisconsin’s Active Installed Capacity)

Does not know the Guidebook

33%

No17%

Yes50%

Influence of the Guidebook. Figure 5-4 illustrates the responses to Question 16. In this question, respondents rated the influence of the Guidebook on their energy practices on a scale of 1 to 5, where 1 is “not at all influential” and 5 is “very influential.” A large majority of the respondents who were aware of the Guidebook gave it a low influence rating. Of the 50 percent of installed capacity that is in facilities where the Guidebook has been used, 43 percent of total installed capacity was in facilities that rated the Guidebook’s influence as “1” (not at all influential”), or “2,” and 7 percent was in facilities that rated the influence as “4.” None gave a “very influential” rating (5) and none gave a neutral rating (3).


5-7


Figure 5-4. Question 16: Influence of the Best Energy Practice Guidebook on Energy Practices. Scale of 1 (“not all influential”) to 5 (“very influential”)

(Percent of Wisconsin’s Active Installed Capacity)

Does not know or has not used Guidebook

50%

" 3 "0%

" 4 "7%

" 5 " Very influential0%

" 1 " (Not at all influential)3%

" 2 "40%

Use of the Guidebook to identify improvements. Question 18 asked if, in addition to the more general best practices that had been asked earlier in the survey, the participant had used the Guidebook to identify any technology or process-specific improvements for their mill.4 It is estimated that 16 mills in Wisconsin, which account for 33 percent of the installed capacity, implemented one or more measures that were originally identified in the Guidebook. This is illustrated in Figure 5-5.

4 Some respondents indicated that they had implemented several of the Best Practices identified in the Guidebook prior to becoming aware of the Guidebook. This study was not designed to provide a baseline for practices other than those that were listed individually.


5-8


Figure 5-5. Question 18: Used Guidebook to Identify Relevant Best Practices (Percent of Wisconsin’s Active Installed Capacity)


50%

No17%

Yes33%

Implemented improvements identified in the Guidebook. If a respondent answered yes to Question 18 (used Guidebook to identify applicable improvements), Question 19 was asked to determine if any of the improvements identified had been implemented. Facilities accounting for 16 percent of the total installed capacity have installed, or are in the process of installing, improvements identified in the Guidebook. This is presented in Figure 5-6.


5-9


Figure 5-6. Question 19: Implemented Best Practices identified in the Guidebook (Percent of Wisconsin’s Active Installed Capacity)


50%

Has implemented practices identified in

Guidebook16%

Has identified practices in Guidebook, but not

implemented them yet17%

Used Guidebook but has not identified potential

practices17%

The 16 percent who used the Guidebook to identify a practice they subsequently adopted is twice as great as the 7 percent who indicated the Guidebook was influential on their energy-related decision making. One possible explanation is that respondents did not necessarily think of adopting these measures as part of “energy-related decision making.” It is also possible that they viewed the Guidebook as convenient but not essential to identifying these measures.

If a respondent indicated that their facility had implemented or was in the process of implementing any of the best practices identified in the Guidebook, they were asked which of the best practices they had implemented. Respondents that answered this question indicated “dryer drainage systems for paper machines,” “dryer management systems,” and “hot make-up air for boiler.”

6-1


6. CONCLUSIONS

The purpose of this analysis was to design and implement quantitative processes to measure Business Program channel and cluster market effects contract metrics now and in future evaluations and to apply these processes to assess the FY06 market effects contract metrics where feasible. The metrics measure program effects that are not directly tracked by the program.

Specifically, this report assesses six contract metrics, three associated with the motor/drive channel, two with the lighting channel and one with the pulp and paper industrial cluster.

6.1 MOTOR/DRIVE AND LIGHTING CHANNELS

The two channels share two metrics, the market share metric and the market presence metric. The motor/drive channel has a second version of the market share metric that addresses stocking.

The motor/drive channel also has another metric based on industry-provided sales data. Assessment methods for this metric have been developed and reported on previously, and are summarized in this report. Evaluation of this metric for FY06 will not be possible until the next round of data become available in the fall of 2006.

All three of the share metrics (one for lighting, two for motors) passed based on retrospective data collected from respondents. The market presence metric for the motor/drive channel passed at the 1 percent critical level. The market presence metric for the lighting channel showed positive improvement but failed at the 5 percent critical level.

The evaluation of the share-oriented contract metrics for FY07 will provide a more rigorous challenge for the channels. They will be based on longitudinal analysis as well as retrospective questions. Under the present plans, a next round evaluation of these metrics will take place for the lighting channel after the end of FY07. We recommend continuing evaluation of the share-oriented contract metrics for the motor/drive channel as well.

The market presence metrics passed comparing to FY05 when channel activity was still to begin. The market presence index is problematic and needs further development for FY07. Under the present plans, a next evaluation of a market presence metric will take place for the lighting channel after the end of FY07. We recommend that a market presence indicator also be evaluated for FY07 for the motor/drive channel.

We also recommend that in the future every metric be assigned an explicit numeric goal by which achievement of the metric can be measured.

6.2 PULP AND PAPER INDUSTRIAL CLUSTER

The pulp and paper industrial cluster has a single metric related to the adoption of the Best Practices Guidebook. Evidence from the survey as well as evidence of endorsement by major industry groups indicates the pulp and paper industry cluster metric was achieved for FY06. Baseline data was gathered for FY06, which will enable measurement of this metric in future evaluations. Baseline findings include:

6. Conclusions…

6-2


1. Industry group endorsement of the Guidebook is demonstrated by the jointly sponsored conference. Ongoing industry support will need to be demonstrated in future years by similar or new promotional efforts.

2. Seven of the eight core best practices5 are currently practiced in facilities accounting for a substantial majority of the pulp and paper installed capacity. No respondents indicated that the Guidebook had any influence on their implementation of these core practices. There is little opportunity for the program to increase the prevalence of these core practices.

3. There is potential to increase awareness of the Guidebook. Respondents in facilities accounting for about one-third of the total active installed capacity did not know about the Guidebook.

4. The Guidebook appears to have had some positive impact in recognition of possible technology- or process-specific improvements. Facilities that account for 33 percent of the total active installed capacity have used the Guidebook to identify technology or process-specific improvements that relate to their mills.

5. The Guidebook appears to have had a positive impact in actual implementation of improvements. Facilities that account for 16 percent of the total active installed capacity have implemented or are in the process of implementing improvements identified in the Guidebook.

6. On the other hand, we did not determine that the Guidebook is the primary reason the improvements identified in it were made. The improvements might have been identified another way if the Guidebook had not been available. Only 7 percent indicated that the Guidebook had influenced their energy-related decision making Thus, the overall level of the Guidebook’s influence at this stage appears to be modest.

7. This study establishes a good baseline by which to measure the long-term impact of the Guidebook.

5 The eight core best practices are: energy cost reduction goals, energy management budget, energy benchmarking, assessment of individual pump systems, persons responsible for identifying facility improvements to reduce energy use, collect and reuse hot mill water, use of variable frequency drives, and use of high-efficiency motors.

A-1


APPENDIX A: MOTOR/DRIVE DEALERS/DISTRIBUTORS QUESTIONNAIRE

FOCUS BUSINESS PROGRAMS CONTRACT METRICS FY06: PARTICIPATING MOTOR DEALERS/DISTRIBUTOR QUESTIONNAIRE

KEMA ID _________________________________________________________

COMPANY ________________________________________________________

CONTACT ________________________________________________________

LEAD-IN: Hello, my name is _________ and I am calling from KEMA. We are an energy consulting firm that’s been hired by the Wisconsin Department of Administration to study the electric motor and drive market in Wisconsin. This is not a marketing call. We are conducting a short survey of motor dealers in Wisconsin. Your individual responses will be kept absolutely confidential and only reported combined with data from other respondents. As an additional incentive for participating in the survey we will draw a single winner from among completed interviewees and that person or the charity of their choice will receive a prize of 500 dollars.

[If Contact name available] May I please speak with ______________________

[If No Contact name available] May I speak with the person who is most familiar with your company’s sales of electric motors and variable frequency drives?

Alternate contact __________________________________________________

[If concerned regarding legitimacy:

If you would like I can give you telephone numbers to contact my project manager at KEMA or our project manager at the Division of energy. Would you like that?]

KEMA - Ken Agnew (608) 277-9696

DOE - Oscar Bloch (608) 264-8267

[If not clear what is in it for them:

The information you can provide is very valuable to the Division of Energy with regards to doing the right thing for the motor market. This is why we are going to give away $500 dollars to one of the participants.]

A: Motor/Drive Dealers/Distributors Questionnaire

A-2


Company Information

First I have some general questions about your company.

1. Do you sell new electric motors or variable frequency drives primarily to installers, contractors or end users. [IF NECESSARY. “This would be in contrast to selling to another dealer that would then sell to the installer or end user”].

Yes ……………………………….1 No ………………………………. 2 Don’t know …………………… -98 Refused ………………………. -99

[IF Q1^=1 THEN THANK AND TERMINATE]

2. At how many different locations in Wisconsin does your company sell electric motors or variable frequency drives?

_________ [ENTER A NUMBER] Don’t know …………………… -98 Refused ………………………. -99

[IF Q2 > 1 THEN ASK Q3]

3. Next, I will ask you some questions regarding your sales of motors and VFDs. Can you provide me with information for your present location only, or for your multiple Wisconsin locations?

All Wisconsin locations ……….. 1 This location only ……………… 2 Don’t know …………………… -98 Refused ………………………. -99

4. [If Q2=1 or Q3=1] Approximately what percentage of your company’s overall sales, in dollars, would be in NEW electric motors and variable frequency drives?

[If Q2>1 AND Q3=2] Approximately what percentage of your location’s overall sales, in dollars, would be in NEW electric motors and variable frequency drives?


5. [If Q2=1 or Q3=1] How many full-time employees work at your company in Wisconsin?

[If Q2>1 AND Q3=2] How many full-time employees work at this location? _________ [ENTER A NUMBER] Don’t know …………………… -98 Refused ………………………. -99


A-3


6. Do you sell NEMA premium electric motors? Yes ……………………………….1 No ………………………………. 2 Don’t know …………………… -98 Refused ………………………. -99

[IF Q5^=1 THEN THANK AND TERMINATE]

Sales of EE compared to standard By Technology

Now I am going to ask you about your sales of NEMA premium motors relative to their standard counterparts, and VFDs. I will be asking the same set of questions for motors in 3 different size groupings. Please answer these questions for your sales to those who install or use the equipment.

A. Motors of 1-5 Horsepower B. Motors of 7.5-20 Horsepower C. Motors of >= 25 Horsepower D. Variable Frequency Drives

[QUESTIONS MARKED WITH * ARE ASKED ONCE FOR EACH MOTORS CATEGORY]

7* Do you sell ** Yes ….........................................................................................................1 No [CLOSE SIZE CATEGORY]................................................................2 Don’t know [ATTEMPT TO LOCATE BETTER CONTACT / CLOSE CATEGORY] -97 Refused [CLOSE SIZE CATEGORY] ....................................................-98

7d Do you sell VFDs Yes ….........................................................................................................1 No [CLOSE SIZE CATEGORY]................................................................2 Don’t know [ATTEMPT TO LOCATE BETTER CONTACT / CLOSE CATEGORY] -97 Refused [CLOSE SIZE CATEGORY] ....................................................-98

[QUESTIONS 8 TO 11 APPLY TO MOTORS ONLY]

8* Consider all of your sales of ** since July 1st Last year, what percentage of those total sales in terms of number of units would you say were the NEMA premium?

_______% [ENTER A NUMBER BETWEEN 0 and 100] Don’t know -97

Refused -98

9* What about the year prior to that? [if NECESSARY, “that is, from July 1, 2004 to June 30, 2005”]


Refused -98


A-4


[IF THE RESPONSE TO EITHER Q7 OR Q8 IS DON’T KNOW (-97) OR REFUSED (-98), ASK Q10*]

10* Can you tell me if your sales if NEMA premium ** relative to standard efficiency has increased, decreased, or stayed the same between the most recent year, since July 1st, 2005 and the year previous to that [if NECESSARY, “that is, from July 1, 2004 to June 30, 2005”] Increased …………………………………………………………………..1 No change [SKIP TO NEXT TECH]..........................................................2 Decreased …………………………………………………………………..3 Don’t know [SKIP TO NEXT TECH] ......................................................-97 Refused [SKIP TO NEXT TECH]...........................................................-98

[IF PERCENTAGE HAS INCREASED OR DECREASED BASED ON Q8*/Q9* OR Q10*, ASK Q11*]

11* Why do you think the percentage of NEMA premium ** sales has increased/decreased?

___________________________________________________________

___________________________________________________________ Don’t know -97 Refused -98

[QUESTIONS 12 TO 14 APPLY TO VFDs ONLY]

12 Consider all of your sales of VFDs since July 1st last year. Has the amount of sales increased or decreased with respect to the 12 months prior to that? Increased …………………………………………………………………..1 No change …………………………………………………………………..2 Decreased …………………………………………………………………..3 Don’t know ………………………………………………………………..-97 Refused ………………………………………………………………..-98

[IF Q12 = 1 OR Q12 = 3 THEN ASK Q13]

13 By what percentage? _________ [ENTER A NUMBER BETWEEN 0 AND 100] -98 = Don’t know -99 = Refused

14 Why do you think the percentage of VFD sales has increased/decreased?

___________________________________________________________

___________________________________________________________ Don’t know ………………….. -97 Refused ………………….. -98


A-5


Now I am going to ask a similar set of questions regarding your inventory.

15* Consider your current stock of **, what percentage of your total stock in terms of number of units would you say are the NEMA premium motors?


Refused -98

16* What about as of July 1st 2005? _______% [ENTER A NUMBER BETWEEN 0 and 100] Don’t know -97

Refused -98

[IF THE RESPONSE TO EITHER Q15 OR Q16 IS DON’T KNOW (-97) OR REFUSED (-98), ASK Q17*]

17* Has the percentage of stock of ** NEMA premium motors increased, decreased or stayed the same between last year and the present time? Increased …………………………………………………………………..1 No change [SKIP TO NEXT TECH]..........................................................2 Decreased …………………………………………………………………..3 Don’t know [SKIP TO NEXT TECH] ......................................................-97 Refused [SKIP TO NEXT TECH]...........................................................-98

[IF PERCENTAGE HAS INCREASED OR DECREASED BASED ON Q15/Q16* OR Q17*, ASK Q18*]

18* Why do you think the percentage of NEMA premium ** stock has increased/decreased?


19 How often do you have to have a motor in stock to make a sale? These are situations when the customer cannot wait and has to buy something off the shelf.

20 Consider your current stock of VFDs. Has your stock increased, decreased, or stayed the same between now and last year? Increased......................................................................................1 No change ...................................................................................2 Decreased ....................................................................................3 Don’t know.................................................................................-97 Refused .....................................................................................-98

[IF Q20 = 1 OR Q20 = 3 THEN ASK Q21]


A-6


21 Why do you think your stock of VFDs has increased/decreased?

___________________________________________________________


Awareness and Effect of Focus

Now, a few general question.

22 Are you familiar with the Focus on Energy Program, the Wisconsin program that provides rebates for NEMA premium motors


[IN THE UNLIKELY CASE THAT Q15 in (2, -98, -99) then skip to Q31]

23 Consider your sales of NEMA premium motors and variable frequency drives. Using a scale of 1 to 5, where 1 means “not at all influential” and 5 means “very influential,” how influential do you think Focus on Energy rebates and other activities have been on your sales from July 1st, 2005 to present.

1 Not at all influential 2 3 4 5 Very Influential .. Don’t know …………………… -98 Refused ………………………. -99

24 Using the same scale of 1 to 5, how influential do you think Focus on Energy rebates and other activities have been on your Stocking practices since July 1st 2005.

1 Not at all influential 2 3 4 5 Very Influential .. Don’t know …………………… -98 Refused ………………………. -99

25 Of all the NEMA premium motors and variable frequency drives you sold since July 1st 2005, approximately what percentage were to customers that received a rebate from Focus?


Refused -98


A-7


26 What about the year prior to that? _______% [ENTER A NUMBER BETWEEN 0 and 100] Don’t know -97

Refused -98

27* Earlier, you said that <Q8**> percent of your total sales for ** in the last 12 months were NEMA Premium motors. What do you think this percentage would be if there had never been a Focus on Energy program?


Refused -98

28* If the Focus on Energy Program ended now, what do you think the percentage would be after two years? [if not between Q8 and Q26 then clarify]


Refused -98

29 Earlier, you said that your sales of VFDs have <Q19>. What do you think would have happened if there had never been a Focus on Energy program? Would have increased .............................................................................1 No change .........................................................................................2 Would have decreased.............................................................................3 Don’t know ......................................................................................-97 Refused ......................................................................................-98

30 If the Focus on Energy Program ended now, would your sales increase or decrease after two years? Increase........................................................................................1 No change ...................................................................................2 Decrease ......................................................................................3 Don’t know.................................................................................-97 Refused .....................................................................................-98

31 About how many units of NEMA Premium motors and VFDs have you sold since July 1, 2005? 1-5 Horsepower..............................................................................______ 7.5-20 Horsepower.........................................................................______ >= 25 Horsepower..........................................................................______ VFDs ...............................................................................______ Don’t know ......................................................................................-97 Refused ......................................................................................-98

[IF Q31 = -97 OR Q31 = -98 THEN ASK Q32]


A-8


32 About what percent of your NEMA Premium motors were… 1-5 Horsepower..............................................................................______ 7.5-20 Horsepower.........................................................................______ >= 25 Horsepower..........................................................................______ VFDs ...............................................................................______ Don’t know ......................................................................................-97 Refused ......................................................................................-98

Now, just a few more general questions.

33 What do you see as the barriers to more sales of NEMA premium motors and variable frequency drives versus their standard efficiency counterparts?

34 Other than providing rebates, what other support does Focus on Energy provide for NEMA premium motors and VFDs?

35 Other than buying down first costs, what strategies do you believe would help to increase sales of premium motors and VFDs?

V Other VERBATIM recorded during the interview.

P Interviewer notes.

Asked by size category:

Ask OEM1 and/or OEM2 depending on info already available and whether one is easier to answer than the other for the respondent-

OEM1* About what percent of your sales in terms of numbers of units go to OEMs?

OEM2* About what percent of your dollar sales go to OEMs?

OEM3* About what percent of OEM purchases are NEMA premium?

B-1


APPENDIX B: LIGHTING CONTRACTOR SURVEY INSTRUMENT

FOCUS BUSINESS PROGRAMS CONTRACT METRICS FY06: PARTICIPATING LIGHTING CONTRACTOR QUESTIONNAIRE

KEMA ID _________________________________________________________

COMPANY ________________________________________________________

CONTACT ________________________________________________________

[If Contact name available] May I please speak with ___________________

LEAD-IN: Hello, my name is _________ and I am calling from KEMA consulting. We are an energy consulting firm that’s been hired by the Wisconsin Department of Administration to study the lighting market in Wisconsin. This is not a marketing call.

[If No Contact name available] May I speak with the person who is most familiar with your company’s sales of lighting products?

We are conducting a short survey of lighting distributors and contractors in Wisconsin regarding the availability of different lighting technologies. Your individual responses will be kept absolutely confidential and only reported combined with data from other respondents. As an additional incentive for participating in the survey we will draw a single winner from among completed interviewees and that person or the charity of their choice will receive a prize of 500$.

Alternative contact____________________________________________

Company Information

First I have some general questions about your company.

1. Do you sell any lighting products directly to the end user?

1. Yes 2. No -98 Don’t know -99 Refused

2. Do you install lighting products?


[ IF #1^=1 AND #2^=1 THEN THANK AND TERMINATE]

3. What percentage of your company’s overall business do these two direct to-the-end-user sales represent? [ENTER A NUMBER BETWEEN 0 and 100]

-98 Don’t know -99 Refused%

[IF 0% OR -99 THEN THANK AND TERMINATE]

B: Lighting Contractor Survey Instrument

B-2


4. How many full-time employees work at this location?

-98 Don’t know -99 Refused#

Sales of EE compared to standard By Technology

Now I am going to ask you about some standard technologies and the high efficient lighting technologies with which they can be replaced. Please answer these questions for your installs and sales directly to end users.

A. 1. Incandescent lights 2. Compact Fluorescent Lamps

B. 1. High Intensity Discharge [HID] lighting of >250 watts

2. Highbay fluorescent fixtures that would be either T8 or T5 C. 1. Standard T8 fluorescents

2. High performance T8s, (CEE –approved, with high lumens and low ballast factor) 3. Low Watt T8s (25, 28 and 30 W T8 Lamps for overlit spaces)

[QUESTIONS 5-9 ARE REPEATED FOR ALL TECHNOLOGY GROUPS BEFORE MOVING ON TO QUESTION 10.]

5* Do you sell or install *1 or *2 (or 3*)?.......................................................... *1 ......................................................................................1 *2 ......................................................................................2 (*3) ....................................................................................3 None .............................................................................-96

Don’t know ....................................................................-97 Refused .........................................................................-98

[IF Q5 ~ IN (2, 3) THEN CLOSE TECH - ASK NO MORE QUESTIONS FOR THIS TECHNOLOGY GROUP FOR THE REMAINDER OF THE INTERVIEW]

[IF Q5 ~ IN (-97), CONSIDER PROMPTING FOR BETTER CONTACT PERSON]

6* Consider all of your sales of *1 and *2 (and 3*) in the last 12 months, what percentage of those total sales in terms of number of units would you say were the *2? (What about 3*)

[ENTER A NUMBER BETWEEN 0 and 100] ..... _______% Don’t know ....................................................................-97

Refused .........................................................................-98

7* What about the 12 months prior to that for *2? (for *3?) [ENTER A NUMBER BETWEEN 0 and 100 AND SKIP TO 9*] .......................................................................... _______% Don’t know ....................................................................-97

Refused .........................................................................-98


B-3


8* Has the percentage of sales of *2 increased, decreased or stayed the same between this last year and the previous year? (sales of *3?) Increased ..........................................................................1 No change [SKIP TO NEXT TECH OR # 10].....................2 Decreased.........................................................................3 Don’t know [SKIP TO NEXT TECH OR # 10] .................-97 Refused [SKIP TO NEXT TECH OR # 10]......................-98

9* What do you think has caused the percentage of *2 to increase/decrease? (For *3?) ___________________________________________________________ Don’t know ....................................................................-97 Refused..........................................................................-98

Awareness and Effect of Focus

10. According to our records, customers of yours have received rebates from the Wisconsin Focus on Energy Program. This is a program that provides rebates for high efficient lighting products? Are you familiar with the program?


[if Q10^=1 then ask if someone else might be more aware of program. If not, then Thank and Close.]

11. What can you tell me about the program?

12. Focus on Energy rebates usually go to the end user not the contractor. Do you generally know if a customer is getting a rebate? How does that work?

Effect of Focus By Technology

Now, I would like to go back through the high efficient lighting technologies I asked you about earlier and ask you about the influence of Focus on Energy. We are still focusing on installs and sales directly to end users.

[IF *5 =2 and/or 3 THEN ASK THE REMAINING QUESTIONS FOR TECHNOLOGY *]

13* Using a scale of 1 to 5, where 1 means “not at all influential” and 5 means “very influential,” how influential do you think Focus on Energy rebates and other activities have been on your sales of *2 in the last 12 months? (of *3 ?)

1 Not at all influential 2 3 4 5 Very Influential .. Don’t know -97

Refused ....................................................................................-98


B-4


14* Of all the *2 units (*3) you sold in the last 12 months, what percentage were to customers that received rebates from Focus?


15* Of all the *2 units (*3) you sold in the 12 months prior to that, what percentage were to customers that received rebates from Focus?


16. Earlier, you said that <Q #6> percent of your total lighting sales for this technology group were *2 (were*3). What do you think the percentage of *2 (% of 3) would be today if there had never been a Focus on Energy program?


17. If the Focus on Energy Program ended now, what do you think the percentage would be after 2 years? [If not between Q#6 and Q#16 then clarify]


18. Why do think this will happen?

________________________________________________________________

Now, two questions to finish up.

19. Consider all the High efficiency technologies we’ve discussed that you carry,

[READ ONLY THOSE TECHS THE CONTRACTOR CARRIES.]

Compact Fluorescent Lamps

Highbay fluorescent fixtures

High performance T8s and

Low Watt T8s

And put any other high efficient lighting technologies you sell or install into an “other” category. What percentage of your high efficiency lighting sales in terms of volume do each of these groups represent.

[RESULT SHOULD ADD TO 100%]

Compact Fluorescent Lamps ..................................................................______%

Highbay fluorescent fixtures ........................................................______%

High performance T8s and .........................................................______%


B-5


Low Watt T8s .............................................................................______%

Other .............................................................................______%

20. Finally, do you have any other comments about the program?

Confirm contact information for prize, Thank and close.

C-1


APPENDIX C: LIST OF PULP AND PAPER MILLS IN THE STATE OF WISCONSIN

Mill Id Managed By Status (a) Products (b) Tons (c) Strata277 American Tissue Mills of WI (American Tissue Corp.) Tissue(8) 8 4278 Appleton Coated L.L.C. Ucfs(360), Ctdfs(185) 545 1279 CityForest Corp. Tissue(49) 49 3280 Domtar Industries Inc. Ucfs(218) 218 2281 Domtar Industries Inc. Ucfs(174) 174 2282 Filter Materials(Gusmer Enterpr.) Spepack(5) 5 4283 Fox River Fiber Co. > Now __ Pulp(77) 77 3284 SMART Papers L.L.C. (1) Ucfs(140) 140 2285 Georgia-Pacific Corp. (1) Tissue(22) 22 4286 Georgia-Pacific Corp. Tissue(425) 425 1287 Georgia-Pacific Corp. Const(NA) 50 4288 Georgia-Pacific Corp. Const(51) 51 3289 Globe Building Materials Inc. Const(73) 73 3290 Stora Enso North America Corp. (1) Pulp(30) 30 4291 Stora Enso North America Corp. Ctdfs(295), Ctdgw(160) 455 1292 International Paper Co.> Now Thilmany (Kohlberg LLC) Spepack(200) 200 2293 Oconto Falls Tissue, Inc. Tissue(68) 68 3294 P.H. Glatfelter Co. (2) Ucfs(158) 158 2295 Packaging Corp. of America Corru(520) 520 1296 Cascades Inc. Tissue(55) 55 3297 Ponderosa Fibres of America Inc. Pulp(77) 77 3298 Procter&Gamble Paper Products Tissue(NA) N/A 4299 Re-Box Packaging, Inc. (2) Pulp(35) 35 4300 SCA Tissue North America, L.L.C. (Svenska Cellulose Ab) Tissue(220) 220 2301 Stora Enso North America Corp. Ctdgw(195) 195 2302 Stora Enso North America Corp. Ctdgw(230) 230 2303 Stora Enso North America Corp. Ctdfs(550), Ctdgw(405) 955 1304 Stora Enso North America Corp. Pulp(401) 401 1305 Ward Paper (International Paper) (1) Ucfs(37) 37 4306 Wausau-Mosinee Paper Corp. Ucfs(120) 120 2307 Wausau-Mosinee Paper Corp. Tissue(113), Spepack(113) 226 2308 Weyerhaeuser Co. Ucfs(150) 150 2

1191 Fox River Paper Co.(Fox Valley) Ucfs(35) 35 41192 Riverside Paper Corp. (1) Ucfs(77) 77 31193 Beloit Box Board Co., Inc. Recybrd(21) 21 41194 Nicolet Paper Co. > Now Thilmany (Kohlberg LLC) Spepack(65) 65 31195 U.S. Paper Mills Corp. Recybrd(35) 35 41196 Fibreform Containers Inc. Spepack(NA) N/A 41197 Georgia-Pacific Corp. Tissue(175) 175 21198 Green Bay Packaging Inc. Liner(225) 225 21199 Procter&Gamble Paper Products Tissue(229) 229 21200 Kimberly-Clark Corp. Tissue(75) 75 31201 Geo. A. Whiting Paper Co. Ucfs(10) 10 41202 Fox River Paper & Mead Corp. > Now Gilbert Paper Ucfs(35) 35 41203 U.S. Paper Mills Corp. Corru(33) 33 41204 Wisconsin Paperboard(Newark Group) Spepack(152) 152 21205 Kimberly-Clark Corp. Tissue(100) 100 21206 Neenah Paper(Kimberly-Clark Corp.) Ucfs(80) 80 31207 Badger Paper Mills, Inc. (1) Ucfs(79) 79 31208 Wausau-Mosinee Paper Corp. Spepack(165) 165 21209 Little Rapids Corp. Tissue(56) 56 31210 Neenah Paper(Kimberly-Clark Corp.) Ucfs(80) 80 31211 Stora Enso North America Corp. Spepack(160) 160 21212 Stora Enso North America Corp. Ctdfs(430) 430 1

(a) Status: Closed or idle, (1) Listed by the CPBIS. (2) Identified by KEMA.(b) The abbreviations of products can be consulted at the CPBIS web page.(c) Thousands of short tons (1 short ton = 2,000 lbs)Source: The Center for Paper Business and Industry StudiesWeb page: http://www.cpbis.gatech.edu/millsonline/wisconsin.php

D-1


APPENDIX D: PULP AND PAPER MILL SURVEY INSTRUMENT

KEMAIDCPBIS Mill IdContact Name

1 I would like to confirm that at this location your company produces pulp or paper? 1=Yes, 2=No

2a Do you generally participate in decisions concerning the specification, purchase, operation, or maintenance of energy using equipment at this location? 1=Yes, 2=No

2bDo you participate in these decision for facilities other than this one? 1=Yes, 2=No

2c IF ( Q2b = 1, "How many total facilities are you involved with?"," - N/A -") Number of facilities

3 What are the primary products produced at this location? Text

4 How many full-time employees work at this location? Number of employees

5aAt the present time, does your company have energy cost reduction goals?

1= Yes, in most instances, 2= Yes, in some instances, 3= No

5b Approximately what year did you start this practice? Year

5c IF( Q5b = 2005,"Approximately what month last year did you start this practice?", "- N/A -") Month

5d What prompted you to do this? Text

6aAt the present time, does your company have an annual budget assigned to energy management activities?





D: Pulp and Paper Mill Survey Instrument

D-2


7a At the present time, are there persons in your organization who have responsibility for identifying facility improvements to reduce energy use and costs?





8a At the present time, does your company compare energy consumption in your mill relative to an industry standard, otherwise known as benchmarking?





9aAt the present time, does your company collect and reuse mill hot water?





10aDoes your company use variable frequency drives on motors with variable loads?





11a

Does your company use high efficiency motors?





D: Pulp and Paper Mill Survey Instrument

D-3


12a Does your company periodically assess the operation of individual pump systems and make changes to ensure that the pump is operating at its best efficiency point?





13 Where do you go for information about your energy using equipment or how to save energy? Text

EFFECT OF BEST ENERGY PRACTICES GUIDEBOOK

14Are you aware of the Pulp and Paper Best Energy Practice Guidebook developed by Wisconsin's Focus on Energy Program? 1=Yes, 2=No

T&T IF( Q14 <> 1, IF (Q14 <> ""," >>> THANK AND TERMINATE <<< ", ""),"")

15 Have you consulted the Guidebook as a resource? 1=Yes, 2=No

16Using a scale of 1 to 5, where 1 means not all influential and 5 means very influential, how influential has the Best Energy Practice Guidebook been on your energy practices since July 1 2005? Scale: 1 to 5

17 Using the same scale of 1 to 5, how influential has Focus on Energy been on your energy practices since July 1 2005? Scale: 1 to 5

18 In addition to the more general Best Practices, the guidebook includes information on 57 specific equipment and process level Best Practices. Since July 1, 2005, have you used the Guidebook to identify potential technology or process-specific best practices related to your mill? 1=Yes, 2=No

T&T IF ( Q18 <> 1, IF (Q18 <> ""," >>> THANK AND TERMINATE <<< ", ""),"")

19 Since July 1, 2005, have you implemented any best practice which you identified in the Best Practices Guidebook? 1=Yes, 2=No

20 Please tell me what Best Practices you have implemented.

V Verbatim

P Notes

COMPLETE DATE

COMPLETE TIME

state of wisconsin department of administration division...

Documents