ATTRIBUTION: HIGHWAYS AND BYWAYS
Moderator: Iris Sulyma, Research 4 Results
A major challenge to energy policy and programme evaluators is discerning impacts caused by programmes (both past and present) from impacts caused by other market forces: that is, attribution of only incremental or additional impacts. This session looks at attribution of energy impacts by:
- providing a comparison of attribution estimates based on surveys of programme participants and programme trade ally participants;
- presenting an innovative, theory-based approach to collecting and analysing sales data from trade allies to estimate attribution; and
- presenting an examination of the influences that programmes have on the structure and operations of the market itself, particularly dommestic manufacturers, in the context of multiple policies and fluctuating market conditions.
Violette and Agapy-Read’s paper outlines a rationale for estimating net savings factors, e.g., free ridership (FR) and spillover (SO), which can be used to adjust savings values that use business-as-usual or common-practice baselines. A survey-based application is presented using three sources of information: (1) programme participant fast feedback surveys (conducted soon after the customer’s decision to participate), (2) programme participant end-of-year telephone surveys, and (3) programme participant trade ally telephone interviews. This allowed for triangulation of results, tests of consistency, and sensitivity analyses – all part of a “best-practices” application. In the conclusions, several issues brought up in the literature are addressed concerning the reliability and cost-effectiveness of survey approaches for addressing attribution compared to other methods used to assess attribution.
Jane Hummer presents an innovative, theory-based approach to collecting and analysing sales data from trade allies to estimate net impacts and increase confidence in impact evaluation results. The fundamental difference between this approach and a more traditional “net-to-gross” analysis is that the analysis starts with an estimate of total market adoption of an energy efficiency measure, rather than programme-reported gross impacts. This approach relies on trade allies to provide sales data that allows the analyst to estimate the total change in efficiency adoption from the pre-programme time frame to the present. The total increase in efficiency adoption then can be easily estimated. Some share of that increase (between 0% and 100%) is due to the programme’s influences. The programme-influenced change in efficiency sales is compared to the number of programme-rebated sales to estimate the net programme impacts. This paper will discuss methods for developing data collection tools based on programme logic, recruiting trade allies into the study, asking the right questions to ensure unbiased and complete responses, structuring and conducting transparent analyses, and synthesizing the results with other research findings. The paper also highlights lessons learned and results from applications of this method on various evaluation projects.
Reeve et al present findings from some new research into how energy efficiency policies for appliances, lighting, and industrial equipment have affected the manufacturers of these products. The paper examines specific cases from countries around the world where Minimum Energy Performance Standards (MEPS) comparative labels, and complementary policies have been used to achieve energy savings while bolstering domestic manufacturers. Obviously results vary. Appliance policies are not an unqualified benefit to domestic manufacturing industries in all cases. This paper explores why manufacturers have fared well in some cases and not so well in others, and explains how complementary policies such as tax credits and direct technical assistance have been used to help manufacturers transition to producing more efficient products.
PAPERS / PRESENTATIONS
Programme Attribution Analysis: Use of Self-Reports and Triangulation [paper]
Daniel Violette, Navigant Consulting, Boulder CO, US
Laura Agapay-Read, Navigant Consulting, Boulder, CO US
How have Energy Efficiency Standards and Labeling Policies Affected Product Manufacturers? [paper] [presentation]
Ari B. Reeves, CLASP, Washington, DC, USA
Amit Khare, CLASP, Washington, DC, USA
Yang Yu, CLASP, Washington, DC, USA