Moderator: Ari Reeves, CLASP
This session will focus on monitoring and evaluation of programs and policies in developing and middle-income countries. The first paper uses Romania as a case study to highlight the opportunities and challenges involved in moving from “monitoring only” to monitoring and evaluating climate change programs and policies. The second paper explains the concept of Energy Efficient Prosperity, which offers a new way of assessing and communicating the benefits of energy efficiency action that is attuned to the needs and circumstances of developing countries in particular. The third paper uses the case of South Africa to delineate between project-, program-, and policy-level evaluations, explains the linkages between the levels, and proposes an approach to carrying out program and policy evaluations. Abstracts of each of the three papers follow.
Enabling an Evaluation Culture: A Roadmap for Building Evaluation Frameworks for National Climate Change Strategies
This paper uses lessons learned from Romania’s experience and an extensive literature review to highlight opportunities and challenges for measuring and evaluating (M&E) national efforts to address climate change and low carbon growth, and is particularly relevant for middle income and former Eastern Bloc countries.
As signatories to the Kyoto Protocol and members of the European Union, countries like Romania have significant obligations to monitor, report and verify (MRV) efforts for greenhouse gas mitigation and climate change adaptation. Yet, these MRV requirements do not have a direct continuous learning or evaluation component.
Most policy-makers in middle income countries are still developing their understanding of what makes good climate change policy. A robust M&E system is essential to ensure that limited resources are used as fully, effectively and efficiently as possible, yet few countries have them. Countries focusing only on meeting international requirements miss a critical opportunity.
Countries should move beyond only monitoring indicators to develop evaluation frameworks that provide policy-makers with the means to judge whether climate policies meet the intended purposes, and to inform future efforts. Countries need practical low-cost approaches tailored to national circumstances that look downwards and upwards, where national decisions are informed by sub-national experiences and progress is shared internally and with the international community.
This paper provides a roadmap for developing climate policy M&E systems and a Romanian case study. It highlights key challenges and strategies to overcome them and outlines good practices, including the institutional capacities, roles and responsibilities needed, and stakeholder engagement strategies.
Energy Efficient Prosperity: A New Concept for Emerging Economies
Primary energy demand is expected to grow by nearly one third over the next 20 years; almost all of this growth will come from emerging economies and two thirds of the growth will be in Asia. The recent COP21 agreement provides a route to reduce carbon emissions but there is still a gap between the agreement at COP21 and the emissions reduction necessary to keep global temperature rise below 2 degrees Celsius above pre-industrial levels.
Analysis by the International Energy Agency (IEA) has demonstrated that the gap can be bridged through tried and tested cost-effective mechanisms. 49% of the carbon savings under this bridge scenario are through energy efficiency. However, only one third of the climate pledges at COP21 included energy efficiency measures among their actions to reduce carbon emissions.
Based on their work with emerging economy governments, the IEA identified a need for new indicators that reflected the multiple benefits of energy efficiency actions, local conditions, capabilities and aspirations; and that were practical for emerging economy policymakers to produce. This paper describes the work conducted by the IEA and their consultants to develop the concept of Energy Efficient Prosperity. This involved a review of literature relating to energy efficiency policies, targets and programmes in six emerging economies; research with around 100 stakeholders; a concept development process; and concept testing in detail with policy makers from three key emerging economies.
The paper recommends how evaluators can use the Energy Efficient Prosperity concept in both ex-ante and ex-post evaluation and describes the next steps that the IEA is taking in developing the concept and providing guidance on its use.
Energy Performance Evaluation Framework in South Africa
International performance evaluations on energy efficiency and demand side management (EEDSM) activities are usually conducted at three different levels, namely project level, program level, and policy level. In South Africa, M&V is largely done at project level, but there is more and more a need of evaluations on the program and policy levels. Currently, there is already a program evaluation guideline, but not consciously implemented. There is no policy evaluation guideline, and there is much confusion in the definition and differences of the three levels of performance evaluation. This paper aims to give clear definitions of the three levels of evaluations and share the South Africa experiences, which will further assist program managers, program evaluation panels, and M&V practitioners to prioritise important energy efficiency factors in design or evaluation of the programs and policies. Evaluations on EEDSM projects are usually based on the energy and power saving, with a brief emission reduction assessment. This is not enough for the evaluations at the program and policy levels since there are many other aspects need to be addressed. At the EEDSM program level, people are also interested in finding out the corresponding social and economic impact, which can be evaluated not only from conventional engineering point of view, but can also be evaluated from comprehensive environment, social, and economic aspects. Based on the information gathered from the project level and program level evaluations, the policy level evaluation is conducted by addressing the most important issues in terms of the key performance indicators from the engineering, environment, social, and economic aspects, in conjunction with the financial viability considerations.
PAPERS / PRESENTATIONS
Enabling an Evaluation Culture: A Roadmap for Building Evaluation Frameworks for National Climate Change Strategies [paper]
Julia Larkin, IDEAS for Energy
Liviu Gheorghe, eco2ro
Viorel Blujdea, Independent Consultant
Energy Efficient Prosperity: A New Concept for Emerging Economies [paper]
Charles Michaelis, Strategy Development Solutions Ltd.
Melanie Slade, International Energy Agency
Energy Performance Evaluation Framework in South Africa [paper]
Jiangfeng Zhang, University of Strathclyde
Xiaohua Xia, University of Pretoria
Xianming Ye, University of South Africa