'Best practices in application of sampling under PoA framework: DOEs perspective' Workshop on Programme of Activities (PoA) under the CDM: Challenges and Road Ahead 07 th - 8 th May 2011, Bonn UNFCCC Secretariat

2 Contents Data sampling – Approach (general) Data sampling & Verification - Best practice example CDM Methodologies (general) Sampling plan- issues to be addressed

3 Data sampling – Approach Sampling determines the reliability of the parameter value estimate expressed in terms of probability of a parameter falling within a specified interval around a parameters true value Tools/guidance (sampling guidance EB50, annex 30) Methodological guidance, otherwise 90/10 confidence/precision as the criteria for reliability of sampling Over estimation of the mean value should be avoided, unbiased and reliable results ensured Simple random sample most commonly used approach each observation chosen randomly and entirely by chance presents unbiased estimate of true population, suitable for relatively homogeneous population Systematic sampling, Stratified random sample, Cluster sampling, Multi stage sampling Explanation of why the sample size is reliable and accurately reflects the population Statistically sound sampling 95/5 precision (95% confidence interval and 5% margin of error) 90/10 precision (90% confidence interval and 10% margin of error) 90/30 precision (for eg. leakage) Unique IDs- double counting avoided, verification status can be determined at all times

4 Real Projects- Verification Approach Gold Standard methodology for Improved Cook-stoves and Kitchen Regimes V.01 ( CDM Meth AMS IIG) 6627 stoves were installed during the first year of the crediting period Scalable to 1 Million plus installations Verification challenges (MP1) Access, Travel time Limited annual ERs (~15000) – Costs On site - Man days Population characteristics, record and installation database of every stove constructed incl. family name, ID number, location, and date of construction for all the households that receive a stove 1,788 monitoring surveys that include leakage, sustainability and qualitative fuel use data ProvinceMunicipalitiesApproximate population Santa Barbara28342,054 Copan23288,766 Lempira28288,766 Intibucá17179,862

5 Verification approach Statistical justification, Reasonable Assurance Key monitored parameters Number of stoves installed (~350 samples) Continued use of stoves over time – Drop off rate (alteration from original configuration) –PDD states that this shall be done through a survey of first 50 beneficiaries who had stoves installed within the first 12 months of the start crediting period –MR1 stated that a total of 1787 households were surveyed in 2009 and 2010; of these only 28 stoves were found to be out of use - drop off rate of 1.57% –Does the sample selected represent the entire population (spatial and temporal) - first 50 beneficiaries only? –Verification of 30 households in one locality (1/2 man day) –random sample indicated 3 drop off (indicating 10% drop off) –3 more localities selected, 30 h/h each –Sample size increased to 120 households across three randomly selected locations (samples) –Drop off rate found constant (7.5%) across the selected samples (90% confidence level, 5-10% error margin)

6 Verification Approach Statistical Justification, Reasonable Assurance Emission reduction achieved per stove per year »Registered PDD mtCO 2 e/year per stove (qualification attached- paired sample test) »MR mtCO 2 e/year per stove Paired sample Kitchen Test for annual emission savings per stove (2010 Paired Fuelwood Consumption Study) Measure daily fuel consumption over a 4-day period in 50 households stoves, random selection, monitored before adoption of the La Justa 2x3 vis-a vis traditional stove (fogon) Actual sample size taken by PP was larger (reaching n=55); and wood was weighed over a 5-day period (not just a 4-day period as required), resulting in four 24-hour periods of fuelwood consumption data rather than 3 Confidence level- 90%, SD of population Reliability directly proportional to numerical sample size Large sample size and paired design, sampling approach, assumptions & justification of approach transparently reported Eliminated systematic underestimation Original datasets available, Yale university staff (Third Party) - on site

7 The sampling plan submitted by project proponents is reviewed using to assess a range of issues and questions, such as: Does the sampling plan present a reasonable approach for obtaining unbiased, reliable estimates of the variables? Is the data collection/measurement method likely to provide reliable data given the nature of the parameters of interest and project, or is it subject to measurement errors? Is the population clearly defined and how well does the proposed approach to developing the sampling frame represent that population? Does the frame contain the information necessary to implement the sampling approach? Is the sampling approach suitable, given the nature of the parameters, the data collection method, and the information in the sampling frame? Is the proposed sample size adequate to achieve the minimum confidence/precision requirements? Is the ex ante estimate of the population variance needed for the calculation of the sample size adequately justified? Are the procedures for the data measurements well defined and do they adequately provide for minimizing non-sampling errors? Is the quality control and assurance strategy adequate? Are there mechanisms for avoiding bias in the answer, including possible fraud? Are the persons conducting the sampling activities qualified? (Ref. EB50, annexure 30)

8 Methodologies (CDM Portfolio) AMS ID : Hydro Projects, AMS IIIF (Municipal Waste), AMS IIG (Efficient cook stoves), AMS IC (eg. Solar thermal), AMS II J (CFL Lighting) Sampling approach justified, consistent monitoring plan applied across all CPAs, technical specifications are similar, operational criteria/equipment provider same Statistically representative Issues: temporal and spatial aspects AMS ID & AMS IF (Hydro, Wind, Biomass, Photovoltaic) & other methodological combinations More complex Sampling approach for each category

9 Employs 59,000 people and operate a network of more than 1,000 offices and laboratories around the world Largest verifier of CERs issued under CDM Questions? Thank You Contact