1 ERCOT LRS Precision Analysis PWG Presentation June 28, 2006

2 Options for Determining Round Two Sample Size Increases Option 1: –Determine minimum sample size increase needed to obtain ±10% Accuracy at 90% Confidence for a selected percent of intervals for the year independently for each Profile Type / Weather Zone Combination Option 2: –Determine minimum sample size increase needed to obtain ±10% Accuracy at 90% Confidence for a selected percent of intervals such that all Profile Type / Weather Zone Combinations meet the accuracy target in all selected interval Option 3: –Determine minimum sample size increase needed to obtain ±10% Accuracy at 90% Confidence for enough intervals to account for a selected percent of the MWh for each Profile Type / Weather Zone Combination Option 4: –Determine minimum sample size increase needed to obtain ±10% Accuracy at 90% Confidence for enough intervals to account for a selected percent of the dollars (ΣMWh * MCPE) for each Profile Type / Weather Zone Combination Option 5: –Iteratively allocate increments of 10 sample points to the Profile Type / Weather Zone Combination which produces the most gain in terms of reducing MWh estimation error Option 6: –Iteratively allocate increments of 10 sample points to the Profile Type / Weather Zone Combination which produces the most gain in terms of reducing Dollar estimation error

3 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type and Weather Zone - Independent of Interval Minimum Sample Size Increase Required For example: to obtain ±10% Accuracy at 90% Confidence for 50% of the intervals for BUSHILF_COAST would require a sample size increase of 10 points

4 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type and Weather Zone - Independent of Interval Minimum Sample Size Increase Required

5 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type and Weather Zone - Independent of Interval Increase of Sample Size

6 Option 1 To obtain ±10% Accuracy at 90% Confidence By Profile Type - Independent of Interval For example: to obtain ±10% Accuracy at 90% Confidence for 50% of the intervals for all Profile Type/Weather Zone combinations would require a sample size increase of 5,358 points

7 Option 2 To obtain ±10% Accuracy at 90% Confidence for all Profile Types and Weather Zones - Within Each Interval For example: to obtain ±10% Accuracy at 90% Confidence for 50% of the intervals such that all Profile Type/Weather Zone combinations within those intervals would achieve that level of accuracy would require a sample size increase of 750 points for BUSHILF and 2,267 points for BUSMEDLF

8 Option 2 To obtain ±10% Accuracy at 90% Confidence for all Profile Types and Weather Zones - Within Each Interval For example: to obtain ±10% Accuracy at 90% Confidence for 50% of the intervals such that all Profile Type/Weather Zone combinations within those intervals would achieve that level of accuracy would require a sample size increase of 5,703 points for BUSLOLF and 7,047 points for BUSNODEM

9 Option 2 To obtain ±10% Accuracy at 90% Confidence for all Profile Types and Weather Zones - Within Each Interval

10 Option 2 To obtain ±10% Accuracy at 90% Confidence for all Profile Types - Within Each Interval For example: to obtain ±10% Accuracy at 90% Confidence for 1% of the intervals such that all Profile Type/Weather Zone combinations within those intervals would require a sample size increase of 7,749 points

11 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type and Weather Zone For example: to obtain ±10% Accuracy at 90% Confidence for intervals accounting for 50% of the BUSMEDLF MWh would require a sample size increase of 170 points

12 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type and Weather Zone

13 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type and Weather Zone

14 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type Continues on next slide For example: to obtain ±10% Accuracy at 90% Confidence for intervals accounting for 50% of the MWh for each of the Profile Type / Weather Zone combinations would require a sample size increase of 4,322 points

15 Option 3 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the MWH within Each Profile Type

16 Option 4 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type & Weather Zone Note: Dollars = Σ (MWh * MCPE)

17 Option 4 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type & Weather Zone

18 Option 4 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type & Weather Zone

19 Option 4 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type Continues on next slide For example: to obtain ±10% Accuracy at 90% Confidence for intervals accounting for 50% of the Dollars for each of the Profile Type / Weather Zone combinations would require a sample size increase of 3,673 points

20 Option 4 To obtain ±10% Accuracy at 90% Confidence for Intervals Accounting for Selected Percents of the dollars within each Profile Type

21 Precision vs Sample Size Increasing sample size has a diminishing return on precision improvement Error Ratio (thus Precision improvement) varies across Profile Types / Weather Zones and across intervals Thus the impact of adding sample points varies by Profile Type and Weather Zone

22 Class Level MWH & Dollars Not all Profile Type / Weather Zone Combinations are created equal in either MWh or Total Dollars (ΣMWh * MCPE) !

23 Class Level MWH & Dollars Not all Profile Type / Weather Zone Combinations are created equal in either MWh or Total Dollars (ΣMWh * MCPE) !

24 Class Level MWH & Dollars Not all Profile Type / Weather Zone Combinations are created equal in either MWh or Total Dollars (ΣMWh * MCPE) !

25 Class Level MWH & Dollars Totals from previous three slides. Is accuracy more important for RESLOWR (41% of Dollars) than for BUSNODEM (2.1% of Dollars)?

26 Class Level MWH & Dollars - Descending Order by Dollars * Note: Dollars = Σ (MWh * MCPE) Continues on next slide Top 4 classes account for 49% of the MWh and 51% of the dollars

27 Class Level MWH & Dollars - Descending Order by Dollars * Dollars = Annual MWh * MCPE Continues on next slide

28 Class Level MWH & Dollars - Descending Order by Dollars * Dollars = Annual MWh * MCPE Continues on next slide

29 Options 5 & 6 These options iteratively allocate increments of 10 sample points to the next Profile Type / Weather Zone Combination in order to produce the most gain in –reducing MWh (Option 5) estimation error (Precision × MWh) summed across all intervals –reducing Dollar (option 6) estimation error (Precision × Dollars) summed across all intervals The allocations are based on –The MWh (or Dollars) associated with each of the Profile Type / Weather Zone combinations in each interval –The Error ratio in each interval for each Profile Type / Weather Zone combination –The cumulative number of sample points allocated by preceding iterations (including the original sample size) –The precision improvement that would be realized by adding 10 sample points, and the diminishing return on that improvement

30 Option 5 – MWh Error Reduction Optimization Cumulative sample sizes are shown in increments of 1,000; they were determined iteratively in increments of 10 sample points as additions to the current sample size.

31 Option 6 – Dollar Error Reduction Optimization

32 Option 5 with Collapsed Profiles/Weather Zones Collapse classes to reduce the number for which the few or no additional sample points are required. BUSNODEM would no additional points.

33 Option 6 with Collapsed Profiles/Weather Zones

34 Conclusions and Follow-up Analysis The iterative sample point allocation process has some intuitive appeal –Seems to allocate sample points where they do the most good –Maximizes UFE reduction However, –How does UFE allocation affect the final accuracy? –If the iterative allocation process is used, will we end up with more or less accurate estimates when they are adjusted for UFE? –ERCOT will try some Monte Carlo simulations to explore this question