Presentation is loading. Please wait.

Presentation is loading. Please wait.

Prepared by Opinion Dynamics Corporation May 2006.

Similar presentations


Presentation on theme: "Prepared by Opinion Dynamics Corporation May 2006."— Presentation transcript:

1 Prepared by Opinion Dynamics Corporation May 2006

2  Opinion Dynamics was contracted to conduct a primary research effort to measure Market Participant perceptions of ERCOT’s performance with respect to meeting its responsibilities.  Follow up and expansion of the 2004 Market Participant Survey.  Results allow for comparisons between market perceptions and operational realities and trending of performance over time. Introduction

3 Three Phases: Methodology Phase 1: Review 2004 Market Participant Survey Instrument Phase 2: In-depth interviews with key ERCOT stakeholders Phase 3: Survey of ERCOT’s Market Participants

4 ODC developed a sample of 920 unique Market Participants using the following lists provided by ERCOT staff:  Appropriate points of contact at market participant firms provided by Account Managers  Current ERCOT Board members  Current Committee members Survey Sample

5 Response Rate 287 completed surveys from a sample of 920 Market Participants (31%) ERCOT Board Membersn=5 Committee Membersn=91 Market Participant Staffn=191 Classification of respondent based on self-selected descriptions – QA1 Response Rate

6 By Market Participant Firm Type

7 Survey Approach: 10 point scale  Many questions based on a 10 point scale: 1-3 = negative response, 8-10 = positive response.  Mean responses will trend toward the middle of a 10 point scale – only those with passionate opinion are likely to provide a rating in top or bottom 3.  In general, mean responses of 6.6 or above are favorable ratings, 7.5 and above are extremely positive responses.

8 Background and Context Market Participant Opinions Regarding ERCOT Staff’s Role In Developing Market Rules Findings Board Member (n=5) Committee Member (n=91) Market Participant Staff (n=191) ERCOT staff should take a position in the development of market rules 60%20%21% ERCOT staff should provide individual professional opinion 40%25%17% ERCOT staff should support the development of market rules by providing data and analysis only -46%27% ERCOT staff should only administer the market rules -3%9% Undecided-5%27%

9 Background and Context Market Participants’ Understanding of ERCOT’s Committee Structure Findings

10 Background and Context Market Participants’ Understanding of ERCOT’s Committee Structure Findings

11 Background and Context Market Participants’ Understanding of ERCOT’s Committee Structure (cont.) Findings

12 Background and Context Interest in Future Training Findings

13 Overview of Perceived Strengths  Performance of ERCOT staff -- particularly account managers -- officers and directors  Communications -- ERCOT website  Timeliness, accuracy and format of data provided  Providing effective training Findings

14 Color Key 6.6 to 7.4 7.5 or above 6.5 or below 2006 2004 2006 2004 2006 2004 Needs Improvement Good Very Good Findings

15 ERCOT Staff Performance: Corporate Objectives (10 point scale, means shown) Findings Ensuring Reliability/Adequacy of Grid Nondiscriminatory access to transmission/ distribution Accurate accounting of electric production & delivery Timely information about customer’s choice of REP 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level.

16 ERCOT Staff Performance: Staff (10 point scale, means shown) Findings Consistency Attitude Industry expertise Responsiveness to Market Participants Overall Performance 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level.

17 ERCOT Staff Performance: Officers & Directors (10 point scale, means shown) Findings Consistency Attitude Industry expertise Responsiveness to Market Participants Overall Performance Management of ERCOT organization 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004

18 ERCOT Staff Performance: Account Managers (10 point scale, means shown) Findings Timely response Knowledge/ Industry Expertise Response accuracy Direction of inquiries Accessibility Attitude/Willingness to resolve problem Overall expectations 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level.

19 ERCOT Staff Performance: Functional Areas (10 point scale, means shown) Findings Market Participant Registration Systems Testing Retail Transaction Processing/ Customer Switching Bi-Lateral Scheduling Grid Operations Settlements and Billing Settlements Dispute Resolution * Significantly different from 2004 at 95% confidence level. 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004

20 ERCOT Staff Performance: PRR (10 point scale, means shown) Findings Timeliness Testing software changes Administering the process Understanding the effects Effective communication Implementing projects 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004

21 ERCOT Staff Performance: Metering* (10 point scale, means shown) Findings Identifying need for data edits Accuracy of data edits Knowledge Completing data edits Timeliness of data edits *New Question in 2006

22 ERCOT Staff Performance: Variance Disputes (10 point scale, means shown) Findings Retail Variance Retail Transaction Variance Data Extract Variance 2004 2006

23 Staff Performance: Finance and Accounting Credit Standards* (10 point scale, means shown) Findings Effectively communicates collateral requirements to market participants Responds well or effectively to questions regarding collateral requirements Responds in a timely manner to questions regarding collateral requirements * New questions in 2006.

24 Communications (10 point scale, means shown) Findings Amount of information Clarity of ERCOT Staff’s Messages Written Communication Verbal Communication Timeliness Use of Appropriate Mailing Lists** 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level. ** New question in 2006.

25 Communications: Functional Areas (10 point scale, means shown) Findings Asset Registration Systems Changes Bidding Systems Planning 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level.

26 Communications: Functional Areas* (cont.) (10 point scale, means shown) Findings Collateral requirements Grid operations Changes to IT systems Congestion management * New questions in 2006.

27 Satisfaction with the Website (10 point scale, means shown) Findings Timeliness of posted information Look of the site Search functionality** Ease of finding documents and info. 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level. ** New question in 2006.

28 Timeliness of Data (10 point scale, means shown) Findings Settlements Bill Transmission Congestion Rights Renewable Energy Credits Data Extracts 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level.

29 Accuracy of Data (10 point scale, means shown) Findings Settlements Bill Transmission Congestion Rights Renewable Energy Credits Data Extracts 2006 2004 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level.

30 Format of Data* (10 point scale, means shown) Findings Settlements Bill Transmission Congestion Rights Renewable Energy Credits * New questions in 2006.

31 Effectiveness of Training (10 point scale, means shown) Findings Settlement and Dispute Seminars Retail Training Seminars System Operations Seminars 2006 2004 2006 2004 2006 2004 * Significantly different from 2004 at 95% confidence level.

32 Overview of Areas for Improvement  Spending priorities  IT Technical Helpdesk  Finance & Accounting Credit Standards  Project Management Findings

33 ERCOT Spending Practices (10 point scale, means shown) Findings Spending Funds Equitably Spending Funds on Things that are Important to Your Company Spending Funds Cost Effectively 2006 2004 2006 2004 2006 2004

34 Usefulness of Technical Helpdesk (10 point scale, means shown) Findings * Significantly different from 2004 at 95% confidence level.

35 Finance and Accounting Credit Standards* (10 point scale, means shown) Findings Credit exposure and related collateral requirements are calculated consistent with protocols Collateral requirements are adequate to mitigate the credit exposure in the market Collateral requirements and credit standards are fairly implemented across the market * New questions in 2006.

36 ERCOT Project Management Office (10 point scale, means shown) Findings Overall performance Consistently applying cost-benefit analysis** Communicating progress of projects Spending money effectively Knowledge/Industry expertise Implementing projects based on system changes 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 2006 2004 Implementing projects that meet expectations ** New question in 2006.

37 Market perceived areas of strength:  Performance with respect to corporate objectives  Systems are providing timely and accurate data  ERCOT staff performance across virtually all functional areas  Performance of Account Managers Conclusions

38 Specific areas for strategic consideration:  Market’s understanding of the role of committees / governance in setting spending priorities  Defining and communicating ERCOT staff’s market function  The market is critical of ERCOT’s project management -- More research on this issue is necessary. Conclusions (cont.) Conclusions


Download ppt "Prepared by Opinion Dynamics Corporation May 2006."

Similar presentations


Ads by Google