1. Results Based Drafting for WECC Drafting Teams W. Shannon Black Manager, Standards Processes

2. Overview ● Why change? ● Changes to the process  Scope  Early determine of the roadmap ● Changes to the documents  New template  Where things go 2

3. Drafting Process / Sequencing

4. Why change? ● “R”equirements need to improve  Current “R”equirements  Are unclear  Administrative not results  Impossible to achieve! Measure!  Voluntary  Devoid of drafting conventions ● Failure to identify the work product up front is slowing the Process ● Where are we headed?

5. Where are we Headed? ● Emphasis on  Defining the Scope (product)  Before we draft  Defining the Requirements  Before we draft  Using proper drafting conventions  Using common sense  Using a sequencing concept  Defining the Review the process  Quality control review at each posting

6. Define the Scope Get consensus before writing requirements NEED: Why the standard is being developed or modified GOALS: What must be accomplished to meet the need OBJECTIVES: Criteria for success OPERATIONAL CONCEPTS: Day in the life STAKEHOLDERS: Individuals or groups impacted or interested DRIVERS: External constraints

7. Define the Requirements Necessary:It is needed? Meet one of the Reliability Principles included in the Request? Redundant? Attainable: Can you meet it? Clear: Can be understood only one way? Measures:Can we comply? Write results-based requirements

8. Review and Approve Collect comments Revise in response to comments Recycle per planned schedule 45 and 30 day Comment windows Complete a quality review check list Submit for approval Team votes – forwards to the Standard Committee

9. Scope What you need to know and agree upon in order to develop a standard and write requirements Gathering the tools for the tool box

10. Defining Scope ● View the maze from the top!  See the problems before wasting time ● You think you know…  …but you don’t know you know. ● Defining scope  Defines the boundaries  Defines the target

11. Defining Scope Operational Concepts/ Defining the Problem Identify the OPERATIONAL CONCEPTS Understand the operational fact pattern What specifically happened to create the problem your team will try and resolve? How do we address it today? Make a list of the steps that are taken TODAY to meet the operational concern How should we address it tomorrow? Make a list of “first blush” suggested changes to the existing operations. 11

12. Defining Scope Identify the STAKEHOLDERS Who was impacted by the fact pattern? Who will be impacted by the remedy? Identify the DRIVERS What’s pushing this whole effort? Get the “agendas” out on the table 12

13. Defining Scope Why identify the stakeholders and drivers? The impact is broader than the Functional Model Vested Interested Lineman / Back Office / After-the-Fact / Schedulers “Mom and Pop” Influential Interests Government Stock holders Participating Interests Usually the “Applicable Entities” 13

14. Defining Scope Why identify the stakeholders and drivers? The end product has to reflect reality Consider potential drivers Technology Deadlines Existing Standards / Existing Processes Personal and Corporate agendas End user expectations 14

15. Purpose Statement What you need to know and agree upon in order to develop a standard and write requirements Gathering the tools for the tool box

16. Defining Scope Focusing on the Purpose Statement Identify the NEED Why the document is being developed or modified? “To ensure uniform maintenance procedures…..” Identify the GOALS/OBJECTIVES What do we have to do to meet the needs? “…by defining maintenance periodicity.” Lays out the criteria for success 16

17. Defining Scope “Purpose” Exercise 17

18. Defining Scope “Purpose” Exercise ● What’s wrong with this picture? ● Purpose Statement for EOP-004-1  “Disturbances or unusual occurrences that jeopardize the operation of the Bulk Electric System, or result in system equipment damage or customer interruptions, need to be studied and understood to minimize the likelihood of similar events in the future.” 18

19. Defining Scope / “Purpose” Small Group Exercise ● What’s missing? ● What’s included that need not be? ● Would it help to know the fact pattern? ● What’s the intent (goal/objective) of the document?  Can you tell? Can an auditor tell? ● How do we know when we “did it”?  Can we compare the Purpose statement to the final document and know “we’re done”? 19

20. Defining Scope “Purpose” statement ● Suggested rewrite  NEED  “To improve BES reliability…”  GOAL  “…by reporting and analyzing Impact Events…”  Objective  “…for the purposes of minimizing repeat events.” ● This is just one solution 20

21. Drafting the Purpose Small Group Discussion ● Review the Team’s Requests  Identify the fact pattern and issues  Sometimes called Operational Concept  Consider what is being done now and what changes need to be made ● Identify the NEED  E.g.. “To create a standardized process…” ● Identify the GOAL  E.g.. “…used by every Balancing Authority…” ● Identify the OBJECTIVE  E.g.. “…to measure Time Error Correction.” 21

22. Requirements

23. Regulatory Facts of Life ● Requirements are boring  This reflects uniformity and pattern ● Requirements look the same  “Each BA shall do X.” ● Requirements are a “language”  People don’t write like they talk  The spoken word is communicated differently than the written word  People dislike taking time to write at all!  Your job is to take the time 23

24. Regulatory Facts of Life Things to Correct ● Don’t provide adequate reliability  “Documentation heavy” ● Can be interpreted different ways  Subjective terms  Poor grammar  Multiple requirements in a single statement ● Don’t clearly state responsibility ● Can’t be measured ● Can’t be enforced 24

25. Regulatory Facts of Life Drafting Conventions ● Avoid Ambiguous Terms  Adverbs (words that end in “ly”)  Pronouns (it…this…)  And / Or  Associated / Affected / Accommodate  Coordinate / Facilitate  Maximize/Minimize  Unique / “Mutually agreed upon”  “be able to” “capable of”  Wouldn’t you rather they just do it? 25

26. Regulatory Facts of Life Drafting Conventions ● Do  Use the structure  Formatted style of language  NERC Functional Model  Make sure the newest staff member understands the document ● Don’t  Incorporate by reference  Create new terms  Abbreviate 26

27. Results Based Requirements Lock Down the Structure ● Basic components  Identifies each affected party  “Each Balancing Authority…”  Is mandatory  “…shall…”  Identifies the task  “…operate [in a specific fashion].”  To achieve a particular result  “…to ensure a balanced interchange.”  Measureable and objective

28. Results Based Requirements Create an Affirmative Result ● To create an Affirmative Requirement  Meet the Purpose of THE ASSIGNED document  Simply because it’s an excellent requirement doesn’t mean it belongs in “this” standard  Increase the reliability of the system  Meet one of the NERC reliability principles  Not just admin for the sake of admin  At an affordable cost (Common Sense Test)  Is actually feasible / possible / affordable  Internally and externally consistent

29. Results Based Requirements Understood the same way by All FERC Electrical Providers NERC If a team sees multiple interpretations – rewrite it! If the newest guy in your shop can’t understand it – rewrite! If your supervisor can’t understand it – rewrite it! If your friends don’t understand the sentence structure – rewrite!

30. It’s New! Enhance your Requirements ● It’s acceptable to use  Graphs  Pictures  Charts  Examples of how the Requirement works  Sample calculations

31. When Requirements go wild! Exercise

32. What’s wrong with this Picture? ● Responsible entities may schedule as appropriate. ● Each TO shall eliminate all lower quality timing loops that occur when two network devices line time off of each other under normal conditions without traceability to a Stratum 1 reference. ● Each Balancing Authorities shall perform all calculations based on the March 10, 2010 Table found at….” ● Each Scheduling Entity shall schedule in accordance with NAESB e-Tag requirements.

33. Rationale

34. What is the Rationale block? 34

35. Why have a Rationale block? “In today’s world of high employee turnover and complex, long-lived products, well- documented rational is essential to better product development.” Ivy Hooks. ● Benefits  Key to understanding  Reduce interpretation problems  Facilitate maintenance and upgrades  Preserve corporate knowledge 35

36. What goes in the box? ● Why a requirement is needed ● What assumptions were made ● What analysis effort drove the requirement ● Information to help understand the requirement ● Source of any numbers 36

37. What does NOT go in the box? ● A rewrite of the requirement ● Hidden requirements ● Copy of another requirement’s rationale ● Everything you know on the topic 37

38. The Requirement must Stand Alone ● “Third, we understand that the proposed results-based standard format may include expanded background sections, expanded descriptions of a Reliability Standard’s purpose, and/or explanations about the intent of individual Requirements. This information may provide useful context but it should not contradict or seek to supersede or interpret the requirements within a Reliability Standard. The requirements within the Reliability Standard should govern, and the application of the Standard should be clear without reference to the background or purpose sections.”  Docket Nos. RR09-7-000 and AD10-14-000, P. 109

39. Measures

40. Measures have Moved 40

41. Measures ● One per customer!  One Measure per Requirement  Draft upon completing the Requirement ● Each Measure must:  Identify the functional entity  Tangible, practical, objective  Identify what evidence or types of evidence could be used to show that an entity is compliant with the requirement. 41

42. Comments

43. Dealing with Comments ● Important to consider all carefully ● Not important to agree to all ● Don’t waste time playing the assumptions game ● Do not hesitate to ask for clarification ● If many comments from one person or group – ask for priorities. ● Categorize if necessary 43

44. Questions?

45. Getting Started ● What’s the issue/fact pattern?  Review the Request ● What needs to be fixed? ● How do we do it now?  List the “Day in the Life of” today ● How do we propose to fix it?  List the proposed “Day in the Life of” for tomorrow  This is the skeleton for our document!