1. Tacoma Power’s Compaq Server and Ethernet RTU Performance Testing Presented By Joe Orth 253/502-8785 jorth@ci.tacoma.wa.us EMS Group Meeting Presented April 3, 2001

2. Tacoma Overview Municipal utility serving multiple jurisdictions in Pierce County governed by Utility Board that is approximately double the size of the Tacoma City Limits. The City operates Tacoma Public Utilities of which Tacoma Power is an operating division (along with Tacoma Water and Tacoma Rail). Within Tacoma Power there are 5 business units: T&D, Power Management, Energy Services, Click!Network and Generation

3. Tacoma Power Overview The City of Tacoma, Washington is about 30 miles south of Seattle in the South Puget Sound region Approximately 150,000 customers System hour peak 2/3/89 of 1,212 MW Currently Struggling to Overcome West Coast Power Market, Government Coup and 50 Year Drought Events

4. Tacoma Power Electrical System In Tacoma Power’s control area there are 4 major substations, 2 of which have 230 kV inter-ties with the BPA. Major substations are interconnected by a combination of 110 kV and 230 kV transmission lines. A 110 kV sub-transmission system distributes power from the major substations to Tacoma Power’s unit substations.

5. Tacoma Power Electrical System The typical unit substation has one 25 MVA 110 kV – 12kV transformer with four 12 kV, 800 Amp feeders which are often loaded to 600 Amps during peak hours. 50+ unit substations. 2040+ miles transmission and distribution lines. Average feeder length 2 to 3 miles. Click!Network has 600+ Miles of Fiber and Coax Installed in City Limits

6. ENERGY CONTROL CENTER and CLICK NETWORK OPERATION CENTER

7. SCADA System Overview New RTUs in 80+ locations. LAN and WAN (Nortel J-Mux) installation at about half of these sites. Provides 10 MB dedicated SCADA LAN acting as flat LAN. Other sites use various communications options. PC Based system communicating with RTUs using DNP over UDP/IP during system transition. Substation and Distribution Automation.

8. SCADA SYSTEM ARCHITECTURE PASS Network Design

9. SCADA System Overview New EMS. Desire to integrate RTUs on LAN directly with new EMS – signed contract with ABB September 2000. Preliminary testing of DNP Serial and Ethernet Data Acquisition Sub- system completed Feb. 2001 using Master Blaster RTU.

10. Master Blaster Overview

11. Pass/Fail Criteria No CPU Utilization 5 Second Average Over 60% LAN Utilization Less Than 20% Excluding One 3 Second Spike.5 Second Max Delay Responding to Operator Request Open One New Graphic In 2 Secs 95% of Time Verify Screens Update Every 2 Seconds Alarm Summary Page Shown on Each Console Verify Vendor DNP Profile Correct Verify Large Data Packets Processed Properly

15. Tests 1 and 2 – May 2000

16. RTU Communications for Test 1 and 2 98 Serial RTUs DNP 3.0 Protocol All Operating at 9600 Baud Each RTU has 408 Status and 408 Analogs Approximate Total 40,000 DIs and AIs Could Not Achieve 2 second Updates @ 9600

29. Test 1 & 2 LAN Data Scenarios

31. Test 3 – June 2000

32. RTU Communications for Test 3 98 Serial RTUs DNP 3.0 Protocol 8 Operating at 56 kbit (fills one ICP Card) Others Operating at 9600 Baud Each RTU has 408 Status and 408 Analogs Approximate Total 40,000 DIs and AIs

38. Test 4 – JANUARY 2001

39. RTU Communications for Test 4 85 RTUs DNP 3.0 Protocol 55 Operating at 56 kbps 30 On 10 MB Ethernet LAN Using UDP/IP These 30 RTUs Spread Over 2 IPs Each RTU has 408 Status and 408 Analogs Approximate Total 34,000 DIs and AIs Test in January 2001

40. Integration Story Due to dedicated, flat Ethernet LAN, confidence with DNP and other factors, UDP/IP was selected. Required TCP/IP for Telnet, etc. Require hot-stand-by on EMS FEP We had thought that this would be fairly simple and not require too much thought…

41. Other Issues #1 Issue – Vendor Specific!!! DNP Confusion LAN Document Interpretation Unique UDP Port Numbers? Unique DNP Addresses? How to Hot-Stand-By on FEP? Unsolicited messages?

42. Other Issues SOE Management? Understand your LAN environment.

43. Goals Up to 60 RTUs on LAN 2 Second Updates Most of the Time Take Advantage of Click!Network Fiber System Will Not Crash Due to Data Overload No Hardware Upgrades Needed for 6 Years No Proprietary Modifications to DNP Minimize Risk - Test to See If The System Could Perform Before Signing a Contract

44. What We Learned Most XP1000 Workstations replaced with single processor DS20Es GB LAN for Real-Time LAN Back-Up Systems Add About 3% of steady state LAN Traffic and 14% peak GS60E and ES40 Performance Close but GS60E Didn’t Have Deep Performance Valleys of ES40 Need Firmware Changes in RTUs to Have Ethernet Communication & Resolve RTU Latency Pre-Contract Award Performance Test Paid Off Big Time and Made Lawyer and Board Happy

46. LAN Activity and Robustness

47. Ethernet RTU Response Data