1. Session Title Smart Grid Communication Architecture
Speaker: Doug McGinnis
Company: Exelon
Session Title
Smart Grid Communication Architecture

2. 2
Smart Grid at Exelon
Smart Home / Business
Real-time usage and pricing statistics
Home Area Network (HAN) composed of smart devices and appliances that know the price of energy
Smart Meters (AMI)
A method to enable two-way information flow
System status, customer outage status, usage and pricing signals delivered to and from location
Smart Distribution System
Real-time reporting of status and outages
Automated controls of relays and reclosers. Efficient field force management
Effective interconnection of renewable energy sources
Smart Utility
More efficient data collection, processing and back office functions
Asset Monitoring
Leveraging integrated communication systems and information processing is critical
Customer End-Use
Advanced Pricing & Billing
In-home Devices
PowerCo
Many working definitions of Smart Grid
Electricity Advisory Committee to U.S. Department of Energy defines Smart Grid as “a broad range of solutions that optimize the energy value change”
Smart Grid leverages advancements in communication and information processing technologies to enable a highly automated integrated intelligent network
Integrated communication systems
Information processing
Enhanced operator controls, decision support, demand response, asset condition assessment
Advanced components (Automated switches, intelligent meters, microprocessor relays…)
Advanced sensing and measurement
Time of use pricing, outage detection and notification, energy management systems, power quality monitoring, etc.
Advanced control
Automatic restoration and isolation, control of distributed resources, etc.
Facilitates the interconnection, operation, and grid management of renewable energy sources (wind, solar, PHEVs). There are 900MWs of wind generation currently connected to the ComEd system with pending interconnection requested for over an additional 20,000MWs. Current grid technologies can only absorb so much of these intermittent energy resources without jeopardizing the stability of the grid.
The Smart Grid Home Area Network (HAN) is a network of smart devices within the home which provides for communications between these devices and may include: utility meters, appliances and in-home monitoring and control devices, e.g., the smart thermostat. The HAN is envisioned to empower customers with information and the ability to control their energy costs and improve our environment.
1234
Vehicle Electrification
Plug-in Hybrid Electric Vehicles
Customer Gateway
Advanced Metering Infrastructure (AMI)
Grid Modernization
Distribution Automation
Smart Substations
Renewable Interconnection

3. Fundamental Design Principles
Security – Robust end-to-end, aligned with NISTIR 7628
Deterministic – Smart Grid applications will share a logically isolated deterministic communications infrastructure.
Interoperable – Industry standard protocols will be utilized with a focus on migrating to IP/Ethernet consistent with industry direction
Privately Owned – Privately owned communications is preferred. Eliminate telecom circuits, O&M cost savings.
No Unanalyzed Single Points of Failure (Self Healing) – The communication architecture will be designed with no unanalyzed single points of failure.
Communications Maintenance Management & Monitoring – Inherent to the communications Architecture will be Communications Maintenance Management & Monitoring, i.e. the ability to maintain, monitor and control network devices.
Tiered Architecture – Unit/Distribution substations are linked to larger Transmission substations in a hierarchical design through new wireless technologies
Relay Protection Communications – Highest level of reliability and availability including diverse backup paths.

4. Architectural Framework
Substation Application Portfolio – 6 application groups exist in the substation and are logically partitioned over common transport
Telemetry – RTU/IED – EMS/DMS communications (encrypted)
CIP Telemetry – NERC CCA sites (encrypted)
DA – Field Distribution Automation traffic aggregated by TGB (encrypted)
Enterprise – Business applications ( , VoIP, surveillance video)
Security – Card readers
AMI – TGB backhaul handoff to Core PoP
Tier 1 – Substation will be a point-of-presence to the SONET backbone.
SONET infrastructure provides Ethernet and TDM provisioning
Routing capability to permit application provisioning over Ethernet (Layer 3)
Quality of service management, data prioritization
Tier 2 – RF bridge between TGB’s and non-PoP substations
WiMax/Ethernet transport
VLAN partitioning per application for separation

6. SONET Design Relay Protection Communications
Parallel dual SONET ring configuration (OC3 & OC48) with PoP’s at each substation
No relay channel failure with loss of a ring
Path diversity – loop topology with fast switching to protected path
OC3 ring – Primary Relay communications
No other applications will use OC3
Static environment
OC48 ring – Backup Relay, TDM & Ethernet service
GigE provisioned to support Ethernet services
TDM circuits for relay protection, voice & serial communications as required

7. Substation SONET Design

8. Substation Communications Architecture
Substation LAN
Access switch – VLAN provisioned (Layer 2)
No inter-application routing will be permitted
Telemetry network access/authentication will be through core SCADA Firewall – NERC CIP compliant
TGB’s and other substation IP devices will be connected to switch partitioned in their respective VLAN’s
Substation WAN
Router (layer 3) will interface with switch and will provision Virtual Route Forwarding Tunnels (VRF)
6 VRF tunnels will be created for logical separation
VRF tunnels will be encrypted using Dynamic Multipoint VPN (DMVPN)
IP addressing schema will be defined for entire substation population based on application requirements

10. Substation LAN – WAN Architecture
Network Core
Work Station
VRF Tunnels
Substation
Telemetry
CIP Telemetry
Field DA
Enterprise
Security
AMI
VoIP
VLAN extended to switch per Application
SCADA
Gigabit Ethernet
Camera CardReader
Switch
Firewall Router
Switch
Enterprise
Core Router
Firewall
RTU
Security
DA TGB
Incorporates Layer 3 VRF Tunneling and Dynamic Multipoint VPN
AMI/RNI
AMI TGB
Ethernet based devices

11. Middle Mile (Tier 2) Architecture Requirements
Transport the Smart Grid application portfolio
AMI backhaul – 70 Aggregation pts)
Distribution Automation Aggregation pts)
Substation Telemetry (56kbps/substation)
Voice/Video (~1Mbps per video stream)
Application Traffic Considerations
Bandwidth consumption (5-20Mbps)
Latency sensitivity (QoS tagging)
Security (PKI)
Logical provisioning of applications (VLAN tagging)

12. Middle Mile Design Considerations
Spectrum Options – Licensed vs. Unlicensed
Unlicensed (ISM)
Free
Limited Power
Uncontrolled Noise Floor
Uncontrolled Interference
900 MHz – Hi Noise
2.4 GHz – Hi Noise
3.65 GHz (Lite License)
5.X GHz
Licensed
High cost – Spectrum Market
Strict FCC deployment rules
High Power Operation
Low/controlled Noise Floor
Interference Remedy
700 MHz Public Safety
900 MHz
2.3 GHz

13. What Spectrum to Use? What characteristics need to be considered?
Free-space path loss is proportional to the square of the distance between the transmitter and receiver, and proportional to the square of the frequency of the radio signal.
Higher the frequency – Lower the signal propagation
What type of coverage are you planning for?
Blanket umbrella (target lower frequencies with minimal interference characteristics)
Surgical microcell (target higher frequencies)
Urban (target lower frequencies with minimal interference characteristics)
Suburban/rural (depends on the type of coverage, Blanket vs. Surgical)

14. Spectrum Evaluation Frequencies Requirements Overall Ranking 2 6 5 1 3
700Mhz
900Mhz
2.3Ghz
3.65GHZ
5.8Ghz
6-11Ghz
Risk
High
Medium
Low
Cost
Coverage
Excellent
Adequate
Good
Equipment Availability
Limited
Growing
Licensed √
No 
Unlicensed
Lightly
No  
Availability – PECO area
Point-to-Point
Point-to-Multi Point
Overall Ranking 2 6 5 1 3 4
Ranking: 1 high - 6 low

15. Tier II Integration (Conceptual) Design

16. Two prevailing backhaul Standards
Technology Standards
Two prevailing backhaul Standards
WiMax
Long Distance 5-10 Miles
P2P & P2MP
Time Slotted Access
Mobility ( e)
5-10 MHz Channels
802.1Q (VLAN, QoS)
Security EAP/TLS/AES
WiFi
Shorter range 1-2 Miles
Meshing capability
Contention Access DSSS
22 MHz Channels
802.1Q (VLAN, QoS)
Security EAP/TLS/WPA 2

17. Bringing it together Backhaul 100 AMI/DA collector sites
Provide backhaul of non-fiber substations
Requires surgical deployment, not blanket coverage
Point-to-point links
Microcell canopies
Decided WiMax using 3.65 GHz Lite-License
Minimal noise at this time
Do not require full territory coverage
Point-to-point links where necessary

18. Tier II Conceptual Design