2. UNC301 - (Microsoft IT) Microsoft Exchange 2007 Architecture and Design at Microsoft Martin Coetzer Technical Consultant Microsoft

3. Agenda Microsoft IT Exchange environment overview Transport and topology Server roles and hardware planning Mobile messaging scenarios – Client Access Server Internet mail connectivity – Edge Transport Server Mailbox server and storage designs Backup approach (current and future)

4. Microsoft IT Exchange Environment Overview Multiple Exchange organizations (Corp, Dogfood, Windeploy, WinSE, Extranet, Managed Solutions) Prior to transition to Exchange 2007 –~100 legacy Exchange 2003 servers / 30 mailbox servers After transition to Exchange 2007 –~64 Exchange 2007 mailbox clusters running in the Corp forest at RTM –Goal to reduce it to ~35 in the nearest future Mailboxes – 130,000+ Public Folders – 240,000 folders Distribution Groups – 170,000+ Maximum message size = 10 MB Availability Goal – 99.99% Mailbox size on Exchange 2003: 200 MB Mailbox size on Exchange 2007: 500 MB and 2 GB

5. Post-Consolidation Environment 4 Sites with Exchange mailbox servers 100 Exchange servers - 30 mailbox server (clustered) - 4000x200MB per cluster node 99.99% availability goal 4 Sites with Exchange mailbox servers 100 Exchange servers - 30 mailbox server (clustered) - 4000x200MB per cluster node 99.99% availability goal

6. Microsoft IT Messaging Trends and Needs Category Exchange 2000 timeframe Exchange 2003 timeframe Exchange 2007 timeframe Mailboxes total70,00090,000130,000 Typical mailbox quota100 MB200 MB500MB / 2GB Mobile users per month OWA EAS Outlook RPC/HTTP Limited N/A 50,000 13,000 25,000 60,000 30,000 60,000 Internet message submissions per day (prior to filtering) 6,000,0009,000,00013,500,000 Spam messages filtered per day2,500,0007,500,00011,000,000 Maximum message size5 MB10 MB10+ MB

7. Microsoft IT Exchange Server 2003 Deployment Limitations of 32-bit platform Scalability limitations –Virtual memory is limited to 4GB Exchange Server 2003 can use up to 3GB max –Kernel Memory limitations Paged Pool 256MB max with /3GB enabled Client connections, Access tokens Non Paged Pool (256MB max without /3GB and 128MB max with /3GB) HTTPs and TCP connections to the Exchange Front End or Gateway Storage Performance demands –Small Database cache size = higher disk I/O and storage demands

8. Goals of Exchange Server 2007 Deployment at Microsoft Overcome scalability limitations of 32-bit platform Maintain 99.99% availability goal Increase the user mailbox size from 200MB to 500MB and 2GB Reduce hardware, storage, and backup costs Establish mailbox server resilience from storage level failures Establish flexible and scalable middle-tier services (e.g. Client Access, Transport)

9. Routing Group Connectors RGC RGC RGC Legacy Exchange Server 2003 Environment

10. Dublin Singapore Sao Paulo Redmond Redmond- Exchange AD Site Link Exchange Server 2007 Environment AD Site with Exchange Servers AD Site without Exchange Servers

11. Dublin Singapore Sao Paulo Redmond Redmond- Exchange AD Site Link Exchange Server 2007 Environment Exchange Routing AD Site with Exchange Servers AD Site without Exchange Servers

12. Dublin Singapore Sao Paulo Redmond- Exchange AD Site Link Custom Site Link ExchangeCost=10 ADCost=999 Redmond Exchange Server 2007 Environment set-adsitelink Dublin-to-RedmondExchange -ExchangeCost 10 set-adsitelink SaoPaulo-to-RedmondExchange -ExchangeCost 10 set-adsitelink Singapore-to-RedmondExchange -ExchangeCost 10

13. Microsoft IT Exchange Server 2007 Server Roles In Exchange Server 2003 timeframe Microsoft IT deployed servers roles dedicated for specific functions Exchange Server 2007 deployment – similar approach –Structured system administration and maintenance –Optimized hardware and software configuration –Targeted load balancing and fault tolerance –Flexible scaling approach –Independent and flexible –server provisioning –Higher Availability

14. Exchange 2007 Load Balancing and Fault Tolerance at Microsoft IT Enterprise scenario – must build load balancing and fault tolerance provisions –Resilience to server level failures –Tolerance to increased load due to planned server downtime and environmental conditions (spam attacks, “snow day”) ComponentLoad BalancingFault Tolerance Mailbox ServerN/AClustering (CCR) NIC Teaming Hub Transport ServerMBX  HUB: Built in Edge  HUB: Send Connector – multiple bridgeheads Edge Transport ServerInternet  Edge: MX equal cost & DNS Round Robin HUB  Edge: Edge Subscriptions Client Access ServerExternal: Network Load Balancing + ISA 2006 WPLB Internal: Network Load Balancing Unified MessagingUM  IP Gateway: Multiple IP Gateways per dial plan IP Gateway  UM: Built in (Round Robin between UM)

15. Hardware Configurations RoleConfiguration BHD/Edge 1 4x2.2 GHz CPU*, 8GB RAM CAS 2 4x2.2 GHz CPU*, 4GB RAM UM 2x2.2 GHz CPU*, 4GB RAM MBX 2000 @ 500MB (CCR) 4x2.6 GHz CPU*, 12GB RAM MBX 2400 @ 2GB (CCR) 4x3.0 GHz CPU*, 16GB RAM MBX 3600 @ 2GB (CCR) 8x2.6 GHz CPU*, 24GB RAM MBX 6000 @ 500MB (CCR) 8x2.66 GHz CPU, 16GB RAM 8x2.66 GHz CPU **, 16GB RAM * * 2 way = One Dual Core CPUs; 4 way = Two Dual Core CPUs ; 8 way = Four Dual Core CPUs ** 8 way = Two Quad Core CPUs 1 1 7:1 Mailbox-to-HUB server performance ratio 2 2 4:1 Mailbox-to-CAS server performance ratio SPECint ® 2000 rates were used to compare different processors ( www.spec.org ) www.spec.org

16. Common URL namespace for mobile messaging clients –OWA, Exchange Active Sync (EAS), Outlook Anywhere (RPC/HTTP) (e.g. https://mail.company.com) https://mail.company.com Integration – all mobile messaging services on a common Exchange Server 2007 CAS platform Mobile Messaging Service Usage –OWA: ~60,000 unique users/month –RPC/HTTP: ~60,000 unique users/month –EAS: ~30,000 unique users/month Highly scalable Exchange Server 2007 CAS infrastructure –Load balancing and fault tolerance Overview

17. Messaging Topology Dublin: -ExternalURL “https://emeamsg.microsoft.com/...” Singapore: -ExternalURL “https://apsmsg.microsoft.com/...” Sao Paulo: -ExternalURL “https://spamsg.microsoft.com/...” Redmond: -ExternalURL “https://msg.microsoft.com/...” -Autodiscover Service URL “https://autodiscover.microsoft.com/autodiscover”

18. Design Published Paths /exchange/* /owa/* /public/* /Microsoft-Server- ActiveSync/* /RPC/* /Autodiscover/* /EWS/* /UnifiedMessaging/* Cookie-based load balancing IP-based load balancing The Exchange front-end servers are part of a Web farm pointed to by an ISA Server 2006 array

19. Microsoft IT Exchange 2007 Edge Server

20. Deployment Goals: Mailbox size: 2GB; Deleted items retention time: 14 days Daily Transaction Log generation rate per mailbox per day: 25MB (average) Database overhead (fluff) factor: 20% Desired max DB size - 200GB: –200 GB / (2GB + 20%) = 85 mailboxes/DB One Database per Storage Group Databases per LUN: 7 (to facilitate weekly backup schedule) –One Backup Stream Per source LUN DB LUN Capacity requirements for 85mbx * 7DBs= 600 mailboxes/LUN –600 mailboxes * 2GB + 20%[DB overhead] + 5%[Ci] + 10%[Free space] = 1,656GB –1,656GB / 275GB * 2 [RAID1+0] = 12 disks (@300GB) DB LUN Performance requirements –Assuming 1.0 IOPS/mbx at 2GB for very heavy user profile with 5MB of RAM/user –600 mailboxes * 1 IOPS/mbx = 600 IOPS / LUN –1:1 Read:Write mix (300 Reads + 300 Writes = 600) 300[R] + 2x300[W] = 900 IOPS 900 / 130 IOPS = 7 disks (10K RPM) Conclusion: Design is capacity bound - 12 disks required for capacity [RAID1+0] Hardware Design: Mailbox Storage Provisioning

21. Hardware Design: Universal Storage Building Block – 2GB Goal: Design simple and scalable storage building block Goal: Design simple and scalable storage building block Approach: Two disk enclosures Cross-shelf mirroring [RAID 1+0] LUN Capacity: 600 mailboxes = 12 disks [RAID 1+0] 7 Storage Groups/Databases Single Logical Drive Two Database LUNs total 1,200 x 2GB mailboxes One Log LUN 2 logical volumes (L:\ + M:\) 825GB of log space 30 x 300GB drives 24 for databases 6 for transaction logs Data:Log – 4:1 Goal: Design simple and scalable storage building block Goal: Design simple and scalable storage building block

22. Hardware Design: Universal Storage Building Block – 500MB LUN Capacity: 830 mailboxes = 14 disks [RAID 1+0] 7 Storage Groups/Databases Single Logical Drive Three Database LUNs total 2,500 x 500MB mailboxes One Log LUN 550GB of log space 50 x 146GB SFF SAS drives 42 for databases 8 for transaction logs

23. Hardware Design: 2,400 x 2GB mailboxes (SAS DAS)

24. Hardware Design: 3,600 x 2GB mailboxes (FC SAN)

25. Streaming Backup Backup Window: 4 hours Schedule: Full – weekly, Incremental - daily

26. System Center Data Protection Manager 2007 VSS based backup from “Active” or “Passive” CCR aware requestor Express Full backup technology (deltas) Incremental backup - every 15 minutes www.microsoft.com/dpm

27. Q&A

28. Thank you http://www.microsoft.com/southafrica/ucs/2007