1. Large-Scale SQL Server Deployments for DBAs
Unisys

2. Agenda Optimization I/O Configuration
Failover Cluster
Resources
Indexes
I/O Configuration
Data Management
Enterprise Class Systems Infrastructure
Intel Platform Architecture
Storage Architecture
Microsoft Systems Architecture

3. SQL Server Failover Clustering
Hot standby solution
Best high-availability configuration
Redundant system
Shared access to the database files
Recovery in seconds
Automatic failure detection
Automatic failover
Minimal client application awareness
Built on the server cluster technology of Windows Clustering

4. Windows Clustering Hardware Components Virtual Server
Cluster node
Heartbeat
External network
Shared cluster disk array
Quorum drive
Virtual Server
Software Components
Cluster name
Cluster IP address
Cluster Administrator account
Cluster resource
Cluster group
Microsoft Distributed Transaction Coordinator (MS DTC)

5. How Failover Clustering Works
Operating-system checks
Heartbeat checks availability of nodes and virtual servers.
SQL Server checks
LooksAlive check runs every five seconds.
IsAlive check runs SELECT query.
Failover to another node
Windows Clustering attempts restart on same node or fails over to another node.
SQL Server service starts.
Brings master online.
Database recovery proceeds.
End users and applications must reconnect.

6. Enhancements to Failover Clustering
SQL Server Setup installs/uninstalls a cluster.
Service packs can be applied directly to virtual servers.
SQL Server supports multiple instances and multiple network addresses.
Failover and failback occur to or from any node in a cluster.
SQL Server 2000 on Windows 2000 Datacenter Server supports four server nodes in a cluster.
All nodes have local copies of SQL Server tools and executables.
Rerunning Setup updates Failover Cluster configurations.
SQL Server Service Manager or SQL Server Enterprise Manager now start/stop SQL Server Services.

7. SQL Server 2000 Failover Clustering
SQL Server 2000 Virtual Server
MSCS
Shared Disk Array
Node A
Node B
Heartbeat
Public Network
Virtual Server Instance
System & User Databases
SQL Server & SQL Server Agent Services
Executables
Network Connection

8. Instance Example

9. Four-Node Clusters
SQL Server 2000 fully supports 4-node failover clustering
Can assume max 1 node failure and assign resources appropriately (e.g. leave one node out of four idle  25% redundancy (improves on Windows 2000 Advanced Server 2-node 50% redundancy)
When running Active/Active/Active/Active for critical workloads ensure that one node has the power/resources to handle the worst case (i.e. memory)

10. SQL Server 2000 Cluster Storage Considerations
Fibre Channel connected to SAN is the preferred approach
Network Attached Storage (NAS)
Not supported for clusters
(Q)
“Because of the risks of network errors compromising database integrity, together with possible performance implications that may result from the use of network file shares to store databases, Microsoft recommends that you store database files either on local disk subsystems or on Storage Area Networks (SANs).”

11. SQL Server 2000 Cluster Storage Considerations
Basic Disks are required by MSCS (Microsoft Cluster Service)
Dynamic Disk & Mount Points are not natively supported (Veritas required)
Software RAID is not supported
Balance letters of the alphabet vs. placement of:
Log files
Data files
Filegroups
tempdb

12. Cluster Fail-over Memory & CPU Configuration Planning
If you poured contents from one glass into another…
Would all contents fit into one glass?
Memory Resources
CPU Resources
64-Bit dynamically manages memory

13. Resources Memory Processor Allocation Very Large Memory Support AWE
AWE & I/O
Settings
Processor Allocation
User Mode Scheduler
Affinity

14. Windows 2000 Very Large Memory Support
Real Memory - 4GB Limit & /3GB switch
Physical Addressing Extension (PAE)
Address Windowing Extensions (AWE)
Large Memory Enabled (LME)
AWE I/O Implications
Datacenter typically has very large memory
Real Memory
32bit address space
2GB or 3GB for OS
PAE
Intel Hardware Feature
Address any given 4GB
AWE
Software Feature
Transparent memory mapping
Real memory used as a window
LME
Hardware support for 64bit address space
I/O implications (similar to real vs. protected)
Much simpler in 64 bit

15. Windows Product Options
3GB
PAE/AWE
Limit
Windows Server No
N/A
4GB
Windows Advanced Server
Yes
8 GB
Windows Datacenter Server
16 GB
32 (64) GB

16. Memory Management 101 Virtual Memory Physical Memory 4G 4GB
Windows 2000 Kernel
2GB or 3GB
4GB
Under 4GB memory = direct page mapping

17. Process Address Spaces
Virtual Memory
Physical Memory
Windows 2000 Kernel
2GB or 3GB
4GB
SQL Instance1
SQL Instance2
SQL Instance3

18. Virtual Memory And AWE
Windows 2000, a 32-bit Operating System, can only address 232 bytes (4GB)… So how do we support more memory?
The next few slides look at how AWE enables SQL Server to use 64GB

19. AWE Memory Model AWE Window resides in real memory (below 4GB)
Data pages mapped to reserved bufferpool above 4GB
Applications call AWE API to get data pages
Data pages referenced or mapped
Allows huge amounts of data to be cached
Look for Page Life Expectancy counter in perfmon
AWE provides a 64GB address space to SQL Server
Largest benefits with systems that have large amounts of frequently referenced pages (“hot” pages)
Note: SQL Server-based applications that one-time scan through large tables and/or have large intermediate results sets may not benefit from much larger bufferpool

20. Mapped Virtual Memory Allocation 1. Physical Memory is allocated
AWE Memory Allocation
Virtual Memory
Physical Memory
Windows 2000 Kernel
2GB or 3GB
AWE Memory Window
SQL Instance1
Mapped Virtual Memory Allocation
(Above 4GB)
1. Physical Memory is allocated

21. AWE Memory Reallocation
Virtual Memory
Physical Memory
Windows 2000 Kernel
2GB or 3GB
AWE Memory Window
SQL Instance1
1) Mapping Removed
2) New Mapping
Mapped Virtual Memory Allocation
(Above 4GB)

22. SQL Server Use of AWE Memory
Using AWE means that SQL Server memory is no longer dynamic (i.e. it can no longer be relinquished if the system is under stress)
Caution when using AWE and SQL Server with other workloads – configure memory appropriately or SQL Server will allocate total available memory (minus 128MB)
Caution when running in a cluster - allow enough memory for failover on each node

23. AWE and I/O Applications can only read/write to AWE Memory Window
Two methods of achieving this:
With LME hardware (hardware can access mapped pages anywhere in memory)
Without LME hardware (hardware can only access mapped pages from <4GB)

24. AWE I/O With LME Required for Datacenter Certification
Virtual Memory
Physical Memory
Windows 2000 Kernel
2GB or 3GB
AWE Memory Window
Network
Disk
SQL Instance1
Mapped Virtual Memory Allocation
(Above 4GB)
Direct I/O

25. Mapped Virtual Memory Allocation
AWE I/O Without LME
Physical Memory
Virtual Memory
Windows 2000 Kernel
2GB or 3GB
AWE Memory Window
SQL Instance1
Network
Disk
Mapped Virtual Memory Allocation
(Above 4GB)
Double-buffered I/O

26. Large Memory Settings To grab 4GB or less: To grab >4gb to 16gb
No /PAE needed in boot.ini
/3GB available to use to grab 3 out of 4 GB of real memory.
AWE enabled but doesn’t do anything for SQL Server
To grab >4gb to 16gb
/PAE enabled in boot.ini
/3GB enabled in boot.ini
AWE enabled
To grab > 16gb
/PAE enabled
/3GB disabled

27. Memory Configuration Settings
Min Server Memory (MB)
SQL Server will maintain this value once it’s committed
Not automatically allocated on startup, unless AWE is enabled
Max Server Memory (MB)
Becomes SQL Server target memory unless (Available Bytes – 5 MB) less than Max Server Memory.
Buffer pages will continue to be committed until target reached.
Under memory pressure SQL Server will pay out buffer pool until Min Server Memory reached.
Set to same as min when AWE enabled
AWE Enabled
Enable AWE window for SQL Server

28. Memory Configuration Settings (cont.)
Locks
Sets maximum number of locks
When set to 0, 2% of memory allocated to pool of locks initially.
Dynamic lock pool do not exceed 40% of total server memory allocation
Set Working Set Size
Reserve physical memory space for SQL Server instead of being swapped out, set min and max to same value when enable this option.
Min memory per query (KB)
Sets minimum memory that must be met by that Memory Grant Manager to avoid query waiting
query wait (s)
Wait time for Memory Grant Manager to acquire min memory per query before query times out.

29. Resources Memory Processor Allocation Very Large Memory Support AWE
AWE & I/O
Settings
Processor Allocation
User Mode Scheduler
Affinity

30. User Mode Scheduler CPU 0 CPU 1 CPU 2 CPU n Network Network UMS
Workers in “Runnable” status assigned to CPU by UMS Scheduler
UMS
Scheduler
UMS
Scheduler
UMS
Scheduler
UMS
Scheduler
Query Results
Network
Worker Pool
Workers
Worker Pool
Workers
Worker Pool
Workers
Worker Pool
Workers
UMS
Work
Queue
UMS
Work
Queue
UMS
Work
Queue
Work requests assigned to workers in pool
UMS
Work
Queue
Network
Work Requests

31. Context Switching No real multiprocessing, just time slicing
SQL Server User Mode Scheduler
Per processor
Scheduler assigns processor
Performance Factors
Perfmon: Context switches/sec (10-15K bad)
CPUs have L2 cache
CPU Affinity mask
Lightweight pooling

32. Related SQL Server Configuration Settings
Max worker threads
Sets maximum workers to service work requests
Assigned evenly across CPUs available to UMS
Lightweight pooling
When set, workers are created as NT fibers
One thread per CPU manages subset of fibers
Priority boost
If set, workers execute at NT priority 14
By default, workers execute at priority 7-8
Max degree of parallelism
Number of processors considered for parallel query execution plan
Cost threshold for Parallelism
Minimum execution cost before optimizer creates parallel execution plan

33. SQL Server Affinity Masks
CPU Affinity
Reduce context switching
Increase cache hits
New SQL Server 2000 SP1 Features!
I/O Affinity
Connection Affinity (VIA only)

34. SQL Server CPU Affinity

35. IO Affinity Storage Area Network
Crossbar
Intra-connect
TLC
CPU
I/O
MSU
Storage Area Network
Example: Set IO Affinity to Processor x F
Sends SQL Server disk I/O to affinitized processors.
( An extreme high end feature )

36. Index Overview Clustered and Non-Clustered Indexes Covering Indexes
Unique Indexes
Indexed Views
Index Maintenance
Recommendations

37. Clustered Indexes
Root Page
Non-leaf pages . . .
--- Leaf
Pages
A-C D-E F-G H-J K-M N-O P-Q R-S T-V W-Z
Example: - “select * from table where lastname like ‘S%’
- Assume 100 names start with ‘S’
- Assume 100+ rows per page
- Clustered index will issue 4 in this example
- contrast with nonclustered noncovering heap table

38. Clustered Indexes Choosing a clustered key
Clusterkey is row locator in all non-clustered indexes
Increase potential for covering queries
Size consideration – long composite clusterkeys may not be appropriate
Small clusterkeys (smallint or INT) save space in non-clustered index:
INT = 4 byte row locator stored in non-clustered index entry
RID = 6 byte row locator stored in non-clustered index entry
Update frequency
Clusterkey columns must be stable
Updates are expensive
Cost increases with number of non-clustered indexes
Continuously ascending clusterkey value is most efficient model for high volume insert:
CREATE table orders
(Order_ID INT IDENTITY PRIMARY KEY CLUSTERED,
No hot spot contention
No page splitting
Suitable as a default clustered index

39. Non-clustered Indexes
Root Page
ANDERSON 4:300
JOHNSON 4:301
SIMONSON 4:302
Branch
Page 4:300
Page 4:301
Page 4:302
ANDERSON 2:100
BENSON 2:101
JOHNSEN 2:102
JOHNSON 2:103
KRANSTEN 2:104
MILLER 2:105
SIMONSON 2:106
TANNER 2:107
Leaf
Page 2:100
Page 2:101
Page 2:102
Page 2:103
Page 2:104
Page 2:105
Page 3:106
Page 2:107
ANDERSON
BENSON
JOHNSEN
JOHNSON
KRANSTEN
MILLER
SIMONSON
TANNER
Forwarding pointer to new row location; avoids index update when RID changes
Heap (Data Pages)
UPDATE CUSTOMER
SET CUSTOMER_COMMENT = ‘ ’
WHERE CUSTOMER NAME = ‘KRANSTEN‘
→..
....KRANSTEN...

40. Example: Non-clustering index carrying clustering key:
Note: NC affected by all Clustering
Key operations including Reindex,
Drop/Create, etc.
Root Page
Non-Leaf
Pages
Leaf Pages
Clustering
Key
……..

41. Example: Drop Clustering Key, change NC key values to RIDs.
Note: Change clustering key and
You change all non-clustering
Indexes automatically
Root Page
Non-Leaf
Pages
Leaf Pages
RID
Data
Pages

42. Non-clustered Indexes Composite Index Considerations
Leading column density
CREATE UNIQUE INDEX IX_PRODUCT_TYPE_BRAND_NAME
ON PRODUCT
(PRODUCT_TYPE,
PRODUCT_BRAND,
PRODUCT_NAME)
Important that statistics exist on all columns of index
sp_createstats ‘indexonly’
Using left-most subset of columns not absolutely necessary on non- clustered indexes:
Low density of interior column (and existing stats) can use index scan WHERE argument to filter prior to bookmark lookup:
SELECT * FROM PRODUCT WHERE PRODUCT_BRAND = 'XYZ'
StmtText
|--Bookmark Lookup(BOOKMARK:([Bmk1000]), OBJECT:([orders].[dbo].[PRODUCT]))
|--Index Scan(OBJECT:(IX_PRODUCT_TYPE_BRAND_NAME), WHERE:(PRODUCT_BRAND='XYZ‘ ))
If leading column used alone in search argument, density is crucial
If columns used together (as search argument) leading column density less important than composite All Density

43. Covering Indexes Materializes query result from a non-clustered index
All necessary data must be in the index
Data pages are not touched (no bookmark lookup)
References to clustered key columns are available and used from the non-clustered index
A composite index is useful even if the first column is not referenced
SELECT SUPPLIER_ID, ORDER_ID, COUNT(*)
FROM ORDER_ITEM
WHERE YEAR(SHIP_DATE) = 1993
GROUP BY SUPPLIER_ID, ORDER_ID
ORDER BY SUPPLIER_ID, ORDER_ID
StmtText
|--Compute Scalar(DEFINE:([Expr1002]=Convert([Expr1006])))
|--Stream Aggregate(GROUP BY:(SUPPLIER_ID, ORDER_ID) DEFINE:([Expr1006]=Count(*)))
|--Sort(ORDER BY:(SUPPLIER_ID ASC, ORDER_ID ASC))
|--Index Scan(OBJECT:(IX_ORDER_ITEM_SHIP_DATE), WHERE:(datepart(year, Convert(SHIP_DATE))=1993))

44. Covering Indexes Considerations
Queries covered by indexes often found during trivial plan phase
Reduced compilation time
Multiple indexes can be used
Index rows retrieved from non-clustered indexes
Joined on row locator (clusterkey or RID)
Not a common access method
Clustered keys always available (exist in row locator)
Can reduce some lock contention
Data page / row are not accessed; not locked
Avoid exaggerating non-clustered key lengths to force covering queries
Longer key increases index size
Volatile columns cause lock contention in index leaf

45. Break

46. Data Management Overview
I/O strategies
Capacity Planning Guidelines
RAID
File System
Database and Object Placement Strategies
SAN
Monitoring

47. Understanding your Disks
Rotational SpeedAvg
Laten cy (ms)
Avg Seek (ms)
Avg Access Time (ms)
Rando m I/O’s per sec (8K)
Sequential I/O’s per sec(64K)
3600
8.3 13
21.3 45 85
5400
5.5 11
16.5 60
115
7200
4.2
7.1
11.3 90
175
10000 3
5.4
8.4
120
230
Check the numbers
There is a difference in speed for new drives
Numbers from HW vendors are perfect world

48. Workload Comparison OLTP environment DSS, OLAP environment
Short transactions
Random read and write I/O
50-60% read, 40-50% write a good start point of workload breakdown
DSS, OLAP environment
Mostly long transactions
Many sequential I/O, typically reads
75-80% read, 20-25% write is a good start point

49. Capacity Planning Approach
Minimally
Size of database / size of disk
Table sizing tool in W2K resource kit and
Not a good approach for performance
Guesstimate
Estimated data throughput / size of disk
Compare with the minimal approach and take the larger # (of disks) of the two.
Sampled approach
Take a small sample workload and monitor on a system with tempdb, log and datafile placed on separate spindles. Using perfmon to capture the I/O statistics and calculate the spindle requirements under real production load.

50. Capacity Planning OLTP Planning
Random read/writes
Plan on more random reads and writes or 80 I/Os per second
Turn on controller cache to match read / write ratio. Example for read intensive applications maybe 75% read 25% write
Larger number of smaller drives = better performance
Put log on its own controller with 100% write on controller (except replication environment)

51. Capacity Planning Data Warehouse
Sequential Read
Plan on 120 I/Os/second per drive due to scans
Set controller for heavy read through cache
In general, spread data and index across many drives, don’t be a hero and guess where to place objects
Consider striping Tempdb separately
Log less important

52. RAID Introduction Stripe Mirror Stripe Mirror Stripe Stripe RAID 0
Data 1
Data 5
Data 9
Data 13
Data 2
Data 6
Data 10
Data 14
Data 3
Data 7
Data 11
Data 15
Data 4
Data 8
Data 12
Data 16
Mirror
RAID 1
Data 1
Data 2
Data 3
Data 4
Data 1
Data 2
Data 3
Data 4
Stripe
RAID 5
Data 1
Data 4
Data 7
Parity
Data 2
Data 5
Parity
Data 10
Data 3
Parity
Data 8
Data 11
Parity
Data 6
Data 9
Data 12
Mirror
Stripe
Stripe
RAID 10
Data 1
Data 3
Data 5
Data 7
Data 2
Data 4
Data 6
Data 8
Data 1
Data 3
Data 5
Data 7
Data 2
Data 4
Data 6
Data 8

53. Which RAID to Choose Using Sequential read/write operations
RAID read/1 write (No redundancy)
RAID 1/RAID split read/2 writes
RAID reads/2 writes
MB/sec
RAID 0
RAID 1
RAID 10
RAID 5

54. RAID 5 vs. RAID 10 Performance
Iometer version , Copyright  by Intel Corporation

55. RAID Recommendations RAID 10 is best RAID 5 as last resort
Very high read-write performance
Higher theoretical fault tolerance
RAID 5 as last resort
Highest usable capacity
Moderate read performance
Poor write performance
Stripe Configuration
64K Stripe Unit
256K Stripe Depth
Data 1
Data 5
Data 9
Data 13
Data 2
Data 6
Data 10
Data 14
Data 3
Data 7
Data 11
Data 15
Data 4
Data 8
Data 12
Data 16
Stripe
Unit
Stripe
Width

56. Raid Implementation in VLDB
Hardware or Software Raid
Performance consideration
Cluster support
Battery backup for controller cache a must
Some controller performance are optimized with number of disks.
IDE Raid
Good sequential performance
Lack of scalability

57. File System Topics NTFS vs. FAT32 Allocation unit size
NTFS required for cluster support
FAT32 has file size limit
Allocation unit size
Stripe size must be multiple of 8K (SQL Server I/O), 4K block (NT I/O) and 512Bytes (Disk I/O)
64K size is optimum for most SQL Server implementation
Larger stripe size (128K, 256K) may be used for data warehouse environment

58. I/O load Balancing System Bus PCI Bus PCI Bus PCI Bus Network
Understand theoretical maximums versus actual, sustainable throughput capability in all bus architectures
PCI
Bus
Network
Factor in network I/O requirements
Distribute spindles in RAID set across available Channels
Do not saturate one PCI bus and underutilize another – balance your work load. Monitor!
Understand your bus architecture – know the point of diminishing returns on spindles per bus

59. IO Utilization Object Placement
Where to place my objects?
Log should be on its own mirrored drive or drives
OS bits, SQL bits and pagefile on separate drive
Use multiple filegroups
Large and heavy hit table and its indices should be placed in separate filegroup for both performance and management reasons
Place table (data), index, and tempdb on separate filegroup and spindle to promote sequential I/O during table/index scan, e.g. index creation (using sort_in_tempdb option)
Multiple datafiles can be used for tempdb if usage is high.
Database Backups should go to separate spindles due to the highly sequential nature of backup and restore operation. Performance improvement can be significant vs. shared spindles in a non-SAN environments, e.g. SCSI sub-systems with limited number of spindles.

60. Storage Area Network SAN Advantage SAN Watch Outs
Large cache and large number of spindles often simplify the storage architecture design
High availability, reliability and often rich feature set for VLDB environment
SAN Watch Outs
Storage design is still necessary, it’s not the solution for everything
Performance information may not be easily accessible
Investment is skilled SAN Administration is a requirement

61. I/O Monitoring Performance monitor counters Vendor Tools
Processor & SQL Server Process: %Privileged Time vs. % Processor Time
PhysicalDisk: % Disk Time, Avg Disk Queue Length, Current Disk Queue Length, Avg. Disk Sec/Read, Avg. Disk Sec/Write
Avoid logging to disks being monitored
Vendor Tools
SAN environment often do not provide the accurate statistics to the OS. Use vendor monitoring tools to obtain the comparable information.
SQL Server Datafile I/O statistics
Transaction log bottleneck - Query virtual table ::fn_virtualfilestats
Select * from ::fn_virtualfilestats(<database id>, -1)
if IOStallMS / (NumberReads + NumberWrites) >= 20ms, then a log I/O bottleneck may exist.

62. Enterprise Class Systems Infrastructure

63. Enterprise Class Systems Infrastructure
Enterprise Wintel Architectures
Enterprise Storage Architectures
Microsoft Systems Architecture

64. Basic Computing Architecture
Cache
Best OK

65. Intel’s Server Solution

66. Profusion Architecture
Current Intel Xeon Processor MP speeds
2GHz, 1.90GHz, 1.60GHz, 1.50GHz, 1.40GHz
Current supported Profusion products
700 MHz, 900 MHz
Stops at 8 processors

67. A High End Architecture

68. ES7000 Scaling Up – all 32 CPUs MSU MSU MSU MSU MSU MSU MSU MSU
Crossbar
Intra-connect
Crossbar
Intra-connect
Crossbar
Intra-connect
Crossbar
Intra-connect
FLC
FLC
FLC
FLC
FLC
FLC
FLC
FLC
The ES7000 represents the worlds first Intel Mainframe rivaling the largest proprietary Unix systems and even traditional mainframes. In fact this architecture is the basis of our next generation mainframe line as a future version will accommodate our proprietary CMOS processors and our mainframe operating systems as well as the Intel 32bit and 64bit CPUs and Windows OS. So the system architecture was built by mainframe engineers for mainframe features and functionality. When we started the design and in fact when we shipped the first systems this year there wasn’t a Windows operating system available to take advantage of this level of scale up. In September 2000 Microsoft released Datacenter edition, the first truly advanced scale-up Windows operating system.
The hardware supports up to 32 processors, 64GB of memory, 96 PCI cards. All of these open components are connected via a crossbar intraconnect and a combination of 5 custom ASICs we developed to “glue “ the open components into a mainframe architecture.
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
I/O
I/O
I/O
I/O
I/O
I/O
I/O
I/O

69. 1 2 3 4 5 6 7 8 ES7000 Partitioning MSU MSU MSU MSU MSU MSU MSU MSU
Crossbar
Intra-connect
Crossbar
Intra-connect
Crossbar
Intra-connect
Crossbar
Intra-connect
TLC
TLC
TLC
TLC
TLC
TLC
TLC
TLC 1 2 3 4 5 6 7 8
Leveraging the ES7000 as a singular 32X SMP system provides a new and powerful capability in the Windows space, but it could be limiting with today’s lack of scalable applications above a 4X or 8X environment. This is where the “scale out” capabilities of the ES7000 really shine. We have designed the architecture with powerful partitioning capabilities. This partitioning can be hard or soft in nature. Hard partitioning prevents other partitions from affecting each other regardless of what happens in any other partitions. The CPU, I/O, Memory are all locked into place and the system partition operates independently of the other partitions. These partitions can also run different operating systems!
We also support soft partitioning where the applications would run under a single instance of the OS but we could assign and reassign processor resources to them anytime dynamically or set up a process to watch the requirements of any process so more resources could be assigned if needed automatically.
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
I/O
I/O
I/O
I/O
I/O
I/O
I/O
I/O

70. 1 2 3 4 ES7000 – Common Usage Model MSU MSU MSU MSU MSU MSU MSU MSU
Crossbar
Intra-connect
Crossbar
Intra-connect
Crossbar
Intra-connect
Crossbar
Intra-connect
TLC
TLC
TLC
TLC
TLC
TLC
TLC
TLC 1 2 3 4
This slide discusses the idea of a 16x SQl server (multi-instance) and another 8-way MC app, and then a testing & staging area.
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
CPU
I/O
I/O
I/O
I/O
I/O
I/O
I/O
I/O

71. High Performance Crossbar
Unisys ES7000 – 32 bit
High Performance Crossbar
Intel
Proc
4th level Cache
12 PCI 66mhz slots
Centralized Memory
12 PCI 66 66mhz slots
Shared Cache
Shared Cache
Shared Cache
Shared Cache
Single-domain 16-way
Dual-domain 32-way

72. Unisys ES7000 - 32 bit Highest performer of Xeon MP based ES7000s
Best availability and manageability features w/ Server Sentinel
For large databases and mission-critical applications
Can be configured from 8 to 32 Xeon processors MP with up to 8 partitions
More than twice the I/O bandwidth of the Orion 200 model
- Up to 64 x 66 MHz and 32 x 33 MHz PCI slots

73. Unisys ES7000 – 32 bit Up to 8x cellular sub-pods Up to 4 crossbars
- 4-way Intel Xeon processors MP
(1.4 GHz or 1.6 GHz, 64-bit bus at 400 MHz FSB)
- 32MB module of shared cache
Up to 4 crossbars
- High-performance, non-blocking
Up to 64GB of centralized memory
- Contains cache coherency mechanism

74. Unisys ES7000 – 64 bit

75. Enterprise Wintel Architecture
Windows Datacenter Program
High end systems
32 Processor
64GB Memory
Higher level of support
Joint support queue
Unisys Server Sentinel
System health monitoring & Self-healing
Automated operations & Problem reporting
Hardware and software inventory
Remote management
Unisys Application Sentinel (SQL Server)
Self-Optimization for SQL Server environments
Analyzes metrics and compares to known good baseline
Chooses setting/resource changes to relieve bottlenecks
Self-Healing for SQL Server environments
Predicts halt/hang condition and triggers graceful failover
Configurable automation level

76. Pilgrim’s Pride $2.2B poultry producer located in Pittsburg, Texas,
3rd largest in US. Its 2 TB database is the largest SAP user
in the world running SQL Server
Business Problem:
Required additional reliable, scalable capacity for a database server that was running out of headroom on a commodity 8-way server. Current number of users is 2600 and is expected to double by 2004 through acquisition and regular business expansion.
Solution:
Two 16-way ES7000s running Windows 2000 Datacenter Server and SQL Server 2000, each with 32GB of memory. Each system is configured with a 12-way DB server and a 4-way application server clustered to failover to the other.

77. Enterprise Class Systems Infrastructure
Enterprise Wintel Architectures
Enterprise Storage Architectures
Microsoft Systems Architecture

78. High-End Storage Arrays
Architecture—Design Alternatives
Yesterday
Bus 1.6 GB/s
Advantages:
Good performance
Simple, low cost
Challenges:
Bus contention
Limited scaling
Switch = 15.9 GB/s
Advantages:
Good performance
Challenges:
Switch contention
Limited scaling
Complex, costly

79. High-End Storage Arrays
Architecture—Design Alternatives
Yesterday
Today
Bus 1.6 GB/s
Advantages:
Good performance
Simple, low cost
Challenges:
Bus contention
Limited scaling
Direct Matrix = 70.4 GB/s
Switch = 15.9 GB/s
Advantages:
Good performance
Challenges:
Switch contention
Limited scaling
Complex, costly
Breakthrough design:
Simple, cost-effective
More reliable
More scalable
Future-proof growth

80. Symmetrix Direct Matrix
Architecture— Interconnect and Cache Architecture
Matrix Interconnect
Pathing
128 direct point- to-point dedicated connections
Matrix backplane
Dynamic Global Cache
Parallelism
32 independent regions
32 concurrent IOs
More
Performance
More IOPs and bandwidth
More burst performance
More business continuity performance
More
Functionality
Full software compatibility
Assured network interoperability
More
Availability
Simpler design
Eliminates bus limitations
Eliminates switch limitations
Better
Economics
Scale high / scale low
More efficient use of hardware
Precisely target configurations

81. Symmetrix Direct Matrix
Symmetrix DMX Direct Matrix Interconnect
Up to 128 direct paths from Directors and cache
500 MB/s per direct path
Up to 64 GB/s maximum rated data bandwidth
Up to 6.4 GB/s message bandwidth
Dynamic Global Cache
Up to 32 concurrent IOs through cache
500 MB/s per cache region
Up to 16 GB/s maximum cache bandwidth
Architected for up to 512 GB Global Memory
500 MHz PowerPC Powerplant
Up to 116 processors
Up to 106,000 Dhrystone MIPS
2 Gb Fibre Channel drive infrastructure
Up to 64 drive loops
Up to 12.8 GB back-end I/O bandwidth
Architected to 2,048 disk drives
Up to 12 high-density connectivity directors
8-port, 2 Gb Fibre Channel Directors
16 to 96 direct connections
Over 8,000 network-connected hosts
Architectural Specifications

82. Synchronous Mirroring
Most Durable Form of Remote Mirroring
Confirmation Returned as Part of I/O Completion
Synchronous vs. Asynchronous, and Adaptive Mode

83. EMC SRDF Architecture uses delta technology (Track Tables)
Fully independent of host operating systems, DBMS, filesystems
Data is mirrored remotely
Bi-directional source-to-target(s) architecture
Mirrored volumes logically synchronized
Added protection against drive, link, and server failures
SRDF is an online, host-independent, mirrored data storage solution that duplicates production site data (source) to a secondary site (target). If the production site becomes inoperable, SRDF enables rapid manual failover to the secondary site, allowing critical data to be available to the business operation in minutes. The mirroring of data can be between two sites with each performing production and secondary site roles (referred to as bi-directional mirroring).
The secondary site (target) can be located at distances that extend up to thousands of miles from the production site (source) using T3/E3, ATM, or IP networks as the intersite link to facilitate SRDF mirror data operations between Symmetrix systems. For sites with distances of a few kilometers up to 200 km apart, the use of ESCON or Fibre Channel protocols facilitates mirrored data operations between sites.
SRDF links
Source
Target

84. SRDF Modes of Operation Synchronous Mode4 3 1
SRDF links
SRDF Synchronous Mode is used primarily in SRDF campus environments. In this mode of operation, Symmetrix maintains a real-time mirror image of the data of the remotely mirrored volumes.
Data on the source (R1) volumes and the target (R2) volumes are always fully synchronized at the completion of an I/O sequence through a first-in, first-out queue (FIFO) model. All data movement is at the block level with synchronized mirroring.
The sequence of operations is:
An I/O write is received from the host/server into the cache of the source.
The I/O is transmitted to the cache of the target.
A receipt acknowledgment is provided by the target back to the cache of the source.
An ending status is presented to the host/server.
Note: The SRDF mode can be changed dynamically depending on such factors as production activity level, time of day, bandwidth concerns, temporary re-structuring of environment, etc. It is very flexible.
Pros: Instant restart, highest in data integrity.
Cons: Distance is limited by latency, more expensive communication/link costs. 2
Source
Target 1
I/O write received from host / server into cache of source 2
I/O is transmitted to the cache of the target 3
Receipt acknowledgment is provided by target back to cache of source 4
Ending status is presented to host / server

85. SRDF and Clustering GeoSpan for MSCS
Provides cluster support across a geography
Private Interconnect
Heartbeat Connector
SRDF is increasingly being used in cluster environments. In a cluster, the server and application providers have usually worked out the issues of failing over from one server to another at the application level. This means that they can restart the work on the surviving node in the cluster. However, if the failed node contained the disk storage and the working data being used by the application, the problem would be that the surviving node could restart the applications but only with new data.
SRDF’s value in this scenario is in the mirrored data from one Symmetrix to another within the cluster. Using SRDF, the surviving node also has all the data because it was previously in an SRDF relationship. Procedures to make the R2 volumes read/write enabled have to be taken. In some cases, this task is automated.
When the time comes to bring the failed node back online, SRDF provides the ability to fail back the other direction and be ready for work very quickly with up-to-the-second live data.
SRDF
Primary Location
“Failover” Location

86. Old Mutual Life Assurance Company
Old Mutual, a Fortune Global 500 company, is a world- class international financial services company that manages more than $234 billion in funds for millions of customers worldwide. Old Mutual has been aggressively expanding its operations in life assurance, asset management, banking and short-term general insurance.
Business Challenge:
Promote business continuity by streamlining an overgrown server environment and providing total disaster recovery and backup to prevent downtime and service interruptions.
Solution:
Unisys ES7000 servers, Microsoft Windows 2000 Advanced Server and SQL Server 2000, and EMC Symmetrix, SRDF, GeoSpan, and PowerPath products. Outage time went from 54 hours per month to near zero.

87. Using TimeFinder for Backups
Quiesce SQL Server
Ensure All Updates Completed to Production Disk Volume
Split Off a Point-In- Time BCV Mirror of Logical Volume
Resume Production Application Processing While …
Performing Backups from the BCV Mirror
TAPE
SYSTEM
Production
BCV for
Backup

88. Storage Management
Functionality—Storage Management Made Simple, Automated, Open
Intelligent Supervision: ControlCenter StorageScope, Database Tuner, Automated Resource Manager, SAN Manager, Common Array Manager; EMCLink
Information Safety:
EMC Data Manager (EDM), ControlCenter Replication Manager, ISV backup applications
For years, EMC has delivered storage management software that makes multi-vendor environments simple, automated, and open. Whether it’s Intelligent Supervision products, Information Safety applications, or Infrastructure Services capabilities, all EMC software is available day one for the Symmetrix DMX.
Combine EMC storage software functionality with DMX performance and availability, and now you can drive higher service levels across more of your organization than ever before.
But that only makes sense if the economics work, so let’s take a look at what we’ve done on this important topic.
Infrastructure Services: PowerPath, Celerra HighRoad, GeoSpan
Device Management: ControlCenter Symmetrix Manager, SRDF/ TimeFinder Manager, Symmetrix Data Mobility Manager (SDMM)
Performance Management: ControlCenter Symmetrix Optimizer, Workload Analyzer
Legend: Multivendor storage management
EMC platform storage management
Replication and Recovery: TimeFinder, SRDF, Consistency Groups

89. Enterprise Class Systems Infrastructure
Enterprise Wintel Architectures
Enterprise Storage Architectures
Microsoft Systems Architecture

90. The Unisys Microsoft Systems Architecture is a best-of-breed, Windows-based infrastructure producing faster implementations with predictable costs, reduced risk and faster time to benefit.

91. Fully redundant data centers supporting “Unisys-unique” capabilities
Unisys MSA EDC Program
Fully redundant data centers supporting “Unisys-unique” capabilities
Disaster recovery
Enhanced systems management
Secure application tier
Large OLTP data bases
Multi-terabyte business intelligence hosting
Server consolidation
Site A - EMC Lab
Site B - Unisys Lab
This is the Unisys EDC Prescriptive Architecture. Let me highlight the truly unique attributes. First and foremost, what you see here is two Enterprise Data Centers. Previously, you saw the picture of our center of excellence in Redmond. Site B is being built there. For those of you who haven’t visited Redmond, let me mention that we have a unique circumstance in that the EMC center of excellence is one floor below us! So we are building Site A in their lab.
These two sites are connected using Nortel Optera switches and utilize EMC GeoSpan. So we will be documenting and qualifying a complete disaster recovery environment for up to 200 kilometers.
We will also be deploying Server Sentinel for enhance systems management.
We have added a secure application tier to the Microsoft design.
Much as we did in the Microsoft design, we are testing and documenting OLTP and business intelligence solutions.
In the area of consolidation, we have taken different approaches in each site. Site A reflects a high degree of consolidation, but none the less, still has some commodity servers. The number is 24…a significant reduction from the 70+ servers in the Microsoft approach. These are being provided by Dell. They are working with us on the effort.
Site B employs slot appliances. As a result, we have completely eliminated commodity servers.

92. A Summary of Today’s Session
SQL Server 2000
SQL Server Failover Clustering
Resource management
Memory
CPUs
Indexing
I/O and Strategies
Enterprise Class System Infrastructure
Enterprise Class Servers
Unisys ES7000
Enterprise Class Storage
EMC Symmetrix
Unisys Services and MSA
Infrastructure engineered for your most demanding requirements

93. Thank You!