1 © Copyright 2012 EMC Corporation. All rights reserved. Building the Right Infrastructure for Microsoft SharePoint.

2 © Copyright 2012 EMC Corporation. All rights reserved. EMC and Microsoft Alliance EMC and Microsoft have been engaged as strategic partners for over 16 years Shared view of importance of information Joint sharing of intellectual property EMC participates in Microsoft Technology Adopter Programs (TAPs) and Rapid Deployment Programs (RDP) CEO-sponsored worldwide go-to-market alliance Over 300 published technical solutions Over 100 data sheets on Microsoft technologies EMC has Hundreds of Microsoft Certified Professionals (MCPs) Microsoft is one of EMC’s Top 5 customers 16+ Microsoft Gold/Silver Competencies 21x Time Partner of Year 2,700+ consultants worldwide Cooperative Support Agreement Equipment in Redmond labs Leading Microsoft consultancies

4 © Copyright 2012 EMC Corporation. All rights reserved. Optimized for Microsoft Complete Integration Across the Microsoft Stack Business Productivity Application Platform Core Infrastructure On-Premise / Cloud ESI/SCOM/SCVMM Packs SMB3.0 Support Thin Provisioning ODX Copy Offload FAST Cache for 9x increase in TPS VFCache for 87% reduction in RT Combined for 11x increase in TPS Manage Storage from SharePoint UI Shrink SQL ContentDB by 90-95% via RBS 30-40% reduced search RT using FAST Cache

5 © Copyright 2012 EMC Corporation. All rights reserved. Challenge: Relentless Growth of Unstructured Data People / Business Processes: 1.Disconnected information silos 2.Information not available to those who need it in real-time 3.Governance IT Related: 1.TCO / Asset Utilization 2.Backup Windows / Recovery Processes 3.Version Control, user mapping, permissions Source : IDC, The Digital Universe Decade

6 © Copyright 2012 EMC Corporation. All rights reserved. Why SharePoint? Why Now?  Why SharePoint? –Fastest growing platform in MSFT’s history –$2B in 2011, 20,000 seats/DAY! –Accessible, extensible, unified content platform  Why now? –67% of customers report rolling out SharePoint to the entire organization –New release (Wave 15) coming in November Gartner Magic Quadrant for Enterprise Content Management

7 © Copyright 2012 EMC Corporation. All rights reserved. Infrastructure Challenges with Current Sharepoint Deployments 1.Unstructured Data Takeover –Primary document types stored in SharePoint are PDF, Word, PowerPoint, and large Excel spreadsheets –Documents typically well over 1MB –Files take over valuable server disk space and resources 2.Avalanche of large media –SharePoint often stores large files like media images and videos –SQL not optimized for file level access in general and streaming in specific –Applications will not return control until file is fully loaded –File size and concurrency will increase the chance of browser timeouts

8 © Copyright 2012 EMC Corporation. All rights reserved. Infrastructure Challenges with Current Sharepoint Deployments 3.Old and unused files hogging valuable SQL storage / CPU –No means to intelligently tier off / archive older stale content –60-80% of SQL BLOB corpus rarely used or not used at all –Equates to over 50% of the SQL Content DB Corpus if not externalized! 4.Lack of scalability –Proliferation of server resources to address performance bottlenecks –Poor economic scaling of SQL server licenses –Native backup / recovery tools too slow for sizable documents ▪Only 600GB backed up in six hours using native tools ▪Only several TB’s backed up in six hours using MSFT DPM –Lack of consistency in recovery of farm, ContentDB, and BLOB Store –Lack of tools to load-balance server resources

9 © Copyright 2012 EMC Corporation. All rights reserved. Building a SharePoint Architecture for Scale Legacy ECM File Shares Future content growth Today Centralize & Take ControlConsolidate & Optimize 5% 95% BLOBs Metadata

11 © Copyright 2012 EMC Corporation. All rights reserved. The Challenge with SharePoint BLOBs PPT PDF XLS DOC TIFF 90-95% unstructured, rest metadata “Typically, as much as 80 percent of data for an enterprise-scale deployment of SharePoint Foundation consists of file-based data streams that are stored as BLOB data. These BLOB objects comprise data associated with SharePoint files. However, maintaining large quantities of BLOB data in a SQL Server database is a suboptimal use of SQL Server resources. You can achieve equal benefit at lower cost with equivalent efficiency by using an external data store to contain BLOB data.” Source: http://msdn.microsoft.com/en-us/library/bb802976.aspxhttp://msdn.microsoft.com/en-us/library/bb802976.aspx Performance for large files, especially write intensive Large file upload time can be improved by as much as 50% with RBS Can’t easily & economically scale Asset Utilization SQL Servers BLOBs don’t benefit from SQL query engine. Why are they there? Long backups / long restores Lack of tiered storage = Poor TCO

13 © Copyright 2012 EMC Corporation. All rights reserved. Metalogix StoragePoint Data Flow SQL Server RBSProviderRBSProvider SP Web Application Service Service Document BLOB write handed off to RBS Provider Invoice # Date Amount Vendor Metadata Document uploaded Metadata uploaded Metadata saved to SQL Server Is there a storage profile? BLOB written to SQL Server Determine Endpoint Synchronous Asynchronous BLOB Migrator Agent YES NO Share Object Store / Cloud

14 © Copyright 2012 EMC Corporation. All rights reserved. StoragePoint – Using Aging Rules VNX/NL-SAS VNX/NL-SAS Compressed Isilon Active Inactive Archive Invoice Date > 1 year Invoice Date > 3 years Older Content to Lower $/GB Tiers On-Premise Private Cloud Public Cloud

16 © Copyright 2012 EMC Corporation. All rights reserved. EMC / Metalogix Reference Architecture FC Network CIFS Network Share NL SAS Share SAS Share Deep Compressed SQL Search DB Reference Architectures: http://www.emc.com/collateral/hardware/technical- documentation/h8185-sharepoint-vnx-metalogix-psg.pdf http://powerlink.emc.com/km/live1/en_US/Offering_Technic al/Presentation/remote-blob-storage-sharepoint-tp.pptx Flash 1 st  75% Cost Reduction via VNX FAST and smart use of SSDs and NL-SAS FAST Cache  Up to 30- 40% reduction in response times for Search FAST VP  80% less performance tuning for SQL Lower SQL Licensing Costs due to server virtualization and RBS tempDB Search Query

17 © Copyright 2012 EMC Corporation. All rights reserved. Applying StoragePoint Advanced Filters FC Network CIFS Network Share FLR Share NL SAS LUN FAST Cache LUN FAST VP Set retention periods at file system creation Default/minimum/maximum Automatic File Lock and Delete VNX File Level Retention:

18 © Copyright 2012 EMC Corporation. All rights reserved. Business Continuity Virtualizing SharePoint is Essential Consolidation Performance Availability Maintenance Load Balancing Achieve 2-10x consolidation ratio, especially for larger deployments Lower SQL Licensing by consolidating vCPU onto physical CPU Improved front end performance with more, smaller WFEs rather than few large WFEs VM based protection for SharePoint provides homogeneous high availability (WSFC, VMware HA) Simplified DR management (Geo-Clustering, vCenter Site Recovery Manager) Live migration of virtual machines (Hyper-V Live Migration, VMware vMotion) Maximized overall performance with balanced cross farm HW utilization (SCVMM PRO, VMware DRS)

19 © Copyright 2012 EMC Corporation. All rights reserved. Three Sources of Savings 1.Better Asset Utilization –File Share typically not well utilized –VNX Utilization ~70% on SAS / NL-SAS and up to 90% on SSD 2.More Efficiency via Flash 1 st –Results in fewer drives, lower energy bills, and lower $/GB –Also lower $/IOPS via FAST Suite 3.Lower SQL licensing via RBS and virtualization –Reduce database size and increase VM density per physical server

20 © Copyright 2012 EMC Corporation. All rights reserved. FLASH 1 st Data Strategy Hot data on FAST Flash SSDs—cold data on dense disks “Hot” high activity As data ages, activity falls, triggering automatic movement to high-capacity disk drives for lowest cost Highly active data is stored on Flash SSDs for fastest response time Movement Trigger Flash SSD “Cold” Low Activity Data Activity High- capacity HDD Data Age

22 © Copyright 2012 EMC Corporation. All rights reserved. VNX5300 20 200GB SSDs 16 600GB 15K SAS 24 3TB NL-SAS VNX5500 32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life. FLASH 1 st Example FAST Cache & FAST VP

23 © Copyright 2012 EMC Corporation. All rights reserved. 32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life. FLASH 1 st Example Footprint PowerIOPS 33K 3,938W45 RU VNX5300 20 200GB SSDs 16 600GB 15K SAS 24 3TB NL-SAS FAST Cache & FAST VP 18 RU 990W 74K

24 © Copyright 2012 EMC Corporation. All rights reserved. VNX FAST Suite A FLASH 1st strategy enabler  FAST Cache continuously ensures that the hottest data is served from high-performance Flash SSDs  FAST VP* optimizes storage pools automatically, ensuring that active data is being served from SSDs, while cold data is moved to lower-cost disk tiers  Together they deliver a fully automated FLASH 1st storage strategy for optimal performance at the lowest cost attainable  Monitor and tune the whole system with the complementary Unisphere Quality of Service Manager and Unisphere Analyzer * Not available for VNX5100 Real-time caching with FAST Cache Scheduled optimization with FAST VP Flash Flash SSD 15K HDD High- capacity HDD

25 © Copyright 2012 EMC Corporation. All rights reserved. VNX FAST Cache boosts SQL2012 DataDiskRAIDFC OLTP DB Files 600K SAS5ON TempDBs600K SAS1/0OFF DB Logs600K SAS1/0OFF FAST Cache: 14 x 100GB SSDs Working Set < FAST Cache Results: 5x Improvement of IOPS and TPS 30 Minute Cache Warming IOPS TPS

26 © Copyright 2012 EMC Corporation. All rights reserved. Lower $/GB and $ / IOPS with RBS Average $/GB SAS–15K $2/GB LOWEST $/IO SSD $.99/IO MetalogixStoragePointMetalogixStoragePoint Example: Avg. Capacity VNX5300 = 50TB Without RBS on SAS-15K: $100K With RBS and 5% SSD and 95% NL-SAS: $40-50K LOWEST $/GB NL-SAS $.43/GB No RBS With RBS

27 © Copyright 2012 EMC Corporation. All rights reserved. Lower SQL Licensing Costs  SQL Server is rarely CPU constrained –For SharePoint the performance bottleneck tends to be the WFE –Idle CPU = Wasted licensing $$  SQL2012 Server Enterprise Licensing: –Physical: per physical core –Virtual: per vCPU (all you can eat per physical core) –No cost difference between a physical core and vCPU  Virtualization = greater VM density and lower SQL licensing costs –Savings determined by vCPU : Core Ratio (e.g. 4:1 ratio =75% savings)  RBS = greater VM density and lower SQL licensing costs –Lower CPU load on SQL servers  Have to weigh against performance / CPU utilization –SQL ContentDB recommended CPU limit 50% –Perfmon helps determine the status quo

28 © Copyright 2012 EMC Corporation. All rights reserved. RBS + VNX = More efficient SQL IO  Overall more efficient handling of IO –Reduction in overall IO by 20-30% –IO coalescing on VNX  Most IO handled by VNX CIFS share –70-80% of backend IO handled by CIFS share –RBS reduces SQL CPU cycles by only storing the varbinary file attribute in SQL –Actual BLOB is stored in more efficient file system Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technic al/Presentation/remote-blob-storage-sharepoint-tp.pptx

29 © Copyright 2012 EMC Corporation. All rights reserved. Results Average page load time improved by 4x after RBS Backup times from SQL and SharePoint reduced by a factor of 13x >90% moved from expensive SQL drives to NL-SAS Challenge Internally facing communications site, which included large amounts of rapidly changing large objects Page load times and backup / recovery times Large Government Contractor EMC VNX and Metalogix StoragePoint Improves Operational Agility and Performance “The dashboard reports showed that I was now saving >90% of my storage on the expensive SQL drives.” Derek Martin Cloud Solutions Architect

30 © Copyright 2012 EMC Corporation. All rights reserved. SharePoint Performance Considerations  Performance for SharePoint highly use case dependent –Focus on Enterprise Content Management*  SharePoint workload for content management –Can be characterized by 60/20/10/10 for Browse/Modify/Search/Upload –File size, distribution, and concurrent access impact performance  Symptoms vs. Remedy –Symptoms often occur during individual tasks (e.g. file upload by user) –Sizing however needs to factor in system level workloads for the entire farm –In addition need to size system for blended workload (e.g. Exchange, SQL, SharePoint)

31 © Copyright 2012 EMC Corporation. All rights reserved. SharePoint Performance Considerations  The storage subsystem is rarely the bottleneck –Generally web front end (WFE) CPU represents the bottleneck –VNX FAST Suite enables very efficient configurations (majority NL-SAS, 2-5% SSDs) –Search functions are the workloads that happen to benefit the most ▪Search critical as SharePoint environments and document repositories grow ▪Microsoft Test Report for Very Large-Scale Document Repositories validates VNX performanceMicrosoft Test Report for Very Large-Scale Document Repositories –VNX typically deployed with mixed workloads (e.g. Exchange, SQL, and SharePoint)  Case for RBS: SQL Database not efficient at handling files IO –Each write represents an update of the table and the log –Non-externalized storage formats use the SQL buffer pool when the data pages are accessed –Large files and high concurrency can lead to buffer overruns compounding the issue –Other processes such as index rebuilds can also compete for the same server resources

32 © Copyright 2012 EMC Corporation. All rights reserved. Best Use of VNX FAST Suite for SharePoint SharePoint Component FAST VPFAST CacheWorkload Search Index -++ R:W Ratio of 90:10; 32K block read, 64K write; Small working set. Search Index re-uses the same blocks on disk as working space to process list items during indexing. Storage IO quiet between crawls. Highly changing, throw-away data. Search Query ++ R:W Ratio of 70:30; 32K read, 64K write; Small working set. Query improves due to random large block reads/writes being serviced by FAST Cache, but Query storage is large and costly in FAST Cache. Highly-read data with small burst write changes. tempdb ++ R:W Ratio of 50:50; 8K read, 32K write. TempDB (database page reuse) is ideally suited to FAST Cache algorithms. Same blocks are re-used on disk and performance of TEMPDB directly affects SharePoint performance request. Content DB ++ R:W Ratio of 95:5; 16K (read and write); Working set reduced via RBS. Low IOPs requirements does not require FAST Cache. DB Index operations see an improvement. BLOB Store -- Low IOPs requirements, large-block I/O.

33 © Copyright 2012 EMC Corporation. All rights reserved. Recovery Options / Tradeoffs  What are the recovery objectives? –Recovery Level: Database, Database + BLOBs, Database + BLOBs + Farm –Recovery Time: how long does it take to recover –Recovery Point: How much can I afford to lose vs. how much am I willing to spend –Distance: Local vs. Remote –Granularity: Item level vs. entire corpus –Management Overhead: how much time do I need to spend on backup / recovery  Challenges with native tools: –Limited in scale, scope, and performance (e.g. STSADM, Central Administration, Recycle Bin) –Configuration data not backed up –Low backup / recovery rate: rated at 600GB in 6 hours (TB’s with DPM) –Manual re-creation of configuration –STSADM / Central Administration not recommended above 85GB for site collections or Content DB larger than 100GB

34 © Copyright 2012 EMC Corporation. All rights reserved. Availability & Protection Methods / Tradeoffs Requirements (cumulative) Backup / RecoveryReplicationHigh Availability Database (incl. BLOBS)SQL native backup STSADM / Central Admin. Avamar SQL Availability GroupsSQL Failover Clustering DB+ Remote BLOB StoreVNX Snaps / Replication Manager SQL Failover Clustering (DB Only) Item-Level RecoveryRecycle Bin Avamar / Ontrack (not RBS aware) VNX Snaps / Replication Manager / Selective Restore Manager or Ontrack SQL Failover Clustering (DB Only) Farm Configuration / VM / OS VNX Snaps / Replication Manager Hypervisor virtual machine clustering Site-to-SiteRecoverPoint / Cluster Enabler / SRM

35 © Copyright 2012 EMC Corporation. All rights reserved. Replication Manager / Item-Level Restore  RM supports recovery of Content DB and Remote BLOB Store via script  Order of restore: –Single ContentDB BLOB Store –StoragePoint DB –Singe ContentDB  For item-level restore: –Point Selective Restore Manager or Ontrack PowerControls to mounted RM database files –Browse to content and restore the required file(s) or Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technic al/Presentation/remote-blob-storage-sharepoint-tp.pptx or

36 © Copyright 2012 EMC Corporation. All rights reserved. Replication Manager Protection Data protectedStart timeEnd timeDuration SharePoint farm and search7:45:267:49:043 minutes 38 seconds StoragePoint7:57:087:57:091 second BLOB stores7:57:217:58:1251 seconds Total time to protect entire farm content:4 minutes 30 seconds Data restoredStart timeEnd timeDuration BLOB store8:48:028:48:1816 seconds StoragePoint8:52:458:54:071 minute 22 seconds Content Database08:59:479:03:223 minutes 35 seconds Total time to restore a Content Database:5 minutes 13 seconds Full farm protection 250 GB Single Content Database Restore

37 © Copyright 2012 EMC Corporation. All rights reserved. SharePoint 2010 Disaster Recovery: Virtualized Farm (Full Replication) SharePoint Farm A Secondary Site Primary Site BLOB Store VMs RecoverPoint VMs Resource/Protection Group level granularity Active/Active (Sync distances) or Active/Passive (Async distances)  Failover automation:  VMware Site Recovery Manager (SRM) Protection Groups for all server roles  EMC Cluster Enabler with Hyper-V failover clustering Databases BLOB Store VMs Databases WFE Application Servers Crawl SQL Cluster

38 © Copyright 2012 EMC Corporation. All rights reserved. Automated DR Consistency Group (RP)VNX VolumesGroupingSRMCE Web Front EndsBoot + Query (optional) Protection GroupCluster Group Query ServersBoot + Query Component Protection GroupCluster Group Index ServersBoot + Index Component Protection GroupCluster Group Application ServersBoot + Application Volumes Protection GroupCluster Group SQL Server(s) Boot + Pagefile (optional) Protection GroupCluster Group SQL System Databases Configuration Databases Search Databases Content Databases RBS BLOB Store Designing DR consistency for SharePoint Search consistencyExternalized Content consistency

39 © Copyright 2012 EMC Corporation. All rights reserved. Provisioning SharePoint - ESI Open / browse local or remote SharePoint farm Provision SharePoint Web Application: –Create disks –Create content database –Create Web Application –Attach the content database Create content database only (no WFE) –Web Application can be created later using SharePoint admin and the database created in ESI Users can select any SQL server that is part of the farm and any instance in that server. leave unchecked

40 © Copyright 2012 EMC Corporation. All rights reserved. Health Monitoring in the Private Cloud EMC System Center Operations Manager (SCOM) Management Packs: Array discovery & health monitoring Full hierarchical storage schema and relationship view Dashboards incl. physical view (SPs, DAEs, NICs, HBAs, Drives) and logical view (RAID Groups, Volumes/Pools, LUNs, Protocols…) Performance Monitoring End to end application service view Map / visualize relationships between storage and SharePoint farm EMC System Center Orchestrator (SCO) Management Packs: Focal point for automation of heterogeneous datacenter management Service Manager for workflows and pre-defined run books 21 storage activities and counting Example Drive failure: 1.Gather SP Collects 2.Post on pre-defined share 3.Send alerts to admin

43 © Copyright 2012 EMC Corporation. All rights reserved. RBS Options & Feature Comparison FeatureRBS FILESTREAMStoragePointSourceOneAvePoint Catalogue Files into native SharePoint ribbonX√XX BLOB Externalization (RBS)√√√*√ Offload BLOBs to remote storage platforms X √√√ BLOB store must be local to SQL Server Outbound BLOB cache for off-premises (i.e. Cloud) operations X√XX Define multiple storage endpoints per content databaseX√√√ Filter endpoints by file type or scope (i.e. list or content type)X√XX Take endpoints offline (and/or send notifications) if they fill up or enter an error state X√√X Use SharePoint filename or folder on offloaded BLOBsX√XX Promote SharePoint properties to endpointX√XX Securely delete (shred) BLOBsX√XX Report and Health Monitor DashboardX√√X Administration User InterfaceX √ √√ Central Admin PowerShell SupportX√√ CompressionX√XX EncryptionX √ XX Transmission and "at rest" Tiered Storage (HSM) supportX√XX Orphaned BLOB Garbage Collection √√√√√ BasicPolicy-basedBasic

1 © Copyright 2012 EMC Corporation. All rights reserved. Building the Right Infrastructure for Microsoft SharePoint.

Similar presentations

Presentation on theme: "1 © Copyright 2012 EMC Corporation. All rights reserved. Building the Right Infrastructure for Microsoft SharePoint."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

1 © Copyright 2012 EMC Corporation. All rights reserved. Building the Right Infrastructure for Microsoft SharePoint.

Similar presentations

Presentation on theme: "1 © Copyright 2012 EMC Corporation. All rights reserved. Building the Right Infrastructure for Microsoft SharePoint."— Presentation transcript:

Similar presentations

About project

Feedback