Download presentation
Presentation is loading. Please wait.
1
SQL Server 2012 内存管理 苏有全 Escalation Engineer
Microsoft SQL Server Support
2
Agenda 介绍SQL Server 2012 内存的基本概念 介绍SQL Server 2012 内存管理以及和以前版本的改变
[Session Code]
3
SQL Server 2012 内存的基本概念 [Session Code]
4
Virtual Address space (VAS)
232 bytes = 4 GB 264 bytes = 16 EB VAS Limit up to 64GB Kernel 8TB Kernel 1GB Kernel 2Gb /PAE AWE User 3GB 4TB Physical Memory Limits (win 2012) User 2Gb-8TB User 2Gb /USERVA /3GB
5
/3GB, PAE and AWE Address Windowing Extension (AWE) API
Mapping/Unmapping a “View” or “Window” of physical memory pages into Virtual Address Space (2/3 GB) in 32 bit machine Data and Index Pages only Lock pages in memory uses AWE API to allocate memory in 64 bit. No need to turn on AWE option. Sp_configure ‘AWE Enabled’ set to 1 (In 32 bit) Must configure max server memory with AWE Removed in SQL2012 AWE Enabled in boot.ini to see the extra memory in 32 bit OS PAE To provide more address space to the application Only 1GB left for the kernel mode. Can starve operating system. /3GB Removal of AWE feature in SQL Server SQL Server 2012 removed the AWE feature used in 32-bit SQL Server. You can find this information in the books online topic "Discontinued Database Engine Functionality in SQL Server 2012". Customers and support engineers will notice the following effects: · The configuration option "awe enabled" is not available as one of the options in the stored procedure sp_configure in SQL Server 2012. · The configuration option "Use AWE to allocate memory" is removed from the memory configuration page of the Server Properties in the SQL Server Management studio. · If you set the configuration option "awe enabled" to 1 in previous versions of SQL Server and attempt to upgrade to SQL Server 2012, the upgrade advisor will provide you with warnings and the upgrade will be blocked. · 32-bit instances of SQL Server 2012 can make use of a maximum of 4 GB of physical memory. If the SQL Server instance needs to use more memory, then this instance needs to be migrated to a 64-bit system and installed as a 64-bit SQL Server. Reasons for removal of AWE feature As you observed in the history section, AWE feature was needed in 32-bit SQL Server to access more memory on x86 systems. The hardware trends changed in the last decade and the following things happened: · The Windows Server operating system is 64-bit only starting with Windows Server 2008 R2 · Newer deployments of SQL Server are primarily happening on 64-bit systems · 64-bit is the current preferred commodity hardware for servers
6
Address Windows Extensions (AWE)
AWE 是 Microsoft Win32® API 的内存管理功能的一组扩展,它允许应用程序寻到多于 4 GB 的内存,而通过标准 32 位寻址可获得的内存为 4 GB。AWE 允许应用程序获取物理内存并作为非换页内存,然后将非换页内存的视图动态映射到 32 位地址空间。虽然 32 位地址空间限制为 4 GB,但是非换页内存却可以远远大于 4 GB。这使大量占用内存的应用程序(如大型数据库系统)可以寻到比 32 位地址空间中支持的更多的内存。 AWE related API: AllocateUserPhysicalPages – Allocates the physical memory VirtualAlloc(MEM_PHYSICAL) – to create window MapUserPhysicalPages/MapUserPhysicalpagesScatter SQL Server AWE feature SQL Server version 2000 made use of the Windows AWE feature. In order to enable the use of AWE in SQL Server you need to perform the following actions: 1. Grant the "lock pages in memory" user right to the SQL Server startup account 2. Use the sp_configure command to enable "awe enabled" option 3. Set the appropriate "max server memory" and "min server memory" values SQL Server was able to use the AWE feature for only the database pages in the buffer pool. All other memory requirements within the SQL Server Database Engine came from the non-AWE portions.
7
AWE Memory Layout Reference IW2K Page 402 – 403 Notes:
Pages not shareable or pageable Must have “Lock Pages In Memory Right” Page protection limited to read/write ---- AllocateUserPhysicalPages – Allocates the physical memory VirtualAlloc(MEM_PHYSICAL) – to create window MapUserPhysicalPages/MapUserPhysicalpagesScatter Limit 16gb issue
8
SQL server 内存的一些术语 Buffer Pool (缓冲池) MemtoLeave (using –g size to set)
data/index pages cache SQL server internal usage (plan/proc cache, compilation etc) MemtoLeave (using –g size to set) [Session Code]
9
SQL server 内存的一些术语 MemToLeave
SQL Server在启动时就保留的连续地址空间,供进程空间内的大于8KB的外部组件用,诸如扩展过程,COM/OLE自动化组件,连接服务器等。 默认值为 256 MB。使用启动参数改变,比如 -g384,-g512 If not enough, you may see below in error log "Failed Virtual Allocate Bytes: FAIL_VIRTUAL_RESERVE <size>" "Failed Virtual Allocate Bytes: FAIL_VIRTUAL_COMMIT <size>" 64bit 不需要 如果内存>4GB, 最好尽快升级到64bit -g startup parameter -g is a startup option in SQL Server. This option exists in SQL Server starting with SQL Server 7.0. SQL Support engineers and customers normally use the -g startup option to change the default value used by SQL Server for a portion of virtual address space termed as "Memory-To-Reserve". This portion is also known as "memory_to_leave or MTL. The "Memory-To-Reserve" and -g configuration option are relevant only for a 32-bit instance of SQL Server. There are changes in SQL Server 2012 that affect which components of SQL Server use this portion. The default value for "Memory-To-Reserve" is 256 MB remains the same even in SQL Server 2012. In previous SQL Server versions until SQL Server 2008 R2, memory allocations made outside of SQL Server Buffer Pool used the "Memory-To-Reserve" portion of virtual address space. These include the following: · Multi-Page allocations from SQL Server [These are allocations which request more than 8KB] · CLR allocations [These include the SQL CLR heaps and its global allocations created during startup] · Memory for thread stacks within SQL Server process · Memory allocation requests made directly to Windows [These include windows heap usage and direct virtual allocations made by modules loaded into SQL Server process. Examples: allocations from extended stored procedure dll's, objects created using OLE Automation procedures (sp_OA calls), allocations from linked server providers.] If the total of these allocations are higher than the default value used for "Memory-To-Reserve", then a customer need to use -g startup parameter to increase this portion of the virtual address space. This is common practice when CLR is enabled in 32-bit instances of SQL Server. In SQL Server 2012, the SQL Server memory manager manages the Multi-page allocations and hence those allocations do not use the "Memory-To-Reserve" portion of the virtual address space. Apart from this change, everything else remains the same in SQL Server 2012 from the perspective of -g startup option. Error log: :05:26.64 Server Microsoft SQL Server (Intel X86) Oct :41:15 Copyright (c) Microsoft Corporation Enterprise Edition on Windows NT 5.2 (Build 3790: Service Pack 2) :05:26.65 Server SQL Server is starting at high priority base (=13). This is an informational message only. No user action is required.=> Do you require raising the priority of SQL Process :05:26.65 Server Detected 8 CPUs. This is an informational message; no user action is required :05:26.72 Server Address Windowing Extensions is enabled. This is an informational message only; no user action is required :05:27.84 spid4s Server name is ‘xxx\xxxx' :24:47.11 spid1s AppDomain 5 (Apple.dbo[runtime].4) is marked for unload due to memory pressure :24:47.11 spid1s AppDomain 5 (Apple.dbo[runtime].4) unloaded :24:47.11 spid1s AppDomain 4 (NetPerfMon.dbo[runtime].3) is marked for unload due to memory pressure :24:47.11 spid1s AppDomain 4 (NetPerfMon.dbo[runtime].3) unloaded :26:53.45 spid53 Error: 18056, Severity: 20, State: :26:53.45 spid53 The client was unable to reuse a session with SPID 53, which had been reset for connection pooling. This error may have been caused by an earlier operation failing. Check the error logs for failed operations immediately before this error message :26:53.57 spid64 Error: 18056, Severity: 20, State: :26:53.57 spid64 The client was unable to reuse a session with SPID 64, which had been reset for connection pooling. This error may have been caused by an earlier operation failing. Check the error logs for failed operations immediately before this error message :30:01.26 Backup BACKUP LOG WITH TRUNCATE_ONLY or WITH NO_LOG is deprecated. The simple recovery model should be used to automatically truncate the transaction log :30:05.42 spid107 Failed Virtual Allocate Bytes: FAIL_VIRTUAL_RESERVE => requesting allocation of 1 MB of contiguous chunk of memory :30:05.52 spid107 Memory Manager VM Reserved = KB VM Committed = KB AWE Allocated = KB Reserved Memory = 1024 KB Reserved Memory In Use = 0 KB :30:05.52 spid107 Memory node Id = 0 VM Reserved = KB VM Committed = KB AWE Allocated = KB SinglePage Allocator = KB MultiPage Allocator = KB :30:05.52 spid :30:05.53 spid107 Downgrading backup log buffers from 1024K to 64K :17:23.94 spid1s AppDomain 6 (Apple.dbo[runtime].5) is marked for unload due to memory pressure :17:23.94 spid1s AppDomain 6 (Apple.dbo[runtime].5) unloaded :18:04.40 spid94 Error: 701, Severity: 17, State: :18:04.40 spid94 There is insufficient system memory to run this query :18:26.63 Server The connection has been lost with Microsoft Distributed Transaction Coordinator (MS DTC). Recovery of any in-doubt distributed transactions involving Microsoft Distributed Transaction Coordinator (MS DTC) will begin once the connection is re-established. This is an informational message only. No user action is required :19:04.39 spid94 Error: 701, Severity: 17, State: :19:04.39 spid94 There is insufficient system memory to run this query :19:35.01 spid79 Error: 701, Severity: 17, State: :19:35.01 spid79 There is insufficient system memory to run this query :54:27.65 Logon Error: 17189, Severity: 16, State: :54:27.65 Logon SQL Server failed with error code 0xc to spawn a thread to process a new login or connection. Check the SQL Server error log and the Windows event logs for information about possible related problems. [CLIENT: ] :54:52.97 Logon Error: 17189, Severity: 16, State: :54:52.97 Logon SQL Server failed with error code 0xc to spawn a thread to process a new login or connection. Check the SQL Server error log and the Windows event logs for information about possible related problems. [CLIENT: ] :54:52.97 Logon Error: 17189, Severity: 16, State: 1. [Session Code]
10
SQL server 内存的一些术语 min server memory 和 max server memory
Resource Governor, Resource Pool NUMA (非一致性内存访问) [Session Code]
11
Symmetric Multi Processing Non Uniform Memory Access
Processor Architectures Memory CPU 0 CPU 1 CPU 2 CPU 3 CPU 4 CPU 5 CPU 6 CPU 7 Symmetric Multi Processing (SMP) Non Uniform Memory Access (NUMA) NUMA NODE 0 Memory CPU 0 CPU 1 CPU 2 CPU 3 NUMA NODE 1 Local Memory Access Foreign Access ‘n’ x local CPU 4 CPU 5 CPU 6 CPU 7 ‘n’ >3 (typically) but varies with hardware design
12
Buffer Pool demo sys.dm_os_sys_info sys.dm_os_buffer_descriptors
bpool_commit_target and bpool_committed (<=SQL2012) committed_kb and committed_target_kb in SQL2012 sys.dm_os_buffer_descriptors WARNING: Be careful using dm_os_buffer_descriptors as it can return 200,000+ rows for just a 1.6GB address space. It doesn’t count all to avoid blocking and contention 谁占用了我的Buffer Pool? cpu_ticks ms_ticks cpu_count hyperthread_ratio physical_memory_kb virtual_memory_kb committed_kb committed_target_kb visible_target_kb stack_size_in_bytes os_quantum os_error_mode os_priority_class max_workers_count scheduler_count scheduler_total_count deadlock_monitor_serial_number sqlserver_start_time_ms_ticks sqlserver_start_time affinity_type affinity_type_desc process_kernel_time_ms process_user_time_ms time_source time_source_desc virtual_machine_type virtual_machine_type_desc :11: AUTO QUERY_PERFORMANCE_COUNTER NONE (1 row(s) affected) 谁占用了我的Buffer Pool? 我在做SQL Server 7.0技术支持的时候有客户问我,“我的SQL Server buffer pool很大,有办法知道是哪些对象吃掉我的buffer Pool内存么?比方说,能否知道是哪个数据库,哪个表,哪个index占用了buffer Pool么?”当时我没有找到这个问题的答案,但是我一直记着这个问题。直到SQL server 2005 版本出现,这个问题迎刃而解。答案就是使用动态视图(DMV) sys.dm_os_buffer_descriptors。 这个DMV非常强大。根据SQL Server 联机丛书,这个视图的作用是 “返回有关 SQL Server 缓冲池中当前所有数据页的信息。可以使用该视图的输出,根据数据库、对象或类型来确定缓冲池内数据库页的分布”。具体点说,这个视图能够返回buffer pool里面一个8K 的data page的下列属性: (1)该页属于哪个数据库 (2)该页属于数据库哪个文件 (3)该页的Page_ID (4)该页的类型。可以根据这个来判断此页时索引页还是数据页 (5)该页内有多少行数据 (6)该页有多少可用空间。 (7)该页从磁盘读取以来是否修改过。 有了上面的信息,我们就可以很方便的统计出几种很有用的数据,如下。 1. Buffer Pool的内存主要是由那个数据库占了? SELECT count(*)*8 as cached_pages_kb,CASE database_id WHEN THEN 'ResourceDb' ELSE db_name(database_id) END AS Database_name FROM sys.dm_os_buffer_descriptors GROUP BY db_name(database_id) ,database_id ORDER BY cached_pages_kb DESC; 结果如下: 从上面的结果可以看到数据库AdventureWorks占用了大概30MB左右的缓冲池空间。 注意该DMV 并不返回Buffer Pool里面有关非数据页(如执行计划的缓存等)的信息。也就是说这个DMV并没有返回Buffer Pool里面所有页面的信息。 2. 再具体一点,当前数据库的哪个表或者索引占用Pool缓冲空间最多? SELECT count(*)*8 AS cached_pages_kb ,obj.name ,obj.index_id,b.type_desc,b.name FROM sys.dm_os_buffer_descriptors AS bd INNER JOIN ( SELECT object_name(object_id) AS name ,index_id ,allocation_unit_id,object_id FROM sys.allocation_units AS au INNER JOIN sys.partitions AS p ON au.container_id = p.hobt_id AND (au.type = 1 OR au.type = 3) UNION ALL ,index_id, allocation_unit_id,object_id ON au.container_id = p.partition_id AND au.type = 2 ) AS obj ON bd.allocation_unit_id = obj.allocation_unit_id LEFT JOIN sys.indexes b on b.object_id = obj.object_id AND b.index_id = obj.index_id WHERE database_id = db_id() GROUP BY obj.name, obj.index_id ,b.name,b.type_desc 输出结果如下 (部分): 从上面的结果可以看到表Individual 在Pool内存里面缓冲最多,可能这个就是经常访问的热表,或者是比较大的表。注意Pool里面的缓冲页是经常变化的。 你如果再跑一次语句,出现在头条的可能是另外一个表了。 3. Buffer Pool缓冲池里面修改过的页总数大小。这个比较容易: SELECT count(*)*8 as cached_pages_kb, convert(varchar(5),convert(decimal(5,2),( *(select count(*) from sys.dm_os_buffer_descriptors b where b.database_id=a.database_id and is_modified=0)/count(*)*100.0)))+'%' modified_percentage ,CASE database_id FROM sys.dm_os_buffer_descriptors a 结果: 从上面的结果可以看到,AdventureWorks数据库大概有13.84%的数据是修改过的。如果一个数据库的大部分(超过80%) 是修改过的,那么这个数据库写操作非常多。反之如果这个比例接近0,那么该数据库的活动几乎是只读的。读写的比例对磁盘的安排是很重要的。当然还有其他性能数据来获得数据库读写的大概比例,这里限于篇幅就不多谈了。 [Session Code]
13
Locked pages in memory What is locked pages? –non pageable pages
Enable Lock Pages in Memory user rights Set “max server memory” KB How to reduce paging of buffer pool memory in the 64-bit version of SQL Server KB ( Enterprise Edition, Data Center Edition Standard Edition, Business Intelligence Edition Other editions SQL 2012 (32-bit and 64-bit) • Assign LPIM user right to SQL Startup account No support for "locked pages" SQL Server 2005/2008/2008R2 (32-bit) • Assign LPIM user right to SQL Startup account • Configure 'awe enabled' option to 1 SQL Server 2005/2008/2008R2 (64-bit) • Assign LPIM user right to SQL Startup account • Apply SQL Hotfix KBA • Enable trace flag 845 How to enable the "locked pages" feature in SQL Server 2012 Use the following methods to determine whether the SQL Server 2012 instance is using "locked pages":The output of the following TSQL query will indicate nonzero values for locked_page_allocations_kb: select osn.node_id, osn.memory_node_id, osn.node_state_desc, omn.locked_page_allocations_kb from sys.dm_os_memory_nodes omn inner join sys.dm_os_nodes osn on (omn.memory_node_id = osn.memory_node_id) where osn.node_state_desc <> 'ONLINE DAC' The current SQL Server error log will report the following message during server startup: Using locked pages in the memory manager The "Memory Manager" section of the DBCC MEMORYSTATUS output will show a nonzero value for the "AWE Allocated" item. How do I know if SQL Server is using "locked pages" for its memory management? Use the information from the following data points to determine this: · SQL Server error log now clearly prints the memory model used: Using locked pages in the memory manager. · Dynamic Management Views sys.dm_os_memory_nodes provide this information as well · The output of the DBCC MEMORYSTATUS command also provides this information Memory Manager KB VM Reserved VM Committed 12436 AWE Allocated 46032 Large Pages Allocated 0 Emergency Memory 1024 Emergency Memory In Use 16 Target Committed Current Committed 58472 Pages Allocated 43560 Pages Reserved 0 Pages Free 2000 Pages In Use 43904 Page Alloc Potential NUMA Growth Phase 0 Last OOM Factor 0 Last OS Error 0 [Session Code]
14
SQL Server 2012 内存管理 memory management
[Session Code]
15
SQL Operating System (SQLOS)
What is it? Granular user mode OS enrichment layer specifically designed and highly optimized for SQL workloads (OLTP & DW) Services provided: scheduling and synchronization support, memory management & caching, resource governance, Diagnostics & debug infrastructure, hosting support, scalability & performance optimization (example >64logical processor support, hot add CPU/memory) Currently the SQLOS is used by SQL Server relational database engine and Reporting Services for system level services Platforms include 32bit, 64bit (Itanium support ends in SQL 2008 R2), SMP, NUMA All SKUs supported except Compact Edition What it is not? An OS Abstraction layer, i.e. does not wrap OS APIs (like .NET)
16
SQLOS System Architecture View
Storage Engine SQLOS API Deadlock Monitor Hosting API Scheduling Memory Manager Synchronization Buffer Pool Resource Monitor Lazy Writer I/O External Components SQLOS Scheduler Monitor Protocols Query Comp. and Execution Engines = thread XEvents Resource Governor
17
Why change the Memory Manager
Difficult to accurately size and govern SQL Server memory consumption sp_configure ‘max server memory’ does not limit all memory committed by SQL Server Resource Governor only governs part of SQL memory consumption – single page allocations Must leave enough memory for multi-page allocations which can be troublesome for consolidation scenarios Complex queries and plans that allocate more than 8KB of memory have poor scalability & performance Does not support different memory models (locked, large pages) No caching for larger allocations greater than 64KB Out-of-memory handling is inconsistent across different internal components Summary of changes Below is a high-level summary of all changes related to memory management in SQL Server 2012. · The new memory manager supports allocations of all sizes. There is no concept of single-page allocator and multi-page allocator. For any of these sizes, the memory manager supports the three available memory models. The three memory models available in the SQL Server are conventional, lock pages, large pages. Large page is available only in 64 bit. · The new memory manager can be tuned for specific workloads [examples: SQL Server relational engine, SQL Server reporting service, etc]. These tuning options are available for the component developers and not exposed to the external customers or DBA's. · The new memory manager supports locality [CPU and NUMA] for allocations of all sizes. The memory node is created for all NUMA nodes even if there are no CPU's used there. That memory node is set offline. · The management of virtual address space is dynamic now - especially in 32-bit instances of SQL Server. · The configuration option "max server memory" now controls all the memory allocations done by the SQL Server memory manager. This is different from what the previous versions of SQL Server accounted for. As a result, resource governor is now able to control all page allocations. · AWE feature to map more physical memory in 32-bit virtual address space is removed from SQL Server memory manager · The SQL Server Buffer is a pure client of the SQL Server memory manager.
18
Memory Manager Architecture
SQL Server 2008 R2 SQL Server 2012 Page Reservation Memory Objects CLR Single-page Allocator Memory Manager Buffer Pool Multi-page Allocator VAS Allocator -g MemToReserve Page Reservation Memory Objects CLR Any size page Allocator Memory Manager Buffer Pool VAS Allocator -g MemToReserve VAS VAS
19
Memory Manager Architecture
SQL Server 2008 R2 SQL Server 2012 Page Reservation Memory Objects CLR Single-page Allocator Memory Manager Buffer Pool Multi-page Allocator VAS Allocator -g MemToReserve Page Reservation Buffer Pool Memory Objects CLR Any size page Allocator VAS Allocator Memory Manager -g MemToReserve VAS VAS
20
New Memory Manager Overview
Hierarchies of fixed size allocators Top level committed blocks VAS Management Fragment Manager Top Level Block Allocator Workspace 1 Workspace 2 Fragment: large reserved region of VAS. Top Level Block: Blocks of VAS allocated directly from fragments. All top level blocks have the same size and partitioned by NUMA Workspace: A set of top level blocks. The allocations from a workspace will have similar lifetime..
21
Example of Workspace Hierarchy
16MB 4MB 128KB 512KB 8KB 16KB 32KB 64KB
22
Example of Workspace Hierarchy
16MB 4MB 128KB 512KB 8KB 16KB 32KB 64KB Workspace hierarchy Tuned for typical SQL Server workloads Allows fast allocation of memory needed for SQL Server workloads
23
Surface Area Configuration Changes
sp_configure ‘max server memory’: Allows users to dynamically configure the minimum and maximum physical memory SQL Server can consume OS OS All size page allocations + CLR DWA, TS Memory allocations in SQL Server process space ‘max server memory’ SQL Server 2012 Memory allocations in SQL Server process space ‘max server memory’ CLR, MPA, DWA, TS Buffer Pool (SPA) SQL 2008 R2 & Before
24
Surface Area Configuration Changes
SQL Server 2008 R2 SQL Server 2012 Startup option –f (minimal configuration) behavior unchanged sp_configure option Default setting Min Value Max Value min server memory 16 (MB) Value less than max server memory setting max server memory (available memory in the system) sp_configure option Default setting Min Value Max Value min server memory 16 (MB) Value less than max server memory setting max server memory (available memory in the system) 32 bit - 64 (MB) 64 bit – 128 (MB)
25
Surface Area Configuration Changes
Configuration Option: sp_configure awe_enabled SQL Server 2008 R2 Latest release to support awe_enabled option, no effect for 64bit SQL Server 2012 No support for >4GB or Hot Add Memory in 32bit architectures Exists in 32bit for use of Locked pages (not needed for 64bit) AWE exposure in 32bit no longer valid (DMVs/PerfMon) 64bit usage of AWE for Locked Pages unchanged Main reasons behind AWE deprecation Windows Server 2008 R2 is only 64bit, dwindling new 32bit SQL Server deployments on large machines, and most commodity hardware is now 64bit
26
Surface Area Configuration Changes
Startup Option: –g memory_to_reserve (32bit) SQL Server 2008 R2 Specifies amount of VAS SQL Server would leave for memory allocations in process made outside of the Buffer Pool – Direct Window Allocations >8K, CLR, and thread stacks Default value is 256MB and relevant only for 32bit systems where VAS is limited SQL Server 2012 Specifies the amount of VAS SQL Server would leave for memory allocations in process made outside of Memory Manager – Direct Windows Allocations & thread stacks Memory Manager now manages >8K and CLR allocations
27
Surface Area Changes - Exposures
Consolidation of single and multipage allocator data Standardize unit of memory reported (KB) where possible sys.dm_os_memory_nodes SQL Server 2012 SQL Server 2008 R2 pages_kb single_pages_kb + multi_pages_kb foreign_committed_kb - sys.dm_os_sys_info SQL Server 2012 SQL Server 2008 R2 physical_memory_kb physical_memory_in_bytes virtual_memory_kb virtual_memory_in_bytes committed_kb bpool_committed committed_target_kb bpool_commit_target visible_target_kb bpool_visible
28
Surface Area Changes - Exposures
sys.dm_os_memory_cache_counters SQL Server 2012 SQL Server 2008 R2 pages_kb single_pages_kb + multi_pages_kb pages_in_use_kb single_pages_in_use_kb + multi_pages_in_use_kb sys.dm_os_memory_cache_entries SQL Server 2012 SQL Server 2008 R2 pages_kb pages_allocated_count sys.dm_os_memory_clerks SQL Server 2012 SQL Server 2008 R2 pages_kb single_pages_kb + multi_pages_kb page_size_in_bytes page_size_bytes sys.dm_os_memory_objects SQL Server 2012 SQL Server 2008 R2 pages_in_bytes pages_allocated_count max_pages_in_bytes max_pages_allocated_count
29
Surface Area Changes - Exposures
SQL Server 2012 SQL Server 2008 R2 Memory Manager \ Large Pages Allocated - Memory Manager \ Emergency Memory Memory Manager \ Reserved Memory Memory Manager \ Emergency Memory In Use Memory Manager \ Reserved Memory In Use Memory Manager \ Target Committed (KB) Buffer Pool \ Target (Pages) Memory Manager \ Current Committed (KB) Buffer Pool \ Committed (Pages) Memory Manager \ Pages Allocated (KB) Buffer Pool \ Stolen (Pages) + Buffer Pool \ Database (Pages) Memory Manager \ Pages Reserved (KB) Buffer Pool \ Reserved (Pages) Memory Manager \ Pages Free (KB) Buffer Pool \ Free (Pages) Memory Manager \ Pages In Use Memory Manager \ Page Alloc Potential (KB) Buffer Pool \ Stolen Potential (Pages) Memory Manager \ NUMA Growth Phase Memory Manager \ Last OOM Factor Buffer Pool \ Last OOM Factor Memory Manager \ Last OS Error
30
Surface Area Changes - Exposures
SQL Server 2012 SQL Server 2008 R2 Memory node Id = n \ Pages Allocated Memory node Id = n \ MultiPage Allocator + Memory node Id = n \ SinglePage Allocator Memory node Id = n \ Target Committed - Memory node Id = n \ Current Committed Memory node Id = n \ Foreign Committed Memory node Id = n \ Away Committed Memory Clerks \ Pages Allocated Memory Clerks \ SinglePage Allocator + Memory Clerks \ MultiPage Allocator
31
Surface Area Changes - Exposures
SQL Server 2012 SQL Server 2008 R2 Object - SQLServer:Memory Manager Object - SQLServer:Buffer Manager Database Cache Memory (KB) Database pages Free Memory (KB) Free pages Reserved Server Memory (KB) Reserved pages Stolen Server Memory (KB) Stolen pages Target Server Memory (KB) Target pages Total Server Memory (KB) Total pages - AWE lookup maps / sec AWE stolen maps / sec AWE write maps / sec AWE unmap calls / sec AWE unmap pages / sec
32
Surface Area Changes - Exposures
SQL Server 2012 SQL Server 2008 R2 Object - SQLServer:Memory Node Object - SQLServer:Buffer Node Database Node Memory (KB) Database pages Free Node Memory (KB) Free pages Foreign Node Memory (KB) Foreign pages Stolen Node Memory (KB) Stolen pages Target Node Memory (KB) Target pages Total Node Memory (KB) Total pages The “SQLServer:Buffer Partition” performance counter has been removed in SQL Server 2012!
33
Surface Area Changes - Exposures
Total Physical memory and the memory model SQL Server is configured to use (conventional, locked, large)
34
SQL server 内存性能瓶颈检测 [Session Code]
35
What is memory pressure
External Internal Physical Physical memory (RAM) running low. working sets trim SQL Server detects it and shrinks high memory consumption internally not enough for some components (say, QE etc) Virtual Running low on available memory commitment (the difference between Memory:Commit Limit and Memory:Committed Bytes) is low (could be due to lack of space in the system page files). Running low on VAS (direct allocations, DLLs loaded in SQL Server VAS, high number of threads) or VAS fragmentation (a lot of VAS is available but in small blocks). MTL area. SQL Server detects this and responds. Windows has a notification mechanism that reports if physical memory is running high or low. SQL Server uses this mechanism in its memory management decisions. For more information, see QueryMemoryResourceNotification in the Win32 and COM Development documentation in the MSDN Library. [Session Code]
36
Tools to detect memory pressure
Performance monitor logs OS Memory object SQL objects SQL server memory DMV(i.e,sys.dm_os_memory_clerks), DBCC MEMORYSTATUS output SQL server error logs, windows event logs etc [Session Code]
37
External Physical Memory Pressure
Memory object Available [M, K]Bytes, pages/sec Process object Working Set and private bytes counter for each process (AWE part is not in the report) For AWE part, check DMV output sys.dm_os_memory_clerks sys.dm_os_sys_memory [Session Code]
38
External Virtual Memory Pressure
Memory: Commit Limit the amount of virtual memory that can be committed without extending page file space. Paging File: %Usage, Paging File: %Usage Peak. Process: Working Set from the Process Private Bytes counters. How much process memory has been paged out sys.dm_os_sys_memory total_page_file_kb available_page_file_kb [Session Code]
39
Internal Physical Memory Pressure
DBCC MEMORYSTATUS Target committed decreases?-> external physical memory pressure. Target committed<current committed? Target committed low, privates bytes high-> find out larger memory consumer using DMV Target and “max server memory” Bufferpool VM committed/ Memory Manager:VM committed <20% when current committed is near target committed sys.dm_os_memory_clerks, sys.dm_os_memory_cache_clock_hands sys.dm_os_memory_cache_counters sys.dm_os_ring_buffers Performance monitor SQL Server: Buffer Manager object High number of Checkpoint pages/sec High number of Lazy writes/sec XEvent Process/System Counts Value Available Physical Memory Available Virtual Memory Available Paging File Working Set Percent of Committed Memory in WS Page Faults System physical memory high System physical memory low Process physical memory low Process virtual memory low (10 row(s) affected) Memory Manager KB VM Reserved VM Committed Locked Pages Allocated Large Pages Allocated Emergency Memory Emergency Memory In Use Target Committed Current Committed Pages Allocated Pages Reserved Pages Free Pages In Use Page Alloc Potential NUMA Growth Phase Last OOM Factor Last OS Error (16 row(s) affected) Memory node Id = KB VM Reserved VM Committed Target Committed Current Committed Foreign Committed Away Committed (9 row(s) affected) Memory node Id = KB VM Reserved VM Committed (3 row(s) affected) MEMORYCLERK_SQLGENERAL (node 0) KB VM Committed SM Reserved SM Committed Pages Allocated (6 row(s) affected) MEMORYCLERK_SQLBUFFERPOOL (node 0) KB VM Reserved VM Committed Pages Allocated MEMORYCLERK_SQLQUERYEXEC (node 0) KB Pages Allocated MEMORYCLERK_SQLOPTIMIZER (node 0) KB Pages Allocated MEMORYCLERK_SQLUTILITIES (node 0) KB Pages Allocated MEMORYCLERK_SQLSTORENG (node 0) KB VM Reserved VM Committed Pages Allocated MEMORYCLERK_SQLCONNECTIONPOOL (node 0) KB Pages Allocated MEMORYCLERK_SQLCLR (node 0) KB VM Reserved VM Committed Pages Allocated MEMORYCLERK_SQLSERVICEBROKER (node 0) KB Pages Allocated MEMORYCLERK_SQLHTTP (node 0) KB Pages Allocated MEMORYCLERK_SNI (node 0) KB Pages Allocated MEMORYCLERK_SNI (node 64) KB Pages Allocated MEMORYCLERK_SNI (Total) KB Pages Allocated MEMORYCLERK_FULLTEXT (node 0) KB Pages Allocated MEMORYCLERK_SQLXP (node 0) KB MEMORYCLERK_XE_BUFFER (node 0) KB VM Reserved VM Committed Pages Allocated MEMORYCLERK_XTP (node 0) KB MEMORYCLERK_HOST (node 0) KB MEMORYCLERK_SOSNODE (node 0) KB Pages Allocated MEMORYCLERK_SOSNODE (node 64) KB Pages Allocated MEMORYCLERK_SOSNODE (Total) KB Pages Allocated MEMORYCLERK_SOSOS (node 0) KB Pages Allocated MEMORYCLERK_SOSMEMMANAGER (node 0) KB VM Reserved VM Committed MEMORYCLERK_FULLTEXT_SHMEM (node 0) KB SM Reserved SM Committed MEMORYCLERK_SQLSERVICEBROKERTRANSPORT (node 0) KB VM Reserved VM Committed Locked Pages Allocated SM Reserved SM Committed Pages Allocated MEMORYCLERK_FILETABLE (node 0) KB MEMORYCLERK_XE (node 0) KB Pages Allocated MEMORYCLERK_SQLLOGPOOL (node 0) KB Pages Allocated MEMORYCLERK_LWC (node 0) KB Pages Allocated MEMORYCLERK_FSCHUNKER (node 0) KB CACHESTORE_OBJCP (node 0) KB CACHESTORE_SQLCP (node 0) KB Pages Allocated CACHESTORE_PHDR (node 0) KB CACHESTORE_XPROC (node 0) KB CACHESTORE_TEMPTABLES (node 0) KB CACHESTORE_NOTIF (node 0) KB CACHESTORE_VIEWDEFINITIONS (node 0) KB CACHESTORE_XMLDBTYPE (node 0) KB CACHESTORE_XMLDBELEMENT (node 0) KB CACHESTORE_XMLDBATTRIBUTE (node 0) KB CACHESTORE_STACKFRAMES (node 0) KB CACHESTORE_STACKFRAMES (node 64) KB CACHESTORE_STACKFRAMES (Total) KB CACHESTORE_BROKERTBLACS (node 0) KB Pages Allocated CACHESTORE_BROKERKEK (node 0) KB CACHESTORE_BROKERDSH (node 0) KB CACHESTORE_BROKERUSERCERTLOOKUP (node 0) KB CACHESTORE_BROKERRSB (node 0) KB CACHESTORE_BROKERREADONLY (node 0) KB Pages Allocated CACHESTORE_BROKERTO (node 0) KB CACHESTORE_EVENTS (node 0) KB CACHESTORE_SEHOBTCOLUMNATTRIBUTE (node 0) KB VM Reserved VM Committed Locked Pages Allocated SM Reserved SM Committed Pages Allocated CACHESTORE_SYSTEMROWSET (node 0) KB Pages Allocated CACHESTORE_CONVPRI (node 0) KB Pages Allocated CACHESTORE_FULLTEXTSTOPLIST (node 0) KB Pages Allocated CACHESTORE_SEARCHPROPERTYLIST (node 0) KB CACHESTORE_COLUMNSTOREOBJECTPOOL (node 0) KB Pages Allocated USERSTORE_SCHEMAMGR (node 0) KB Pages Allocated USERSTORE_DBMETADATA (node 0) KB Pages Allocated USERSTORE_TOKENPERM (node 0) KB Pages Allocated USERSTORE_OBJPERM (node 0) KB Pages Allocated USERSTORE_SXC (node 0) KB USERSTORE_SXC (node 64) KB USERSTORE_SXC (Total) KB Pages Allocated OBJECTSTORE_LBSS (node 0) KB OBJECTSTORE_SNI_PACKET (node 0) KB Pages Allocated OBJECTSTORE_SNI_PACKET (node 64) KB OBJECTSTORE_SNI_PACKET (Total) KB Pages Allocated OBJECTSTORE_SERVICE_BROKER (node 0) KB Pages Allocated OBJECTSTORE_LOCK_MANAGER (node 0) KB VM Reserved VM Committed Pages Allocated OBJECTSTORE_LOCK_MANAGER (node 64) KB OBJECTSTORE_LOCK_MANAGER (Total) KB Pages Allocated OBJECTSTORE_SECAUDIT_EVENT_BUFFER (node 0) KB VM Reserved VM Committed Locked Pages Allocated SM Reserved SM Committed Pages Allocated OBJECTSTORE_XACT_CACHE (node 0) KB OBJECTSTORE_XACT_CACHE (node 64) KB OBJECTSTORE_XACT_CACHE (Total) KB Buffer Pool Value Database Simulated Target Dirty In IO Latched Page Life Expectancy (7 row(s) affected) Procedure Cache Value TotalProcs TotalPages InUsePages Global Memory Objects Pages Resource Locks XDES DirtyPageTracking SETLS SubpDesc Allocators SE SchemaManager SE Column Metadata Cache SE Column Metadata Cache Store SQLCache Replication ServerGlobal XP Global SortTables (14 row(s) affected) Query Memory Objects (internal) Value Grants Waiting Available Current Max Future Max Physical Max Next Request Waiting For Cost Timeout Wait Time (11 row(s) affected) Small Query Memory Objects (internal) Value Available Current Max Future Max (5 row(s) affected) Remote Query Memory Objects (internal) Value Available Current Max (4 row(s) affected) Query Memory Objects (default) Value Available Current Max Future Max Physical Max Small Query Memory Objects (default) Value Available Current Max Future Max Remote Query Memory Objects (default) Value Available Current Max Optimization Queue (internal) Value Overall Memory Target Memory Last Notification Timeout Early Termination Factor Small Gateway (internal) Value Configured Units Available Units Acquires Waiters Threshold Factor Threshold Medium Gateway (internal) Value Configured Units Available Units Threshold Factor Threshold Big Gateway (internal) Value Configured Units Available Units Threshold Factor Optimization Queue (default) Value Target Memory Small Gateway (default) Value Medium Gateway (default) Value Big Gateway (default) Value Memory Pool Manager Pages Reserved Current Reserved Limit (2 row(s) affected) Memory Pool (internal) Pages Allocations Predicted Private Target Private Limit Total Target Total Limit OOM Count MEMORYBROKER_FOR_CACHE (internal) Pages Allocations Rate Target Allocations Future Allocations Overall MEMORYBROKER_FOR_STEAL (internal) Pages Allocations Target Allocations MEMORYBROKER_FOR_RESERVE (internal) Pages Allocations Target Allocations MEMORYBROKER_FOR_COMMITTED (internal) Pages Allocations Target Allocations MEMORYBROKER_FOR_XTP (internal) Pages Allocations Target Allocations Memory Pool (default) Pages Allocations Predicted MEMORYBROKER_FOR_CACHE (default) Pages Allocations Rate Target Allocations MEMORYBROKER_FOR_STEAL (default) Pages Allocations Target Allocations MEMORYBROKER_FOR_RESERVE (default) Pages Target Allocations Future Allocations MEMORYBROKER_FOR_XTP (default) Pages Memory Broker Clerk (Buffer Pool) Pages Total Simulation Benefit Internal Benefit External Benefit Value Of Memory Periodic Freed Internal Freed (8 row(s) affected) DBCC execution completed. If DBCC printed error messages, contact your system administrator. RING_BUFFER_OOM This ring buffer contains records indicating server out-of-memory conditions. A record may look like this. Record id="0" type="RING_BUFFER_OOM" time=" "> <OOM> <Action>FAIL_PAGE_ALLOCATION</Action> <Resources>1</Resources> <Task>0x FDC8</Task> <Pool>258</Pool> </OOM> <MemoryNode id="0"> <ReservedMemory> </ReservedMemory> <CommittedMemory>48096</CommittedMemory> <SharedMemory>0</SharedMemory> <AWEMemory>114688</AWEMemory> <SinglePagesMemory>28848</SinglePagesMemory> <MultiplePagesMemory>26368</MultiplePagesMemory> </MemoryNode> <MemoryRecord> <MemoryUtilization>100</MemoryUtilization> <TotalPhysicalMemory> </TotalPhysicalMemory> <AvailablePhysicalMemory> </AvailablePhysicalMemory> <TotalPageFile> </TotalPageFile> <AvailablePageFile> </AvailablePageFile> <TotalVirtualAddressSpace> </TotalVirtualAddressSpace> <AvailableVirtualAddressSpace> </AvailableVirtualAddressSpace> <AvailableExtendedVirtualAddressSpace>0</AvailableExtendedVirtualAddressSpace> </MemoryRecord> <Stack> ... (output truncated) </Stack> </Record> [Session Code]
40
Internal Virtual Memory Pressure
Querying the sys.dm_os_virtual_address_dump DMV results a map of SQL Server VAS by allocation, effectively returning the same information as a VirtualQuery Windows API call. VMMAP tool /* Map the process virtual address space by querying sys.dm_os_virtual_address_dump */ SELECT region_base_address 'Base addr' ,region_size_in_bytes / 1024 size_kb ,case (region_state) when CONVERT(int, 0x1000) then 'COMMITTED' when CONVERT(int, 0x2000) then 'RESERVED' when CONVERT(int, 0x10000) then 'FREE' end State ,case when (region_current_protection = 0) then 'NONE' when (region_current_protection = CONVERT(int, 0x104)) then 'READ/WRITE/GUARD' when (region_current_protection ^ 1 = 0) then 'NO ACCESS' when (region_current_protection ^ 2 = 0) then 'READ' when (region_current_protection ^ 4 = 0) then 'READ/WRITE' when (region_current_protection ^ 8 = 0) then 'WRITE/COPY' when (region_current_protection ^ CONVERT(int, 0x20) = 0) then 'EXECUTE/READ' when (region_current_protection ^ CONVERT(int, 0x40) = 0) then 'EXECUTE/READ/WRITE' when (region_current_protection ^ CONVERT(int, 0x80) = 0) then 'EXECUTE/WRITE/COPY' end Protection ,case (region_type) when 0 then 'FREE' when CONVERT(int, 0x20000) then 'PRIVATE' when CONVERT(int, 0x40000) then 'MAPPED' when CONVERT(int, 0x ) then 'IMAGE' end 'Region Type' FROM sys.dm_os_virtual_address_dump order by region_base_address GO Another simple query to run on sys.dm_os_virtual_address_dump for a quick snapshot of allocated memory is to sum the different types of allocation. E.g. /* Get total Committed, Reserved, Free pages at a glance */ select SUM(case (region_state) when CONVERT(int, 0x1000) then region_size_in_bytes /1024 else 0 end) "Committed K", SUM(case (region_state) when CONVERT(int, 0x2000) then region_size_in_bytes /1024 else 0 end) "Reserved K", SUM(case (region_state) when CONVERT(int, 0x10000) then region_size_in_bytes /1024 else 0 end) "Free K" from sys.dm_os_virtual_address_dump where region_base_address > 0 who is using/eating up the Virtual Address space on your SQL Server Will tell you the size of the biggest contiguous block in VAS view source print? 1SELECT convert(varchar,getdate(),120) as [Timestamp], max(region_size_in_bytes)/1024 [Total max contiguous block size in KB] from sys.dm_os_virtual_address_dump where region_state = 0x MEM_FREE 2. Will also tell us size of largest contiguous block plus the region marked as MEM_RESERVE (this is your non-BPool area reserved during SQL Startup, sometimes referred to as MTL – MemToLeave) 01With VASummary(Size,Reserved,Free) AS 02(SELECT 03 Size = VaDump.Size, 04 Reserved = SUM(CASE(CONVERT(INT, VaDump.Base)^0) 05 WHEN 0 THEN 0 ELSE 1 END), 06 Free = SUM(CASE(CONVERT(INT, VaDump.Base)^0) 07 WHEN 0 THEN 1 ELSE 0 END) 08FROM 09( 10 SELECT CONVERT(VARBINARY, SUM(region_size_in_bytes)) 11 AS Size, region_allocation_base_address AS Base 12 FROM sys.dm_os_virtual_address_dump 13 WHERE region_allocation_base_address <> 0x0 14 GROUP BY region_allocation_base_address 15UNION 16 SELECT CONVERT(VARBINARY, region_size_in_bytes), region_allocation_base_address 17 FROM sys.dm_os_virtual_address_dump 18 WHERE region_allocation_base_address = 0x0 19) 20AS VaDump 21GROUP BY Size) 22SELECT SUM(CONVERT(BIGINT,Size)*Free)/1024 AS [Total avail Mem, KB] ,CAST(MAX(Size) AS BIGINT)/1024 AS [Max free size, KB] 23FROM VASummary 24WHERE Free <> 0 3. Now, below query will identify the memory reserved by non-BPool components in SQL Server 1select SUM(virtual_memory_reserved_kb)/1024 as virtual_memory_reserved_mb from sys.dm_os_memory_clerks where type not like '%bufferpool%' 4. To identify if any of the space used is by SQL Server procedure cache itself, you can use this query: 1SELECT SUM(PAGESUSED)*8/1024 'MB of MemToLeave memory consumed by procedures' FROM MASTER.DBO.SYSCACHEOBJECTS WHERE PAGESUSED >1 [Session Code]
41
Common memory error: 701 - There is insufficient system memory in resource pool 'pool_name' to run this query. Performance counters sys.dm_os_memory_clerks sys.dm_resource_governor_resource_pools DBCC FREESYSTEMCACHE Optimize query 701 - There is insufficient system memory in resource pool 'pool_name' to run this query. Causes This is generic out-of-memory error for the server. It indicates a failed memory allocation. It can be due to a variety of reasons, including hitting memory limits on the current workload. With increased memory requirements for SQL Server 2008 and SQL Server 2005 and certain configuration settings (such as the max server memory option and Resource Governor configuration settings), users are more likely to see this error. Usually the transaction that failed is not the cause of this error. Check the out_of_memory_count column of the sys.dm_resource_governor_resource_pools DMV. If this count is localized to a particular resource pool, Resource Governor configuration is the most likely reason. Troubleshooting Regardless of whether the error is consistent and repeatable (that is, it stays in the same state) or random (that is, it appears at random times with different states), you should investigate server memory distribution during the time you see this error. When this error is present, it is possible that the diagnostic queries will fail. When you see this error, the best place to start investigation is the error log. It should contain output that looks something like this. :27:15.43 spid Failed allocate pages: FAIL_PAGE_ALLOCATION 1 or :27:15.43 spid Failed Virtual Allocate Bytes: FAIL_VIRTUAL_RESERVE 65536 The possible failures are: FAIL_PAGE_ALLOCATION followed by the number of pages attempted to allocate. FAIL_VIRTUAL_RESERVE followed by the number of bytes attempted to reserve. FAIL_VIRTUAL_COMMIT followed by the number of bytes attempted to commit. Usually the task that first encountered the out-of-memory error is not the task that caused the condition. Most likely it is a cumulative effect of multiple tasks running. For the very common case of a single page allocation failure, your investigation should take the global picture into account. The next piece of information from error log is the memory status output. Depending on the failure, you should look for single page, multipage, virtual reserved or committed numbers for individual memory clerks. Identifying the biggest memory consumers is key to proceeding with investigation. You may find that the biggest consumers are of the following type: MEMORYCLERK_* means that the server configuration or workload requires so much memory to be allocated. The offending workload can sometimes be identified just by the memory clerks, but more often you will have to drill further into the memory objects associated with the clerks in order to find out what causes such memory consumption. CACHESTORE_*, USERSTORE_*, OBJECTSTORE_* are the types of caches. Big consumption by a cache may mean the following: Memory is allocated out of the cache but is not inserted yet as an entry that can be evicted. This is very similar to the MEMORYCLERK case discussed earlier. All cache entries are in use so they cannot be evicted. You can confirm this by looking at the sys.dm_os_memory_cache_counters DMV and comparing the entries_count and entries_in_use_count columns. Most cache entries are not in use. This case most likely indicates a bug in the server. MEMORYCLERK_SQLQERESERVATIONS shows how much memory has been reserved by the query execution (QE) to run queries with sorts/joins. The memory status output in the error log also shows which Resource Governor resource pool memory is exhausted. The memory brokers for every pool show the memory distribution between stolen (compilation), cached, and reserved (granted) memory. The numbers for the three brokers correspond to the three bullet points in the previous list. Unfortunately there is no way to find out how much memory is allocated for a pool from a given clerk or memory object. The sys.dm_os_memory_cache_entries DMV is extended to show the pool_id each entry is associated with. Possible solutions include the following: Remove external memory pressure. Increase the max server memory setting, and then adjust the MIN_MEMORY_PERCENT and MAX_MEMORY_PERCENT settings for the resource pool. Free caches by using one of the following commands: DBCC FREESYSTEMCACHE, DBCC FREESESSIONCACHE, or DBCC FREEPROCCACHE. If the problem reappears, reduce the workload. [Session Code]
42
References Locked pages KB CPU affinity Books Online Topic AWE deprecation announce in SQL 2008 R2 Performance Troubleshooting whitepaper SQL Consolidation prescriptive guidance KB Articles The "awe enabled" SQL Server feature is deprecated How to enable the "locked pages" feature in SQL Server Memory configuration and sizing considerations in SQL Server SQL Server setup upgrade rules for lock pages check does not work properly FIX: Out-of-memory error when you run an instance of SQL Server 2012 on a computer that uses NUMA FIX: Locked page allocations are enabled without any warning after you upgrade to SQL Server KB articles planned for future: SQL 2012 KBA: How to use the new memory related extended events to track memory problems SQL 2012 KBA: How to use the DBCC MEMORYSTATUS command to monitor memory usage SQL 2012 KBA: How to use the memory manager system monitor counters SQL 2012 KBA: Effects of new licensing policy on NUMA systems and memory management BLOGS Upgrading to SQL Server 2012 with Lock Pages in Memory
43
THANK YOU!
44
What else changed? Or didn’t …
CPU affinity configuration works as before Continue to use the DDL ALTER SERVER CONFIGURATION SET PROCESS AFFINITY No changes in Resource Governor behavior except that it now controls all page allocations VAS commitment for CLR now classified and charged against the internal memory pool Database cache pages are still not governed by Resource Governor
45
New Memory Manager Benefits
Predictable sizing and governing of memory consumption ‘max server memory’ limits all memory committed by SQL Server Resource Governor governs all SQL memory consumption Improved performance & scalability of complex queries and operations making greater than 8KB allocations Improved CPU and NUMA locality for memory accesses Advantage of a single Memory Manager that services all size page allocations requests Consistent out-of-memory handling & management across different internal components
46
Sizing the Memory Manager
Setting ‘max server memory’ Can be set on a dedicated server, but not necessary Must leave enough memory for OS and SQL Server worker thread stacks (~2MB/thread on x64) For multi-instance and virtual environments Set when using Large Pages memory model Setting ‘min server memory’ To guarantee a necessary amount of memory To make the memory model static Locked Pages for stability should not be necessary on deployments using Windows Server 2008 and beyond
47
Multi-Instance Consolidation Scenarios
Run workloads, observe characteristics and understand the baseline set of requirements for an instance Determine the processor and memory requirements for each instance Isolate the processors and memory for each instance ALTER SERVER CONFIGURATION SET PROCESS AFFINITY Do not set affinity from Task Manager Keep instances within node boundaries Always set ‘max server memory’ when using affinity Avoid using more memory than available in nodes Better to change configuration with idle server
48
Multi-Instance Consolidation Example
Machine: 12 LPs - 48 GB RAM Node 0 (16GB) Node 1 (16GB) Node 2 (16GB) LPs: 8 LPs: 2 LPs: 2 Memory: 32GB Mem: 12GB Mem: 4GB LP = Logical Processor: Computing engine in the OS, application and driver view
49
Terms and Concepts Physical Memory Virtual Address Space (VAS)
Refers to Random Access Memory (RAM) Virtual Address Space (VAS) A contiguous region of memory addresses Every process has its own which provides isolation Up to 4GB on 32 bit and 8TB on 64 bit, can become fragmented A region has to be ‘reserved’ and then ‘committed’ before using Commitment is backing reserved regions of VAS with physical memory Conventional: Normal physical page size (8KB), page-able, dynamic Locked Pages: Normal physical page size, non-page-able, dynamic Large Pages: Large physical page size (>=2MB), non-page-able, static In SQL Server a page is a fully committed VAS region Page size is 8KB, and allocated through Single Page Allocation/Allocator Contiguous allocation of >8KB is called Multi-Page Allocation Direct Windows Allocations uses VirtualAlloc() and HeapAlloc() calls
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.