Presentation is loading. Please wait.

Presentation is loading. Please wait.

Vulkan Memory Types Ashley Smith November 2018.

Similar presentations


Presentation on theme: "Vulkan Memory Types Ashley Smith November 2018."— Presentation transcript:

1 Vulkan Memory Types Ashley Smith November 2018

2 Agenda Heaps and types Tips and tricks The VMA library Conclusion

3 Effort vs control Effort required Legacy APIs Level of control
Achieve smaller footprint Better optimize for specific platforms Aliasing! Effort required Legacy APIs Level of control We have more control with Vulkan But we require more effort

4 Heaps and types

5 Figure out what type, alignment and size
Allocation Figure out what type, alignment and size Allocate memory Create a resource Resource Memory Bind With legacy APIs, applications take the lower path With lower level APIs, applications take the higher path More operations required

6 Allocating some memory
VkResult vkAllocateMemory( VkDevice device, const VkMemoryAllocateInfo* pAllocateInfo, const VkAllocationCallbacks* pAllocator, VkDeviceMemory* pMemory); typedef struct VkMemoryAllocateInfo { VkStructureType sType; const void* pNext; VkDeviceSize allocationSize; uint32_t memoryTypeIndex; } VkMemoryAllocateInfo; Allocate memory with a call to vkAllocateMemory()

7 Allocating some memory
VkResult vkAllocateMemory( VkDevice device, const VkMemoryAllocateInfo* pAllocateInfo, const VkAllocationCallbacks* pAllocator, VkDeviceMemory* pMemory); typedef struct VkMemoryAllocateInfo { VkStructureType sType; const void* pNext; VkDeviceSize allocationSize; uint32_t memoryTypeIndex; } VkMemoryAllocateInfo; But what do we pass in to memoryTypeIndex ?

8 Allocating some memory
vkGetPhysicalDeviceMemoryProperties( VkPhysicalDevice physicalDevice, VkPhysicalDeviceMemoryProperties* pMemoryProperties); typedef struct VkPhysicalDeviceMemoryProperties { uint32_t memoryTypeCount; VkMemoryType memoryTypes[VK_MAX_MEMORY_TYPES]; uint32_t memoryHeapCount; VkMemoryHeap memoryHeaps[VK_MAX_MEMORY_HEAPS]; } VkPhysicalDeviceMemoryProperties; We need to query the memory types and heaps that are present on the device Select an appropriate place for our resource

9 Allocating some memory
vkGetPhysicalDeviceMemoryProperties( VkPhysicalDevice physicalDevice, VkPhysicalDeviceMemoryProperties* pMemoryProperties); typedef struct VkPhysicalDeviceMemoryProperties { uint32_t memoryTypeCount; VkMemoryType memoryTypes[VK_MAX_MEMORY_TYPES]; uint32_t memoryHeapCount; VkMemoryHeap memoryHeaps[VK_MAX_MEMORY_HEAPS]; } VkPhysicalDeviceMemoryProperties; Next we will explain the memory types available

10 Allocating some memory
typedef enum VkMemoryPropertyFlagBits { VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT = 0x , VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT = 0x , VK_MEMORY_PROPERTY_HOST_COHERENT_BIT = 0x , VK_MEMORY_PROPERTY_HOST_CACHED_BIT = 0x , VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT = 0x , VK_MEMORY_PROPERTY_PROTECTED_BIT = 0x , } VkMemoryPropertyFlagBits; We will only be focusing on these memory properties today

11 Memory types vs. heaps Memory Type Heap 0 Type 0 Heap 1 Type 1 Heap 2
(RX Vega 64) Memory Type Heap 0 Type 0 Heap 1 Type 1 Heap 2 Type 2 Type 3 Keep in mind the size of each heap

12 Memory types cheat sheet
(RX Vega 64) Size Memory Type $ $ R W R W Most of VRAM 1 2 Fixed 256MiB 3 $ - Storage - Visible - Cached - Fast - Slow

13 Memory types cheat sheet
(RX Vega 64) Memory Type $ $ R W R W Size Most of VRAM Maps to VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT in VkMemoryPropertyFlagBits. 1 2 Fixed 256MiB 3 $ - Storage - Visible - Cached - Fast - Slow

14 Memory types cheat sheet
(RX Vega 64) $ $ R W R W Size Memory Type Most of VRAM 1 Maps to VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT and VK_MEMORY_PROPERTY_CACHED_BIT respectively. 2 Fixed 256MiB 3 $ - Storage - Visible - Cached - Fast - Slow

15 Memory types cheat sheet
(RX Vega 64) Memory Type $ $ R W R W Size Most of VRAM 1 2 Fixed 256MiB 3 $ - Storage - Visible - Cached - Fast - Slow Type 0: Video memory. Not visible by CPU at all, you can’t map it. Fast for access on the GPU. Straightforward. Type 1: Not in GPU memory, instead it’s in CPU memory, however, like all memory types it’s still visible and cached by the GPU. It is however, *not* cached by the CPU. So that means reads will be uncached. Writes may be write-combined to achieve decent write speed. Type 2: We’re now back in GPU memory again. Except this time the memory is visible by the CPU. However, it’s not cached - Write it and never read it from CPU. Type 3: We’re now back over in host memory again, with Type 3. This type is similar to Type 1, except it’s cached by the CPU (write-back). This means the GPU reads have to snoop CPU caches. Making both R&W quite slow.

16 Memory types cheat sheet
(RX Vega 64) Memory Type $ $ R W R W Size Most of VRAM Okay, not that bad since we benefit from GPU caches, but certainly worse than just reading from DEVICE_LOCAL. 1 2 Fixed 256MiB 3 $ - Storage - Visible - Cached - Fast - Slow

17 Memory types cheat sheet
(RX Vega 64) On current GPUs & drivers, PC Windows® everything that is HOST_VISIBLE is also marked COHERENT. Memory Type $ $ R W R W Size Most of VRAM 1 On other architectures you may need: vkInvalidateMappedMemoryRanges before reads and vkFlushMappedMemoryRanges after writes. BEWARE: Unmapping won’t do this for you! 2 Fixed 256MiB 3 $ - Storage - Visible - Cached - Fast - Slow

18 Memory types vs. heaps Memory Type Usage Notes Heap 0 Type 0
(RX Vega 64) Memory Type Usage Notes Heap 0 Type 0 Use for “upload-once” resources, render targets, depth buffers, etc. Heap 1 Type 1 Staging for uploads to VRAM. Fixed size of 256MiB. Use for dynamic, write once, read once resources. Heap 2 Type 2 Type 3 Written by GPU, read by CPU. E.g.: Downsampled luminance.

19 Warning VkMemoryAllocateInfo x = { .memoryTypeIndex = 2 }; vkAllocateMemory(&x, &mem);

20 Warning vkGetPhysicalDeviceMemoryProperties(&properties); VkMemoryAllocateInfo x = { .memoryTypeIndex = findAppropriateMemoryHeap(properties); }; vkAllocateMemory(&x, &mem);

21 Warning Not a good idea to hardcode the memory type indices
Driver may change in future May be different on other/newer hardware Magic numbers Query the info using vkGetPhysicalDeviceMemoryProperties Map to engine-specific enums

22 Tips and tricks

23 Allocation strategies
Is this a good idea? for( int i = 0; i < 1000; i++ ) { my_new_objects[i] = new x; }

24 Allocation strategies
Is this a good idea? for( int i = 0; i < 1000; i++ ) { my_new_objects[i] = new x; }

25 Allocation strategies
Is this a good idea? //for( int i = 0; i < 1000; i++ ) { // my_new_objects[i] = new x; //} my_new_objects = new x[1000];

26 Allocation strategies
Same idea on GPU for similar reasons: Fragmentation Performance Data locality Allocate reasonably large chunks of memory (256MiB) Just 16 allocations fills 4GiB of VRAM Good balance between flexibility and performance On Windows®7 Vulkan memory allocations have larger overhead Sub-allocate the memory for resources from these blocks

27 Allocation strategies
Linear allocator Stack allocator Double stack allocator Block allocator Ring buffer - Used memory - Free memory

28 Over subscription When you run out of memory:
VK_ERROR_OUT_OF_DEVICE_MEMORY VK_SUCCESS

29 Over subscription VK_ERROR_OUT_OF_DEVICE_MEMORY Allocation fails
Application must handle out-of-memory conditions Out-of-memory potentially changes per driver/hardware

30 Over subscription VK_SUCCESS Allocation succeeds
Some blocks are silently migrated to system memory Why would you want this? Useful for development purposes - Artists don’t always stick to budgets Some of your blocks might get paged anyway (you’re not alone on the machine) Application doesn’t have to handle out-of-memory Accessing blocks migrated to system memory can degrade GPU performance

31 Over subscription No way is exposed to control residency manually
No way is exposed to query the used/free memory To make things worse, there are other implicit resources which need memory too: Swap chains Command buffers Descriptors Shaders / PSOs Query results Use VkMemoryHeap::size then apply some “informed adjustments”: Flags Adjustment DEVICE_LOCAL VkMemoryHeap::size * 0.8f DEVICE_LOCAL | HOST_VISIBLE VkMemoryHeap::size * 0.66f Can now be controlled with VK_AMD_memory_overallocation_behavior However memory can still be migrated – still suggest “fudges” on limits

32 Vulkan memory allocator (VMA)

33 Vulkan memory allocator (VMA)
Free Open source MIT license Single header Simple, C99 interface. Same style as Vulkan™ Battle tested, already getting some love in the community

34 Vulkan memory allocator (VMA)
Function that help to choose the correct and optimal memory type based on intended usage Functions that allocate memory blocks, reserve and return parts of them to the user Allocation tracker, look at used/unused, and fragmentation Respects alignment and buffer/image granularity

35 Vulkan memory allocator (VMA)
VkBufferCreateInfo bufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO }; bufferInfo.size = 65536; bufferInfo.usage = VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT; VmaAllocationCreateInfo allocInfo = {}; allocInfo.usage = VMA_MEMORY_USAGE_GPU_ONLY; VkBuffer buffer; VmaAllocation allocation; vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, NULL);

36 Vulkan memory allocator (VMA)
Even has some tooling! VMA can dump allocator state to JSON Python script generates PNG file which shows the allocator contents Defragmentation support Unintialised memory checks, Out of bounds checks

37 Conclusion Vulkan is lower-level and requires explicit memory management Creating resources is a multi-stage process Former driver magic is now under your control You need to deal with differences between GPUs By following good practices you can achieve optimal performance on any GPU Vulkan Memory Allocator (VMA) is battle-tested and can really help a lot

38 Thank-you Adam Sawicki Dominik Baumeister Lou Kramer
Matthäus G. Chajdas Rys Sommefeldt Timothy Lottes Nicolas Thibieroz Alon Or-Bach

39

40 Memory aliasing As resolutions get larger, render targets follow suit
As many resources are transient, aliasing can be a solution to keep render target/UAV memory in check G-Buffers: Render G-Buffer Lighting Render Alphas Post-Processing Render passes:

41 Memory aliasing As resolutions get larger, render targets follow suit
As many resources are transient, aliasing can be a solution to keep render target/UAV memory in check Once lighting is done, we don’t need the g-buffer anymore. Let’s use it! G-Buffers: Render G-Buffer Lighting Render Alphas Post-Processing Render passes:

42 Memory aliasing As resolutions get larger, render targets follow suit
As many resources are transient, aliasing can be a solution to keep render target/UAV memory in check Post processing often does a lot of ping-ponging of RTs. Why not use again here? G-Buffers: Render G-Buffer Lighting Render Alphas Post-Processing Render passes:

43 Memory aliasing As resolutions get larger, render targets follow suit
As many resources are transient, aliasing can be a solution to keep render target/UAV memory in check Maybe some compute shader in here needs a nice big UAV for something. No need to allocate, alias with a Render target. G-Buffers: Render G-Buffer Lighting Render Alphas Post-Processing Render passes: Showing resource lifetime Frame graph style

44 Memory aliasing As resolutions get larger, render targets follow suit
As many resources are transient, aliasing can be a solution to keep render target/UAV memory in check For second, third, etc. use of aliased resource best to assume it contains garbage G-Buffers: Render G-Buffer Lighting Render Alphas Post-Processing Render passes: VK_IMAGE_LAYOUT_UNINITIALIZED

45 Memory aliasing As resolutions get larger, render targets follow suit
As many resources are transient, aliasing can be a solution to keep render target/UAV memory in check For second, third, etc. use of aliased resource best to assume it contains garbage >50% memory saved in some titles [ODonnell17] G-Buffers: Render G-Buffer Lighting Render Alphas Post-Processing Render passes: [ODonnell17] Yuriy O’Donnell – FrameGraph: Extensible Rendering Architecture in Frostbite

46 Conclusion Vulkan is lower-level and requires explicit memory management Creating resources is a multi-stage process Former driver magic is now under your control You need to deal with differences between GPUs By following good practices you can achieve optimal performance on any GPU Vulkan Memory Allocator (VMA) is battle-tested and can really help a lot

47 Thank-you Adam Sawicki Dominik Baumeister Lou Kramer
Matthäus G. Chajdas Rys Sommefeldt Timothy Lottes Nicolas Thibieroz Alon Or-Bach


Download ppt "Vulkan Memory Types Ashley Smith November 2018."

Similar presentations


Ads by Google