Staying within budget

When developing a graphics-intensive game or program, it is important to avoid allocating more GPU memory than it is physically available.

When the memory is over-committed, various bad things can happen, depending on the specific GPU, graphics driver, and operating system:

• It may just work without any problems.
• The application may slow down because some memory blocks are moved to system RAM and the GPU has to access them through PCI Express bus.
• A new allocation may take very long time to complete, even few seconds, and possibly freeze entire system.
• The new allocation may fail with VK_ERROR_OUT_OF_DEVICE_MEMORY.
• It may even result in GPU crash (TDR), observed as VK_ERROR_DEVICE_LOST returned somewhere later.

Querying for budget

To query for current memory usage and available budget, use function vmaGetHeapBudgets(). Returned structure VmaBudget contains quantities expressed in bytes, per Vulkan memory heap.

Please note that this function returns different information and works faster than vmaCalculateStatistics(). vmaGetHeapBudgets() can be called every frame or even before every allocation, while vmaCalculateStatistics() is intended to be used rarely, only to obtain statistical information, e.g. for debugging purposes.

It is recommended to use VK_EXT_memory_budget device extension to obtain information about the budget from Vulkan device. VMA is able to use this extension automatically. When not enabled, the allocator behaves same way, but then it estimates current usage and available budget based on its internal information and Vulkan memory heap sizes, which may be less precise. In order to use this extension:

1. Make sure extensions VK_EXT_memory_budget and VK_KHR_get_physical_device_properties2 required by it are available and enable them. Please note that the first is a device extension and the second is instance extension!
2. Use flag VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT when creating VmaAllocator object.
3. Make sure to call vmaSetCurrentFrameIndex() every frame. Budget is queried from Vulkan inside of it to avoid overhead of querying it with every allocation.

Controlling memory usage

There are many ways in which you can try to stay within the budget.

First, when making new allocation requires allocating a new memory block, the library tries not to exceed the budget automatically. If a block with default recommended size (e.g. 256 MB) would go over budget, a smaller block is allocated, possibly even dedicated memory for just this resource.

If the size of the requested resource plus current memory usage is more than the budget, by default the library still tries to create it, leaving it to the Vulkan implementation whether the allocation succeeds or fails. You can change this behavior by using VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT flag. With it, the allocation is not made if it would exceed the budget or if the budget is already exceeded. VMA then tries to make the allocation from the next eligible Vulkan memory type. The all of them fail, the call then fails with VK_ERROR_OUT_OF_DEVICE_MEMORY. Example usage pattern may be to pass the VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT flag when creating resources that are not essential for the application (e.g. the texture of a specific object) and not to pass it when creating critically important resources (e.g. render targets).

On AMD graphics cards there is a custom vendor extension available: VK_AMD_memory_overallocation_behavior that allows to control the behavior of the Vulkan implementation in out-of-memory cases - whether it should fail with an error code or still allow the allocation. Usage of this extension involves only passing extra structure on Vulkan device creation, so it is out of scope of this library.

Finally, you can also use VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT flag to make sure a new allocation is created only when it fits inside one of the existing memory blocks. If it would require to allocate a new block, if fails instead with VK_ERROR_OUT_OF_DEVICE_MEMORY. This also ensures that the function call is very fast because it never goes to Vulkan to obtain a new block.

Note

Creating Custom memory pools with VmaPoolCreateInfo::minBlockCount set to more than 0 will currently try to allocate memory blocks without checking whether they fit within budget.