Defragmentation

Interleaved allocations and deallocations of many objects of varying size can cause fragmentation over time, which can lead to a situation where the library is unable to find a continuous range of free memory for a new allocation despite there is enough free space, just scattered across many small free ranges between existing allocations.

To mitigate this problem, you can use defragmentation feature. It doesn't happen automatically though and needs your cooperation, because VMA is a low level library that only allocates memory. It cannot recreate buffers and images in a new place as it doesn't remember the contents of VkBufferCreateInfo / VkImageCreateInfo structures. It cannot copy their contents as it doesn't record any commands to a command buffer.

Example:

VmaDefragmentationInfo defragInfo = {};
defragInfo.pool = myPool;
defragInfo.flags = VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT;

VmaDefragmentationContext defragCtx;
VkResult res = vmaBeginDefragmentation(allocator, &defragInfo, &defragCtx);
// Check res...

for(;;)
{
    VmaDefragmentationPassMoveInfo pass;
    res = vmaBeginDefragmentationPass(allocator, defragCtx, &pass);
    if(res == VK_SUCCESS)
        break;
    else if(res != VK_INCOMPLETE)
        // Handle error...

    for(uint32_t i = 0; i < pass.moveCount; ++i)
    {
        // Inspect pass.pMoves[i].srcAllocation, identify what buffer/image it represents.
        VmaAllocationInfo allocInfo;
        vmaGetAllocationInfo(allocator, pMoves[i].srcAllocation, &allocInfo);
        MyEngineResourceData* resData = (MyEngineResourceData*)allocInfo.pUserData;
            
        // Recreate and bind this buffer/image at: pass.pMoves[i].dstMemory, pass.pMoves[i].dstOffset.
        VkImageCreateInfo imgCreateInfo = ...
        VkImage newImg;
        res = vkCreateImage(device, &imgCreateInfo, nullptr, &newImg);
        // Check res...
        res = vmaBindImageMemory(allocator, pMoves[i].dstTmpAllocation, newImg);
        // Check res...

        // Issue a vkCmdCopyBuffer/vkCmdCopyImage to copy its content to the new place.
        vkCmdCopyImage(cmdBuf, resData->img, ..., newImg, ...);
    }
        
    // Make sure the copy commands finished executing.
    vkWaitForFences(...);

    // Destroy old buffers/images bound with pass.pMoves[i].srcAllocation.
    for(uint32_t i = 0; i < pass.moveCount; ++i)
    {
        // ...
        vkDestroyImage(device, resData->img, nullptr);
    }

    // Update appropriate descriptors to point to the new places...
        
    res = vmaEndDefragmentationPass(allocator, defragCtx, &pass);
    if(res == VK_SUCCESS)
        break;
    else if(res != VK_INCOMPLETE)
        // Handle error...
}

vmaEndDefragmentation(allocator, defragCtx, nullptr);

Although functions like vmaCreateBuffer(), vmaCreateImage(), vmaDestroyBuffer(), vmaDestroyImage() create/destroy an allocation and a buffer/image at once, these are just a shortcut for creating the resource, allocating memory, and binding them together. Defragmentation works on memory allocations only. You must handle the rest manually. Defragmentation is an iterative process that should repreat "passes" as long as related functions return VK_INCOMPLETE not VK_SUCCESS. In each pass:

1. vmaBeginDefragmentationPass() function call:
   • Calculates and returns the list of allocations to be moved in this pass. Note this can be a time-consuming process.
   • Reserves destination memory for them by creating temporary destination allocations that you can query for their VkDeviceMemory + offset using vmaGetAllocationInfo().
2. Inside the pass, you should:
   • Inspect the returned list of allocations to be moved.
   • Create new buffers/images and bind them at the returned destination temporary allocations.
   • Copy data from source to destination resources if necessary.
   • Destroy the source buffers/images, but NOT their allocations.
3. vmaEndDefragmentationPass() function call:
   • Frees the source memory reserved for the allocations that are moved.
   • Modifies source VmaAllocation objects that are moved to point to the destination reserved memory.
   • Frees VkDeviceMemory blocks that became empty.

Unlike in previous iterations of the defragmentation API, there is no list of "movable" allocations passed as a parameter. Defragmentation algorithm tries to move all suitable allocations. You can, however, refuse to move some of them inside a defragmentation pass, by setting pass.pMoves[i].operation to VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE. This is not recommended and may result in suboptimal packing of the allocations after defragmentation. If you cannot ensure any allocation can be moved, it is better to keep movable allocations separate in a custom pool.

Inside a pass, for each allocation that should be moved:

• You should copy its data from the source to the destination place by calling e.g. vkCmdCopyBuffer(), vkCmdCopyImage().
  • You need to make sure these commands finished executing before destroying the source buffers/images and before calling vmaEndDefragmentationPass().
• If a resource doesn't contain any meaningful data, e.g. it is a transient color attachment image to be cleared, filled, and used temporarily in each rendering frame, you can just recreate this image without copying its data.
• If the resource is in HOST_VISIBLE and HOST_CACHED memory, you can copy its data on the CPU using memcpy().
• If you cannot move the allocation, you can set pass.pMoves[i].operation to VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE. This will cancel the move.
  • vmaEndDefragmentationPass() will then free the destination memory not the source memory of the allocation, leaving it unchanged.
• If you decide the allocation is unimportant and can be destroyed instead of moved (e.g. it wasn't used for long time), you can set pass.pMoves[i].operation to VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY.
  • vmaEndDefragmentationPass() will then free both source and destination memory, and will destroy the source VmaAllocation object.

You can defragment a specific custom pool by setting VmaDefragmentationInfo::pool (like in the example above) or all the default pools by setting this member to null.

Defragmentation is always performed in each pool separately. Allocations are never moved between different Vulkan memory types. The size of the destination memory reserved for a moved allocation is the same as the original one. Alignment of an allocation as it was determined using vkGetBufferMemoryRequirements() etc. is also respected after defragmentation. Buffers/images should be recreated with the same VkBufferCreateInfo / VkImageCreateInfo parameters as the original ones.

You can perform the defragmentation incrementally to limit the number of allocations and bytes to be moved in each pass, e.g. to call it in sync with render frames and not to experience too big hitches. See members: VmaDefragmentationInfo::maxBytesPerPass, VmaDefragmentationInfo::maxAllocationsPerPass.

It is also safe to perform the defragmentation asynchronously to render frames and other Vulkan and VMA usage, possibly from multiple threads, with the exception that allocations returned in VmaDefragmentationPassMoveInfo::pMoves shouldn't be destroyed until the defragmentation pass is ended.

Mapping is preserved on allocations that are moved during defragmentation. Whether through VMA_ALLOCATION_CREATE_MAPPED_BIT or vmaMapMemory(), the allocations are mapped at their new place. Of course, pointer to the mapped data changes, so it needs to be queried using VmaAllocationInfo::pMappedData.

Note

Defragmentation is not supported in custom pools created with VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT.