Hi ! This article will deal with the memory management in Vulkan. But first, I am going to tell you what happened in my life.
State of my life
Again, it has been more than one month I did not write anything. So, where am I? I am in the last year of Télécom SudParis. I am following High Tech Imaging courses. It is the image specialization in my school. The funny part of it is : in parallel, I am a lecturer in a video games specialization. I taught OpenGL (3.3 because I cannot make an OpenGL 4 courses (everyone does not have a good hardware for that)). I got an internship in Dassault Systemes (France). It will begin the first February. I will work on the soft shadow engine (OpenGL 4.5).
Vulkan
To begin, some articles that I wrote before this one can contain mistakes, or some things are not well explained, or not very optimized.
Why came back to Vulkan?
I came back to Vulkan because I wanted to make one of the first “amateur” renderer using Vulkan. Also, I wanted to have a better improvement of memory management, memory barrier and other joys like that. Moreover, I made a repository with “a lot” of Vulkan Example : Vulkan example repository.
I did not mean to replace the Sascha Willems ones. But I propose my way to do it, in C++, using Vulkan HPP.
Memory Management with Vulkan
Different kind of memory
Heap
One graphic card can read memory from different heap. It can read memory from its own heap, or the system heap (RAM).
Type
It exists a different kind of memory type. For example, it exists memories that are host cached, or host coherent, or device local and other.
Host and device
Host
This memory resides in the RAM. This heap should have generally one (or several) type that own the bit “HOST_VISIBLE”. It means to Vulkan that it could be mapped persistently. Going that way, you get the pointer and you can write from the CPU on it.
Device Local
This memory resides on the graphic card. It is freaking fast and is not generally host_visible. That means you have to use a staging resource to write something to it or use the GPU itself.
Allocation in Vulkan
In Vulkan, the number of allocation per heap is driver limited. That means you can not do a lot of allocation and you must not use one allocation by buffer or image but one allocation for several buffers and images.
In this article, I will not take care about the CPU cache or anything like that, I will only focus my explanations on how to have the better from the GPU-side.
How will we do it?
As you can see, we have a block, that could represent the memory for one buffer, or for one image, we have a chunk that represents one allocation (via vkAllocateMemory) and we have a DeviceAllocator that manages all chunks.
A block, as it is named, defines a little region within one allocation.
So, it has an offset, one size, and a boolean to know if it is used or not.
It may own a ptr if it is an
Chunk
A chunk is a memory region that contains a list of blocks. It represents a single allocation.
What a chunk could let us to do?
Since a deallocation is really easy (only to put the block to free), one allocation requires a bit of attention. You need to check if the block is free, and if it is free, you need to check for its size, and, if necessary, create another block if the size of the allocation is less than the available size. You also need take care about memory alignment !
void Chunk::deallocate(const Block &block) {
auto blockIt(std::find(mBlocks.begin(), mBlocks.end(), block));
assert(blockIt != mBlocks.end());
// Just put the block to free
blockIt->free = true;
}
bool Chunk::allocate(vk::DeviceSize size, vk::DeviceSize alignment, Block &block) {
// if chunk is too small
if(size > mSize)
return false;
for(uint32_t i = 0; i < mBlocks.size(); ++i) {
if(mBlocks[i].free) {
// Compute virtual size after taking care about offsetAlignment
uint32_t newSize = mBlocks[i].size;
if(mBlocks[i].offset % alignment != 0)
newSize -= alignment - mBlocks[i].offset % alignment;
// If match
if(newSize >= size) {
// We compute offset and size that care about alignment (for this Block)
mBlocks[i].size = newSize;
if(mBlocks[i].offset % alignment != 0)
mBlocks[i].offset += alignment - mBlocks[i].offset % alignment;
// Compute the ptr address
if(mPtr != nullptr)
mBlocks[i].ptr = (char*)mPtr + mBlocks[i].offset;
// if perfect match
if(mBlocks[i].size == size) {
mBlocks[i].free = false;
block = mBlocks[i];
return true;
}
Block nextBlock;
nextBlock.free = true;
nextBlock.offset = mBlocks[i].offset + size;
nextBlock.memory = mMemory;
nextBlock.size = mBlocks[i].size - size;
mBlocks.emplace_back(nextBlock); // We add the newBlock
mBlocks[i].size = size;
mBlocks[i].free = false;
block = mBlocks[i];
return true;
}
}
}
return false;
}
Chunk Allocator
Maybe it is bad-named, but the chunk allocator let us to separate the creation of one chunk from the chunk itself. We give it one size and it operates all the verifications we need.
class ChunkAllocator : private NotCopyable
{
public:
ChunkAllocator(Device &device, vk::DeviceSize size);
// if size > mSize, allocate to the next power of 2
std::unique_ptr<Chunk> allocate(vk::DeviceSize size, int memoryTypeIndex);
private:
Device mDevice;
vk::DeviceSize mSize;
};
vk::DeviceSize nextPowerOfTwo(vk::DeviceSize size) {
vk::DeviceSize power = (vk::DeviceSize)std::log2l(size) + 1;
return (vk::DeviceSize)1 << power;
}
bool isPowerOfTwo(vk::DeviceSize size) {
vk::DeviceSize mask = 0;
vk::DeviceSize power = (vk::DeviceSize)std::log2l(size);
for(vk::DeviceSize i = 0; i < power; ++i)
mask += (vk::DeviceSize)1 << i;
return !(size & mask);
}
ChunkAllocator::ChunkAllocator(Device &device, vk::DeviceSize size) :
mDevice(device),
mSize(size) {
assert(isPowerOfTwo(size));
}
std::unique_ptr<Chunk> ChunkAllocator::allocate(vk::DeviceSize size,
int memoryTypeIndex) {
size = (size > mSize) ? nextPowerOfTwo(size) : mSize;
return std::make_unique<Chunk>(mDevice, size, memoryTypeIndex);
}
Device Allocator
I began to make an abstract class for Vulkan allocation :
/**
* @brief The AbstractAllocator Let the user to allocate or deallocate some blocks
*/
class AbstractAllocator : private NotCopyable
{
public:
AbstractAllocator(Device const &device) :
mDevice(std::make_shared<Device>(device)) {
}
virtual Block allocate(vk::DeviceSize size, vk::DeviceSize alignment, int memoryTypeIndex) = 0;
virtual void deallocate(Block &block) = 0;
Device getDevice() const {
return *mDevice;
}
virtual ~AbstractAllocator() = 0;
protected:
std::shared_ptr<Device> mDevice;
};
inline AbstractAllocator::~AbstractAllocator() {
}
As you noticed, it is really easy. You can allocate or deallocate from this allocator. Next, I created a DeviceAllocator that inherits from AbstractAllocator.
class DeviceAllocator : public AbstractAllocator
{
public:
DeviceAllocator(Device device, vk::DeviceSize size);
Block allocate(vk::DeviceSize size, vk::DeviceSize alignment, int memoryTypeIndex);
void deallocate(Block &block);
private:
ChunkAllocator mChunkAllocator;
std::vector<std::shared_ptr<Chunk>> mChunks;
};
This allocator contains a list of chunks, and contains one ChunkAllocator to allocate chunks.
The allocation is really easy. We have to check if it exists a “good chunk” and if we can allocate from it. Otherwise, we create another chunk and it is over !
Since I came back to Vulkan, I really had a better understanding of this new API. I can write article in better quality than in march.
I hope you enjoyed this remake of memory management.
My next article will be about buffer, and staging resource. It will be a little article. I will write as well an article that explains how to load textures and their mipmaps.
Hi!
Once again, I am going to present you some vulkan features, like pipelines, barriers, memory management, and all things useful for prior ones. This article will be long, but it will be separating into several chapters.
Memory Management
In Vulkan application, it is up to the developer to manage himself the memory. The number of allocations is limited. Make one allocation for one buffer, or one image is really a bad design in Vulkan. One good design is to make a big allocation (let’s call that a chunk), and manage it yourself, and allocate buffer or image within the chunk.
A Chunk Allocator
We need a simple object which has responsibility for allocations of chunks. It just has to select the good heap and call allocate and free from Vulkan API.
#include "chunkallocator.hpp"
#include "System/exception.hpp"
ChunkAllocator::ChunkAllocator(Device &device) : mDevice(device)
{
}
std::tuple<VkDeviceMemory, VkMemoryPropertyFlags, VkDeviceSize, char*>
ChunkAllocator::allocate(VkMemoryPropertyFlags flags, VkDeviceSize size) {
VkPhysicalDeviceMemoryProperties const &property = mDevice.memoryProperties();
int index = -1;
// Looking for a heap with good flags and good size
for(auto i(0u); i < property.memoryTypeCount; ++i)
if((property.memoryTypes[i].propertyFlags & flags) == flags)
if(size < property.memoryHeaps[property.memoryTypes[i].heapIndex].size)
index = i;
if(index == -1)
throw std::runtime_error("No good heap found");
VkMemoryAllocateInfo info = {};
info.sType = VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO;
info.pNext = nullptr;
info.allocationSize = size;
info.memoryTypeIndex = index;
// Perform the allocation
VkDeviceMemory mem;
vulkanCheckError(vkAllocateMemory(mDevice, &info, nullptr, &mem));
mDeviceMemories.push_back(mem);
char *ptr;
// We map the memory if it is host visible
if(flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT)
vulkanCheckError(vkMapMemory(mDevice, mem, 0, VK_WHOLE_SIZE, 0, (void**)&ptr));
return std::tuple<VkDeviceMemory, VkMemoryPropertyFlags, VkDeviceSize, char*>
(mem, flags, size, ptr);
}
ChunkAllocator::~ChunkAllocator() {
// We free all memory objects
for(auto &mem : mDeviceMemories)
vkFreeMemory(mDevice, mem, nullptr);
}
This piece of code is quite simple and easy to read.
Memory Pool
Memory pools are structures used to optimize dynamic allocation performances. In video games, it is not an option to use a memory pool. Ideas are the same I told in the first part. Allocate a chunk, and sub allocate yourself within the chunk. I made a simple generic memory pool.
There is a little scheme which explains what I wanted to do.
As you can see, video memory is separated into several parts (4 here) and each “Block” in the linked list describes one sub-allocation.
One block is described by :
Size of the block
Offset of the block relatively with the DeviceMemory
A pointer to set data from the host (map)
Boolean to know about the freeness of the block
A sub-allocation within a chunk is performed as follows :
Traverse the linked list until we find a well-sized free block
Modify the size and set the boolean to false
Create a new block, set size, offset and put boolean to true and insert it after the current one.
A free is quite simple, you just have to put the boolean to true.
A good other method could be a “shrink to fit”. If there are some following others with the boolean set to true, we merge all blocks into one.
#include "memorypool.hpp"
#include <cassert>
MemoryPool::MemoryPool(Device &device) :
mDevice(device), mChunkAllocator(device) {}
Allocation MemoryPool::allocate(VkDeviceSize size, VkMemoryPropertyFlags flags) {
if(size % 128 != 0)
size = size + (128 - (size % 128)); // 128 bytes alignment
assert(size % 128 == 0);
for(auto &chunk: mChunks) {
// if flags are okay
if((chunk.flags & flags) == flags) {
int indexBlock = -1;
// We are looking for a good block
for(auto i(0u); i < chunk.blocks.size(); ++i) {
if(chunk.blocks[i].isFree) {
if(chunk.blocks[i].size > size) {
indexBlock = i;
break;
}
}
}
// If a block is find
if(indexBlock != -1) {
Block newBlock;
// Set the new block
newBlock.isFree = true;
newBlock.offset = chunk.blocks[indexBlock].offset + size;
newBlock.size = chunk.blocks[indexBlock].size - size;
newBlock.ptr = chunk.blocks[indexBlock].ptr + size;
// Modify the current block
chunk.blocks[indexBlock].isFree = false;
chunk.blocks[indexBlock].size = size;
// If allocation does not fit perfectly the block
if(newBlock.size != 0)
chunk.blocks.emplace(chunk.blocks.begin() + indexBlock + 1, newBlock);
return Allocation(chunk.memory, chunk.blocks[indexBlock].offset, size, chunk.blocks[indexBlock].ptr);
}
}
}
// if we reach there, we have to allocate a new chunk
addChunk(mChunkAllocator.allocate(flags, 1 << 25));
return allocate(size, flags);
}
void MemoryPool::free(Allocation const &alloc) {
for(auto &chunk: mChunks)
if(chunk.memory == std::get<0>(alloc)) // Search the good memory device
for(auto &block : chunk.blocks)
if(block.offset == std::get<1>(alloc)) // Search the good offset
block.isFree = true; // put it to free
}
void MemoryPool::addChunk(const std::tuple<VkDeviceMemory, VkMemoryPropertyFlags, VkDeviceSize, char *> &ptr) {
Chunk chunk;
Block block;
// Add a block mapped along the whole chunk
block.isFree = true;
block.offset = 0;
block.size = std::get<2>(ptr);
block.ptr = std::get<3>(ptr);
chunk.flags = std::get<1>(ptr);
chunk.memory = std::get<0>(ptr);
chunk.size = std::get<2>(ptr);
chunk.ptr = std::get<3>(ptr);
chunk.blocks.emplace_back(block);
mChunks.emplace_back(chunk);
}
Buffers
Buffers are a well-known part in OpenGL. In Vulkan, it is approximately the same, but you have to manage yourself the memory through one memory pool.
When you create one buffer, you have to give him a size, an usage (uniform buffer, index buffer, vertex buffer, …). You also could ask for a sparse buffer (Sparse resources will be a subject of an article one day ^_^). You also could tell him to be in a mode concurrent. Thanks to that, you could access the same buffer through two different queues.
I chose to have a host visible and host coherent memory. But it is not especially useful. Indeed, to achieve a better performance, you could want to use a non coherent memory (but you will have to flush/invalidate your memory!!).
For the host visible memory, it is not especially useful as well, indeed, for indirect rendering, it could be smart to perform culling with the GPU to fill all structures!
Shaders
Shaders are Different parts of your pipelines. It is an approximation obviously. But, for each part (vertex processing, geometry processing, fragment processing…), shader associated is invoked. In Vulkan, shaders are wrote with SPIR-V.
SPIR-V is “.class” are for Java. You may compile your GLSL sources to SPIR-V using glslangvalidator.
Why is SPIR-V so powerful ?
SPIR-V allows developers to provide their application without the shader’s source.
SPIR-V is an intermediate representation. Thanks to that, vendor implementation does not have to write a specific language compiler. It results in a lower complexity for the driver and it could more optimize, and compile it faster.
Shaders in Vulkan
Contrary to OpenGL’s shader, it is really easy to compile in Vulkan.
My implementation keeps in memory all shaders into a hashtable. It lets to prevent any shader’s recompilation.
Pipelines are objects used for dispatch (compute pipelines) or render something (graphic pipelines).
The beginning of this part is going to be a summarize of the Vulkan’s specs.
Descriptors
Shaders access buffer and image resources through special variables. These variables are organized into a set of bindings. One set is described by one descriptor.
Descriptor Set Layout
They describe one set. One set is compound with an array of bindings. Each bindings are described by :
A binding number
One type : Image, uniform buffer, SSBO, …
The number of values (Could be an array of textures)
Stage where shader could access the binding.
Allocation of Descriptor Sets
They are allocated from descriptor pool objects.
One descriptor pool object is described by a number of set allocation possible, and an array of descriptor type / count it can allocate.
Once you have the descriptor pool, you could allocate from it sets (using both descriptor pool and descriptor set layout).
When you destroy the pool, sets also are destroyed.
Give buffer / image to sets
Now, we have descriptors, but we have to tell Vulkan where shaders can get data from.
Pipeline Layouts
Pipeline layouts are a kind of bridge between the pipeline and descriptor sets. They let you manage push constant as well (we’ll see them in a future article).
Implementation
Since descriptor sets are not coupled with pipelines layout. We could separate pipeline layout and descriptor pool / sets, but currently, I prefer to keep them coupled. It is a choice, and it will maybe change in the future.
I am going to explain quickly what memory barriers are.
The idea behind the memory barrier is ensured writes are performed.
When you performed one compute or one render, it is your duty to ensure that data will be visible when you want to re-use them.
In our main.cpp example, I draw a triangle into a frame buffer and present it.
Image barriers are compound with access, layout, and pipeline barrier with stage.
Since the presentation is a read of a framebuffer, srcAccessMask is VK_ACCESS_MEMORY_READ_BIT.
Now, we want to render inside this image via a framebuffer, so dstAccessMask is VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT.
We were presented the image, and now we want to render inside it, so, layouts are obvious.
When we submit image memory barrier to the command buffer, we have to tell it which stages are affected. Here, we wait for all commands and we begin for the first stage of the pipeline.
The only difference is the order and stageMasks. Here we wait for the color attachement (and not the Fragment one !!!!) and we begin with the end of the stages (It is not really easy to explain… but it does not sound not logic).
Steps to render something using pipelines are:
Create pipelines
Create command pools, command buffer and begin them
Create vertex / index buffers
Bind pipelines to their subpass, bind buffers and descriptor sets
Hi there !
A Few weeks ago, February 16th to be precise, Vulkan, the new graphic API from Khronos was released. It is a new API which gives much more control about the GPUs than OpenGL (API I loved before Vulkan ^_^).
OpenGL’s problems
Driver Overhead
Fast rendering problems could be from the driver, video games don’t use perfectly the GPU (maybe 80% instead of 95-100% of use). Driver overheads have big costs and more recent OpenGL version tend to solve this problem with Bindless Textures, multi draws, direct state access, etc.
Keep in mind that each GPU calls could have a big cost.
Cass Everitt, Tim Foley, John McDonald, Graham Sellers presented Approaching Zero Driver Overhead with OpenGL in 2014.
Multi threading
With OpenGL, it is not possible to have an efficient multi threading, because an OpenGL context is for one and only one thread that is why it is not so easy to make a draw call from another thread ^_^.
Vulkan
Vulkan is not really a low level API, but it provides a far better abstraction for moderns hardwares. Vulkan is more than AZDO, it is, as Graham Sellers said, PDCTZO (Pretty Darn Close To Zero Overhead).
Series of articles about Lava
What is Lava ?
Lava is the name I gave to my new graphic (physics?) engine. It will let me learn how Vulkan work, play with it, implement some global illumination algorithms, and probably share with you my learnings and feelings about Vulkan. It is possible that I’ll make some mistakes, so, If I do, please let me know !
Why Lava ?
Vulkan makes me think about Volcano that make me think about Lava, so… I chose it 😀 .
Initialization
Now begins what I wanted to discuss, initialization of Vulkan.
First of all, you have to really know and understand what you will attend to do. For the beginning, we are going to see how to have a simple pink window.
When you are developing with Vulkan, I advise you to have specifications from Khronos on another window (or screen if you are using multiple screens).
To have an easier way to manage windows, I am using GLFW 3.2, and yes, you are mandatory to compile it yourself ^_^, but it is not difficult at all, so it is not a big deal.
Instance
Contrary to OpenGL, in Vulkan, there is no global state, an instance could be similar to an OpenGL Context. An instance doesn’t know anything about other instances, is utterly isolate. The creation of an instance is really easy.
From this Instance, you could retrieve all GPUs on your computer.
You could create a connection between your application and the GPU you want using a VkDevice.
Creating this connection, you have to create as well queues.
Queues are used to perform tasks, you submit the task to a queue and it will be performed.
The queues are separated between several families.
A good way could be use several queues, for example, one for the physics and one for the graphics (or even 2 or three for this last).
You could as well give a priority (between 0 and 1) to a queue. Thanks to that, if you consider a task not so important, you just have to give to the used queue a low priority :).
The images represent a mono or multi dimensional array (1D, 2D or 3D).
The images don’t give any get or set for data. If you want to use them in your application, then you must use ImageViews.
ImageViews are directly relied to an image. The creation of an ImageView is not really complicated.
A window is assigned to a Surface (VkSurfaceKHR). To draw something, you have to render into this surface via swapchains.
From notions of Swapchains
In Vulkan, you have to manage the double buffering by yourself via Swapchain. When you create a swapchain, you link it to a Surface and tell it how many images you need. For a double buffering, you need 2 images.
Once the swapchain was created, you should retrieve images and create frame buffers using them.
The steps to have a correct swapchain is :
Create a Window
Create a Surface assigned to this Window
Create a Swapchain with several images assigned to this Surface
void SurfaceWindow::begin() {
// No checking because could be in lost state if change res
vkAcquireNextImageKHR(mDevice, mSwapchain, UINT64_MAX, VK_NULL_HANDLE, VK_NULL_HANDLE, &mCurrentSwapImage);
}
void SurfaceWindow::end(Queue &queue) {
VkPresentInfoKHR info;
info.sType = VK_STRUCTURE_TYPE_PRESENT_INFO_KHR;
info.pNext = nullptr;
info.waitSemaphoreCount = 0;
info.pWaitSemaphores = nullptr;
info.swapchainCount = 1;
info.pSwapchains = &mSwapchain;
info.pImageIndices = &mCurrentSwapImage;
info.pResults = nullptr;
vkQueuePresentKHR(queue, &info);
}
To notions of Render Pass
Right now, Vulkan should be initialized. To render something, we have to use render pass, and command buffer.
Command Buffers
Command buffer is quite similar to vertex array object (VAO) or display list (old old old OpenGL 😀 ).
You begin the recorded state, you record some “information” and you end the recorded state.
Command buffers are allocated from the CommandPool.
Vulkan provides two types of Command Buffer.
Primary level : They should be submitted within a queue.
Secondary level : They should be executed by a primary level command buffer.
One render pass is executed on one framebuffer. The creation is not easy at all. One render pass is componed with one or several subpasses.
I remind that framebuffers could have several attachments.
Each attachment are not mandatory to be used for all subpasses.
This piece of code to create one renderpass is not definitive at all and will be changed as soon as possible ^^. But for our example, it is correct.