Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It's impossible to bind to a host memory which is not coherent. #85

Open
nikitablack opened this issue Dec 14, 2021 · 5 comments
Open

Comments

@nikitablack
Copy link

Hi.

Maybe I'm wrong but when I look into the Allocator implementation I see the following in the Allocator::allocate method:

  • If MemoryLocation::CpuToGpu was specified the function tries to find a memory type index for the preferred bits vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT | vk::MemoryPropertyFlags::HOST_CACHED.
  • If the previous attempt failed the function tries to find a memory type index for the required bits vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT.
  • If the second attempt fails the function returns Err(AllocationError::NoCompatibleMemoryTypeFound).

So there's no chance to allocate a host visible, non-coherent memory.

Also, in Allocator::new, there's no need to check for host_visible_not_coherent. From the Spec 11.2.1. Device Memory Properties:

There must be at least one memory type with both the VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT and VK_MEMORY_PROPERTY_HOST_COHERENT_BIT bits set in its propertyFlags.

@MarijnS95
Copy link
Member

MarijnS95 commented Dec 14, 2021

So there's no chance to allocate a host visible, non-coherent memory.

As far as I'm aware the main reason is that this requires users of the gpu-allocator crate to be aware that they must explicitly call vkFlushMappedMemoryRanges after writing to the mapped range from the host, whereas it is implicit for coherent memory. As you correctly quote below, such a memory type is guaranteed to exist.

Also, in Allocator::new, there's no need to check for host_visible_not_coherent. From the Spec 11.2.1. Device Memory Properties:

There must be at least one memory type with both the VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT and VK_MEMORY_PROPERTY_HOST_COHERENT_BIT bits set in its propertyFlags.

That spec rule does not disallow a visible, non-coherent memory type to exist (this quote seems to at least partially answer your first question). In any case host_visible_not_coherent is used for nothing more than warn!ing that such a memory type is available (in addition to the mandatory HOST_VISIBLE | HOST_COHERENT). I presume the justification for using such a memory type over coherent memory is a possible performance advantage when flushing writes explicitly at one specific point in time through vkFlushMappedMemoryRanges.

@nikitablack
Copy link
Author

Well, as I learned today, the existence of host coherent memory doesn't guarantee that it's possible to bind to such memory. In other words, it's not enough to check that the memory exists and MemoryLocation::CpuToGpu shouldn't imply that the flags are supported for every memory requirement. Currently, the function returns an error, but wouldn't it be better to allocate and bind to a host visible non-coherent memory and let the user flush manually.

@manon-traverse
Copy link
Collaborator

@nikitablack That's interesting. We always assumed that since there must always be a host visible and coherent memory type we could rely on using that. With not being possible to bind, do you mean you can´t bind it to a buffer or image object? That would be quite strange, since it would kind of defeat the purpose of being able to allocate that memory.

@nikitablack
Copy link
Author

nikitablack commented Dec 15, 2021

I was also sure that having a host coherent memory means I can safely use it for any type of object. It's a host memory after all, I can place whatever I want there. But in a recent discussion on a channel, a person showed that he was not able to find a memory type index for an image object with VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT flags. After some discussion, I was convinced that the Spec doesn't guarantee that having a certain memory type means it can be used for every purpose. His code was quite special because he tried to find memory for an image with optimal tiling. And now I'm not 100% sure if it a combination of flags that failed or maybe a single VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT is enough to fail because there's not much sense to have an image with such tiling property on the host (because a GPU can use a proprietary format for performance reasons).

But the point here is that the Spec doesn't give any guarantees in this regard. Unfortunately, I can't find the correct wording in the Spec.

Update: by not being able to bind I mean not being able to find an appropriate memory type index, sorry for the confusion.

@nikitablack
Copy link
Author

I found this in 12.7. Resource Memory Association:

If buffer is a VkBuffer not created with the VK_BUFFER_CREATE_SPARSE_BINDING_BIT bit set, or if image is linear image, then the memoryTypeBits member always contains at least one bit set corresponding to a VkMemoryType with a propertyFlags that has both the VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT bit and the VK_MEMORY_PROPERTY_HOST_COHERENT_BIT bit set. In other words, mappable coherent memory can always be attached to these objects.

As you can see, there's a limitation for certain kinds of objects. I understand that this would be an extremely specific use case and if you say that it's not worth changing the library because of features that nobody uses, it's perfectly understandable, feel free to close the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants