MPV use of MoltenVK for Vulkan?

[billhollings]

I came across a recent press article that made me aware that MPV is using MoltenVK for Vulkan activity.

I'm curious what MPV is using Vulkan for. Other than the video extensions, which MoltenVK doesn't support (yet), what Vulkan activity does MPV engage with?

I'm also curious about whether MoltenVK is suiting your needs. Feel free to post any requirements to the MoltenVK issues list. That includes the Vulkan video extensions themselves. The more clarity we have on who is using those extensions, and for what, the better we can prioritize them.

[Akemi]

i actually added vulkan support, vial moltenvk and a metal layer, in 2020 as a work in progress PR but only recently (october 2023) got to merge it. the delay was mostly because of my private circumstances and the resulting lack of time. anyway, i had some problems first, which were fixed in the more recent versions of moltenvk (for some history see the mentioned PR and the comments). though i used and tested this backend since 2020.

in general, we are using vulkan (moltenvk) for our whole rendering pipeline. our original video output was opengl based and with that all our shaders. to utilise those opengl shaders we use libplacebo, and with that shaderc, to translate our shaders to spri-v and vulkan. moltenvk itself is used in our gpu(-next) video output in the macvk backend as the vulkan driver (loader).

what we do on rendering are things like, utilising scaling algorithms, hdr tone mapping, colour conversations/management, dithering, interpolation, debanding, custom shaders support, and more.

i still have some issues that come to mind that i kinda would like to be fixed, though i am not sure if those are on our end or moltenvk for some of them:

1. this workaround that sets the drawable size to 1x1 pixel, can cause various issues on our end like flickering, black screens and broken rendering. see also PR mac/vulkan: workaround for MoltenVK problem that causes flicker #13651 and commit for a workaround in mpv.
2. something i couldn't pin down yet with rendering timings on higher refresh rate displays (>=120hz). vsyncs are not as stable and it can jitter quite a bit, which leads to mistimed frames or completely dropped ones.
3. we have a crashing problem with colour space hinting (eg when moltenvk has to set the colorSpace on the metal layer and/or the EDR metadata) which only happens with moltenvk and none of the other drivers on other platforms. also not sure if this problem is on our end, some expectations are wrong, or something completely different. seems like a double free.
4. i still try to find out what the prerequisite are to 'activate' direct rendering and not composited , eg what the metal hud says when activating it via MTL_HUD_ENABLED=1.

sry for rambling a bit too much, if i got some terms or facts wrong, and/or simplified things too much. anyway if you have any more specific questions i will try to answer them as best as i can. for specific features and/or extensions that might be needed, @haasn who did the majority of libplacebo might be the better person to ask.

[billhollings]

Thanks for all the feedback.

As you get a better handle on the source of these issues, in some manner that we can understand them (replicate them, or can provide stack traces, etc), please post them to the MoltenVK issues list, so we can have a look at them.