cdi generate cannot find config files in non-standard locations

[jocado]

Hi,

When generating CDI config, if the json config files for glvnd, vulkanoregl` are not in a "standard" location, the are not found. It fails as below:

# nvidia-ctk cdi generate --nvidia-ctk-path ${SNAP}/usr/bin/nvidia-ctk --library-search-path ${SNAP}/graphics/lib/x86_64-linux-gnu 2>&1 |grep json
time="2024-04-24T18:46:35Z" level=warning msg="Could not locate glvnd/egl_vendor.d/10_nvidia.json: pattern glvnd/egl_vendor.d/10_nvidia.json not found"
time="2024-04-24T18:46:35Z" level=warning msg="Could not locate vulkan/icd.d/nvidia_icd.json: pattern vulkan/icd.d/nvidia_icd.json not found"
time="2024-04-24T18:46:35Z" level=warning msg="Could not locate vulkan/icd.d/nvidia_layers.json: pattern vulkan/icd.d/nvidia_layers.json not found"
time="2024-04-24T18:46:35Z" level=warning msg="Could not locate vulkan/implicit_layer.d/nvidia_layers.json: pattern vulkan/implicit_layer.d/nvidia_layers.json not found"
time="2024-04-24T18:46:35Z" level=warning msg="Could not locate egl/egl_external_platform.d/15_nvidia_gbm.json: pattern egl/egl_external_platform.d/15_nvidia_gbm.json not found"
time="2024-04-24T18:46:35Z" level=warning msg="Could not locate egl/egl_external_platform.d/10_nvidia_wayland.json: pattern egl/egl_external_platform.d/10_nvidia_wayland.json not found"

It's possible to specify alternate search paths for libraries via --library-search-path. It would be awesome to be able to do the same for config file paths. This was discussed a bit over in #82, specifically #82 (comment) , but it didn't make it into release as there were some issues testing it at the time and there was no pressing need for the config files then either. I have now found some use cases for adding the config files via CDI, and it would be great to be able to do that.

I'm testing this for use in the docker snap packaging, where the non-kernel driver artefacts are in non-standard locations on Ubuntu Core systems. This should further expand the uses that can leverage nvidia hardware via docker.

Currently running latest toolkit version, v1.15.0

If you need any further logs or info, please let me know.

FYI @elezar

Thank you.

[jocado]

Hi @elezar

Thanks for the pointer here #82 (comment)

Apologies, I hand't realised it was added in the end.

When I went back and checked the chagne logs, I noticed that there were two mentioned ways added to find config files:

• --config-search-path
• XDG_DATA_DIRS

I tried --config-search-path, but it doesn't seem to work. However, when I added the path via XDG_DATA_DIRS it did fine the configs. That's great, and solves my issue.

However, just to clear up the mystery of the differences between the two search options. Are they looking for different types of config files, and I'm misunderstanding the intention, or should they work largely the same and find the same config files ?

Here is the debug log of the attempt to use --config-search-path

# nvidia-ctk cdi generate --nvidia-ctk-path ${SNAP}/usr/bin/nvidia-ctk --library-search-path ${SNAP}/graphics/lib/x86_64-linux-gnu --config-search-path ${SNAP}/graphics/share 2>&1 
INFO[0000] Auto-detected mode as "nvml"                 
INFO[0000] Selecting /dev/nvidia0 as /dev/nvidia0       
INFO[0000] Selecting /dev/dri/card1 as /dev/dri/card1   
WARN[0000] Could not locate /dev/dri/controlD65: pattern /dev/dri/controlD65 not found 
INFO[0000] Selecting /dev/dri/renderD128 as /dev/dri/renderD128 
INFO[0000] Using driver version 525.147.05              
WARN[0000] failed to get custom firmware class path: open /sys/module/firmware_class/parameters/path: permission denied 
INFO[0000] Selecting /dev/nvidia-modeset as /dev/nvidia-modeset 
INFO[0000] Selecting /dev/nvidia-uvm-tools as /dev/nvidia-uvm-tools 
INFO[0000] Selecting /dev/nvidia-uvm as /dev/nvidia-uvm 
INFO[0000] Selecting /dev/nvidiactl as /dev/nvidiactl   
INFO[0000] Selecting /snap/docker/x7/graphics/lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1.1.0 as /snap/docker/x7/graphics/lib/x86_64-linux-gnu/libnvidia-egl-gbm.so.1.1.0 
WARN[0000] Could not locate glvnd/egl_vendor.d/10_nvidia.json: pattern glvnd/egl_vendor.d/10_nvidia.json not found 
WARN[0000] Could not locate vulkan/icd.d/nvidia_icd.json: pattern vulkan/icd.d/nvidia_icd.json not found 
WARN[0000] Could not locate vulkan/icd.d/nvidia_layers.json: pattern vulkan/icd.d/nvidia_layers.json not found 
WARN[0000] Could not locate vulkan/implicit_layer.d/nvidia_layers.json: pattern vulkan/implicit_layer.d/nvidia_layers.json not found 
WARN[0000] Could not locate egl/egl_external_platform.d/15_nvidia_gbm.json: pattern egl/egl_external_platform.d/15_nvidia_gbm.json not found 
WARN[0000] Could not locate egl/egl_external_platform.d/10_nvidia_wayland.json: pattern egl/egl_external_platform.d/10_nvidia_wayland.json not found 

Thanks!