Scope of rspirv

[Yamakaky]

For cendre, I'll have to execute the shaders in SPIR-V. Do you want to do something in common or is it out of the scope of what you want to do?
One possibility would be to convert SPIR-V to LLVM IR, run a JIT compiler then execute it. Maybe with a wrapper main which takes the uniforms, inputs, outputs... and calls the real shader main.

[antiagainst]

No, evaluating/running the SPIR-V module is not on my plate in the foreseeable future. :) I'm basically only interested at the IL level (SPIR-V and its manipulation, optimization, validation, etc.) and up (frond-end language GLSL/HLSL compilation to SPIR-V).

Evaluating/running SPIR-V modules requires Vulkan environment set up. That's a whole lot of new stuff to handle (managing resources, etc.). vulkano comes into the picture here but it's real GPU backed; I cannot only run a single shader for, like, geometry stage. You'll have to set up the whole pipeline. A software implementation of Vulkan is more handy for this case. So cendre is naturally more aligned to that purpose.

But I'd definitely like to help on my rspirv side if you'd like some new features. :)

[Yamakaky]

Looking from http://llvm.org/docs/tutorial/BuildingAJIT1.html, writing a JIT compiler seems manageable. I'll have to write some C++ I think, though. The more difficult part would be to convert SPIR-V to LLVM IT, I think. https://github.com/KhronosGroup/SPIRV-LLVM seems to be able to do that, so it could be included in this crate ;)

[antiagainst]

Ah, I didn't see your edited text previously. Sorry.

Now that sounds interesting to me. But going through SPIRV-LLVM may not be the way I'd prefer. As indicted in KhronosGroup/SPIRV-LLVM#202, SPIRV-LLVM makes assumptions of the OpenGL environment, and it is a little bit difficult to add Vulkan backend to it. Actually I do intend to write a SPIR-V to LLVM translator from the very beginning of this project (see README.md). And there are several llvm library bindings existing on creates.io. I just need to traverse the memory representation defined in this crate and call into llvm builders in the Rust bindings and generate LLVM IRs. Seems doable. :) (Yep, I know there will be lots of obstacles along the way, e.g., structured control flow, storage class, etc.) I'll take on it then!

[Yamakaky]

Sorry, wrong phrasing: I didn't meant to include spirv-llvm in rspirv, I meant to add a spirv-llvm IR converter to rspirv ^^
I'm motivated to work on it, since I'll have to do it anyway for cendre, so we could do it together.

[antiagainst]

Yep, definitely!

[Yamakaky]

As I see it, we can either do a direct mr::Module -> LLVM IR transformation, or first transform mr::Module to a tree IR and then convert it to LLVM's one. What do you prefer?

[antiagainst]

@Yamakaky: sorry for the late reply. I think first translating the current representation into a new structured IR would be the preferred way. The current memory representation is essentially just a data representation, without any interconnections between various instructions. It is not convenient to use for future tasks like validation (cendre would also need this to sanitize the input I think), optimization, etc. I've been wanting to add such a new representation for translating front end language, too. So multiple future tasks would benefit from it. It would be nice to go that way.

I'll write up my thinking about the new structured representation and also how to translate various SPIR-V instructions in LLVM in two new issues and we can discuss there. :)

[Yamakaky]

In fact, there are a lot of things I don't have to check. See https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#shader-modules, "Valid Usage" section. There are layers (provided by khronos) that verify the correct usage of the API, including SPIR-V shaders.
I agree for the rest.

[Yamakaky]

I'm starting with the types.

[antiagainst]

@Yamakaky: right, the driver is allowed to crash on spotting invalid usages of the API and also SPIR-V. However I would think if we can do more than just crashing in cendre it would be quite helpful for developers. A software implementation of Vulkan is more flexible; if it can print more diagnostics and pinpoint exactly what is wrong with my shaders that would be awesome for me as a developer.

[Yamakaky]

The driver don't have to verify it because you can enable validation layers during testing. See https://github.com/KhronosGroup/Vulkan-LoaderAndValidationLayers/blob/master/loader/LoaderAndLayerInterface.md#layer-library-interface.

[antiagainst]

@Yamakaky: Yep, you are right.

However, current existing validation layers are basically for catching API usage errors. SPIR-V validation is mainly done via spirv-val in SPIRV-Tools. AFAIK, there is no validation layer wrapping around spirv-val to provide SPIR-V validation to Vulkan devs yet.

My previous suggestion was to integrate the SPIR-V validation inside cendre, which could provide more detailed useful information when something goes wrong. (And that can be turned on with some options maybe.) But on a second thought, this indeed goes against Vulkan's layered design. And SPIR-V validation should be no difference with API usage validation. It should be outsourced to its dedicated layer. :)

Another benefit that can be provided by cendre but not from validation layers may be inspecting internal running status of SPIR-V (e.g., to know the values of various variables in my SPIR-V at a certain point of execution) for debugging purposes. Since that requires dynamic information.

Forgive me for throwing random thoughts on how cendre should evolve. I just feel that it has great potential to improve dev experiences from various points. :)

[Yamakaky]

Fun fact: we also can do a validation layer around rspirv ;) Like the object lifetime layer, it could keep track of dynamic information. Indeed, that would be a nice addition to the vulkan dev toolkit.

Please continue suggesting thinks for cendre, including issues.