Add Triton language

[siboehm]

This closes #5530.

From my POV, this is what's still TODO (would be happy about any input on these):

1. Add Triton installation (from source) to CE infra repo.
2. Adjust the path to Triton's compile.py script. It's hardcoded to work on my local machine right now.
3. Make sure the nvdisasm path in properties is correct. I was following nvcc, but not sure I got it right.
4. Is there any way to set default compiler flags for the default example.py? It will not compile without setting -n kernel --signature "*fp32:16, *fp16:16, *fp16:16, i32, 1024" -w 1 -g "M/16, 1, 1". That's not mandatory I guess and users would figure it out eventually from the error messages, but it'd make the experience a little nicer.

Lmk if you want me to move the doc stuff into a separate PR.

[siboehm]

Adding Triton properly requires conda env support on CE infra. Matt mentioned he'd been meaning to add it, but this PR will probably be blocked until then.

If anyone wants to try this locally:

1. Create conda env with torch + python + triton>=2.1
2. adjust triton.defaults.properties to make sure python / nvdiasms / triton compile script path is correct
3. make dev EXTRA_ARGS="--debug --language triton"

[bwasti]

should be /home/ce?

[mattgodbolt]

Defaults are whatever make sense as a default. I'm not any /home/ is a sensible default, but I'm also not sure what else to suggest here. Typically we refer in the defaults to things like /usr/bin/gcc etc, where compilers are (probably) installed. It's expected users will override in their *.local.properties, and the triton.amazon.properties will be completely different (referring to something in /opt/compiler-explorer/... somewhere)

[siboehm]

I picked this default since I was hoping it makes it very obvious what it should be set to in the *.local.properties. I don't think it's common to have a package like Triton be installed globally using the system Python, but happy to take suggestions.

[mattgodbolt]

I'd go with just triton or leave it unset entirely to make it error. But not a big deal really.

[siboehm]

Ok I dug a bit deeper into the Triton codebase, and the "jit-ness" is really ingrained, in the sense that turning Triton into a cross-compiler is a fairly big refactoring, because it issues calls like torch.cuda.get_device_info() all over the place to retrieve things like SMEM size, even when running in AOT mode.

Running CE locally will still be pretty useful to many (like me) for figuring out what Triton is doing. I don't know how others feel about having compilers in the CE main repo that only run locally.

The other option would be to make sure Triton only ever runs on the GPU CE instances, and somehow make it clear to users that it cannot cross-compile and hence always targets the native GPU, and that they have to run a local instance themselves to fix that.

[mattgodbolt]

From Discord, worth looking in to https://github.com/conda/conda-pack (noted by @siboehm as a potential help for installation)

[partouf]

I think we can merge this PR as is, as long as it doesn't break anything. It won't add the language to the website, but people will still be able to try it out.