Generate Types

[bconnorwhite]

This uses tree sitter to generate types for the Rust loader.

The nice thing about tree sitter is that there are parsers for most languages, so it should be fairly easy to add new type mappings.

Unfortunately, there's an open issue in Bun that prevents tree sitter from working with Bun, so I'm spawning a node process instead to do the actual parsing. This would be pretty easy to move back to Bun though once the issue is fixed.

---

Here's an example of how it works:

import { CString, JSCallback, ptr } from "bun:ffi";
import { add, call, echo, hello, mul, not, sub } from "./math.rs";

console.log(add(3, 2));
console.log(sub(3, 2));
console.log(mul(3.2, 2.1));
console.log(not(true));

hello();

echo(ptr(Buffer.from("hello from JS\0", "utf-8")));

const callback = new JSCallback(
  (ptr, length) => /Hello/.test(new CString(ptr, length)),
  {
    returns: "bool",
    args: ["ptr", "usize"]
  }
);

const result = call(callback);
console.log(result);

Rust:

use std::os::raw::c_char;

#[no_mangle]
pub extern "C" fn add(a: u32, b: u32) -> u32 {
    a + b
}

#[no_mangle]
pub extern "C" fn sub(a: i32, b: i32) -> i32 {
    a - b
}

#[no_mangle]
pub extern "C" fn mul(a: f32, b: f32) -> f32 {
    a * b
}

#[no_mangle]
pub extern "C" fn not(a: bool) -> bool {
    !a
}

#[no_mangle]
pub extern "C" fn hello() -> () {
    println!("Hello, world!");
}

#[no_mangle]
pub extern "C" fn echo(c_string: *const c_char) -> () {
    // Convert the C string to a Rust string for use within Rust
    let rust_str = unsafe {
        std::ffi::CStr::from_ptr(c_string).to_str().unwrap()
    };
    println!("{}", rust_str);
}

#[no_mangle]
pub extern "C" fn call(callback: extern fn(*const u8, usize) -> bool) -> bool {
    const string: &str = "Hello, world!";
    callback(string.as_ptr(), string.len())
}

The resulting config.ts:

import { LoaderConfig, T } from "hyperimport";
export default {
	buildCommand: ["rustc","--crate-type","cdylib","<path>/src/math.rs","--out-dir","build/math.rs"],
	outDir: "build/math.rs",
	symbols: {
		add: {
			args: [T.u32, T.u32],
			returns: T.u32
		},
		call: {
			args: [T.function],
			returns: T.bool
		},
		echo: {
			args: [T.ptr],
			returns: T.void
		},
		hello: {
			args: [],
			returns: T.void
		},
		mul: {
			args: [T.f32, T.f32],
			returns: T.f32
		},
		not: {
			args: [T.bool],
			returns: T.bool
		},
		sub: {
			args: [T.i32, T.i32],
			returns: T.i32
		},
	}
} satisfies LoaderConfig.Main;

[bconnorwhite]

This should maybe be in a separate PR, it just regenerates the config types if the source has changed

[gaurishhs]

Instead of node, tree-sitter provides wasm bindings which can be used https://github.com/tree-sitter/tree-sitter/blob/master/lib/binding_web/README.md

[gaurishhs]

we should probably conditionally import per-lang packages instead of including this in deps

[gaurishhs]

conditional import + try to load for all the loaders specified in bunfig.toml