How to expose host functions/data to the runtime

[fermuch]

Please, help me see how to get this idea through with extism (all of this is fictional, only meant to serve as an example of how to build a real system with extism):

Let's imagine we have a board we sell to our clients. That's the host for extism. Assume our host is an Arduino-like environment. So, the host calls extism via its C-interface.

The board has several sensors (temperature, humidity, amount of CO2 on the air, ....).
Based on that information the board should, from time to time, run a user-installed plug-in and decide what to do (turn on the AC, close the curtains, run the air purifier...).

In this fictional system, we want:

• A plugabble system to run custom logic (what extism is solving)
• The plug-in to run every second, and decide what to do next.

But, providing all sensor data every time is time-consuming and can be difficult to extend later (what if there is a new sensor? or a sensor which is plugabble so might not be connected right now?).

The ideal scenario would be for the plug-in to call for these resources, as needed. If you want to read current temperature, the plug-in should call the function readTemperature().

As far as I can see, there is no way to do that. How can the Host expose a resource (a function, or additional data) to the plug-in?

And you might say "expose a dynamic object (json) so additional values can be added in the future and optional sensors can be null when not enabled". Allow me to add some points to this:

• Instead of presenting everything, it should be possible for the plug-in to request what it needs. This is way more efficient than polling everything before calling the plug-in.
• Imagine the plug-in needs some way to store and check state. For example, it wants to average the last 5 measurements of temperature. The host should provide a save(name, value) function to the client, so it can save data, and a read(name) so it can read data. How would that work on extism?

[bhelx]

I believe what you are talking about is importing functions from the host. You can do this with webassembly. We have to do it to get all of this to work. However I don't know if we've made it ergonomic for our users yet. Let me get the team to chime in.

Imagine the plug-in needs some way to store and check state. For example, it wants to average the last 5 measurements of temperature. The host should provide a save(name, value) function to the client, so it can save data, and a read(name) so it can read data. How would that work on extism?

Btw, we do provide host functions for saving state we call them variables.

[fermuch]

I know state is preserved with variables across calls. That far I know. What I was talking about was about storing state outside the running context, as in: storing to the disk. Storing a snapshot of the VM might be possible, but is very expensive for a micro controller.

[zshipko]

Right now there's no way to expose functions from the host to a plugin. It's something we've discussed and are looking into for the Rust SDK, but it seems unlikely that we will have a generic way of doing this that works across all the supported SDK languages.

[nilslice]

Update: we have a PR almost ready which exposes this functionality for the Rust SDK: #148

[zshipko]

This PR was just merged, which means it is possible to expose functions from the host now! The PR also has an example of how to use it.

[fermuch]

Great news! Just to add to the record: I've been investigating, and Wren uses a Slots system to provide external communication. Maybe a system like that might be useful to make what Rust SDK has now be available to multiple hosts?

[bhelx]

Heads up, this has been released and here are the docs: https://extism.org/docs/concepts/host-functions/

[piaoger]

It seems that we cannot use a handler(or callback) as input of a plugin function. The exposed host function can be used as global handler for now.

[nilslice]

As of v0.2.0, Host Functions are now available in several SDKs. See the overview here, and the examples for supported SDKs linked from within:

https://extism.org/docs/concepts/host-functions

[Simonl9l]

it would be great to update this reply to be the the "Marked as Answer"!