-
|
This is a discussion for the future plugin system, created in order to prevent further clutter of #122. Latest update: #3806 (comment)
The plugin system is being worked on at #8675. The language will remain scheme, but the implementation will make it possible to easily swap languages if for example we choose to implement our own scheme. If we decide to stick with Steel, it can also load cdylibs (see https://github.com/mattwparas/helix-config/blob/master/term.scm as an example).
|
Beta Was this translation helpful? Give feedback.
Replies: 64 comments 252 replies
-
|
what is the current status? it is hard to follow the issue. is wasm still on the table? i think it was a great idea. if not what other alternatives are taken into account? |
Beta Was this translation helpful? Give feedback.
-
|
@d12bb I added it, but this list is not all that useful, as it includes things that are outdated from the conversation. Nevertheless, I'm happy to keep it up-to-date. |
Beta Was this translation helpful? Give feedback.
-
I'm not personally a fan of Lua after using it with Neovim for a while, although it is much better than VimScript 😉 That being said, picking Lua would ease the switch from Neovim to Helix for some users. I like the idea of using a small embedded lisp. |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
Another +1 for With that combo you get the best of both worlds: a lisp-like language with the batteries of Lua. I just kept looking for excuses to write more and more Fennel code :-) That was a lot of fun... With Lua you'd not just make it easier for some Neovim users to come over, you'd also make it easier for plugin authors to port their work over. |
Beta Was this translation helpful? Give feedback.
-
|
@d12bb afaik guile is not available on windows, so how would that work? |
Beta Was this translation helpful? Give feedback.
-
|
Proposing a candidate for a plugin system: rhai. |
Beta Was this translation helpful? Give feedback.
-
|
Rhai is too new and has a bus factor of one. If the maintainer stops maintaining it or it turns out we need certain features or bugfixes then we're stuck. |
Beta Was this translation helpful? Give feedback.
-
|
Would Javascript be an option? Active development and its probably rather easy to expose an API. |
Beta Was this translation helpful? Give feedback.
-
|
I think he is just younger and isnt necessarily aware of the fact that there are probably hundreds of people subscribed to this thread receiving an email on every comment. @LeoniePhiline the energy you have is great, but the maintainers here have to work on this on their free time and they are streched quite thin. Theyve invested countless hours into this and they will have to suffer the consequences of the decision they make. Be careful when you post a comment as this could be badly intererpreted. Someone here could become a co contributor on a project you make later or it could become something you wish you didnt say and cant take back. You likely have a bright future ahead, careful on the way there! @archseer @pascalkuthe thanks a lot for the work and patience you put on this project. For every little thing that is annoying, there are probably hundreds of people who are very grateful. There is so much work left to do, but so much has been done already ! Keep up the good work, im really looking forward to what helix brings up next ! |
Beta Was this translation helpful? Give feedback.
-
|
Honestly I don't really like the idea of Lisp, and I have no control over what the creators think, because they are the creators for a reason, but I would give a suggestion that I doubt they would accept or even see: Ruby. Again, ruby is not widely used and I doubt they will implement it. It was just a suggestion that doesn't really add much. |
Beta Was this translation helpful? Give feedback.
-
|
I agree on choosing lisp over wasm might be why there doesnt seem to be plugins as of end of July 2023. It seems wasm would be a perfect solution to allow a larger plugin ecosystem and more development, Far more people on the planet could use a wasm interface over a lisp interface. |
Beta Was this translation helpful? Give feedback.
-
Or perhaps it's because we had three separate attempts at integrating WASM that didn't lead anywhere? :) Integrating WASM isn't just a hundred lines of code either, and the lack of a standard ABI interface means we'd still be targetting something on WASM rather than WASM. e.g. if we integrate Guile via WASM, that's a WASM interface, but I'm pretty sure it's not the one you were thinking of! (or a more serious example, quill only targets rust so far) |
Beta Was this translation helpful? Give feedback.
-
|
I posted above, but would it be worth examining the wasm 2.0 announcement from a few days ago? Or was this included in your assessment. Although, I do get the main point about wasm just being way too unstable to get any real progress done |
Beta Was this translation helpful? Give feedback.
-
|
@kirawi Can we start a curated list of plugin ideas right away. It might seem too early but if we have a list right now then it might be useful for future devs to take suggestions from. We could ask users to suggest neovim/emacs plugins they would like to see in helix and after a huge list gets created, we could put it in a poll where the devs can get a clear idea of what plugins or plugin ideas are necessary. |
Beta Was this translation helpful? Give feedback.
-
|
It would be useful. A few plugin ideas can be found in the issue tracker with the |
Beta Was this translation helpful? Give feedback.
-
|
This is what I was suggesting in this comment. For me, magit is unparalleled as a git tool, and I'd love to see integration that tight in helix. Perhaps in core. It is discussed in #227. |
Beta Was this translation helpful? Give feedback.
-
|
Now we should think about the best way to do it. To me I thought we should first create an overloaded list and then put it in a poll because I think we can't edit polls to add new plugins(or plugin ideas) later. If we can then it's a nice thing. But since we probably cannot, this should have it's own place on the helix website where a plugin idea could be added to the list on user suggestions. I can full help to maintain such page. If authorization is an issue then I'll create a repo right away where I will add user suggested plugins as they suggest me in chats, discussions or issues. Some of you guys might be concerned about my recent disrespectful behavior at element, for which I apologize again. I'll try to stay absolutely neutral with that page and put my full efforts in maintaining that page if you guys help me fix marksman for markdown in helix first ; ) |
Beta Was this translation helpful? Give feedback.
-
|
This could be useful to kind of get a clearer picture of what features the community and maintainers would like to be part of the core experience vs what belongs as a community maintained plug-in. But as for what to work on, I think once a plug-in system lands, this will largely be determined by the developers who put the effort into it. Devs will naturally pick the features that best motivate them. And what the most popular plugins will end up being determined naturally I think. |
Beta Was this translation helpful? Give feedback.
-
|
Here is the list: https://github.com/zim0369/helix_plugins |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Just as a quick preface to understand where I am coming from: to me, the most important benefit of a plugin system would be bringing some features that are critical to me in an editor to helix (most notably VCS integration, something like nvim-lightspeed and a filetree). It seems the current plan is to embed scheme, because WASM component models are not stable enough yet. Therefore, I would like to propose a rust based plugin system: This approach allows plugin developers to use the full rust ecosystem. Furthermore, plugin authors could easily write "library plugins". These would be crates that depend on the The obvious drawback to rust based plugins would be the need for compiling these plugins. For people unfamiliar with rust/simple plugins, a scripting language could still be added to helix in the future (WASM, scheme or something else). I have many more thoughts/ideas regarding the detailed design and comparisons to other approaches, but I just wanted to share my basic idea and gather your feedback. Could you see this as an approach being a good fit for helix? Because this is a pretty long post, a short summary (with some added points):
|
Beta Was this translation helpful? Give feedback.
-
|
I personally quite like the idea of rust-only WASM and we could definitely overcome the WASM limitations in that case but I am not sure if the maintainers are open to that. Another example where native plugins are required is FFI. Furthermore WASM needs to be compiled anyway. I do agree that WASM is a nicer solution than native in cases where native is not required . That's why I would like to see both options (with perhaps only a trusted few plugins available as native plugins like VCS integration) and native as an easier first step. |
Beta Was this translation helpful? Give feedback.
-
I have to agree with that, as a 'rust-only' approach completely eliminates the benefit wasm as a plugin system gives you. |
Beta Was this translation helpful? Give feedback.
-
|
edit: in all this time using Emacs which crashes, blocks, races too much mainly when using third-party packages/extensions, I can say I prefer rust safety and thus a rust-only wasm over lisp/scheme. I agree with @pascalkuthe Rust a compiled language; that means no iterative/REPL-driven, easy, fast development as allowed by something like lisp/scheme, which for me is a big plus for extending the system (imagine just if Helix ever implements a much needed for serious use proper client-server architecture, I want this thing uptime 24/7, we need this to be dynamic). Y'all should look into GNU Emacs and its incredible approach (C for underlying software, and Lisp for extending it). A major issue with it is the language in that people really want to make Emacs their total computing environment, so they want to access a modern Web browser on it and so on, which only ecosystems like javascript, python have libraries for
also it is a big issue that in GNU Emacs Elisp is actually a bad language for today's concurrent computing world. So a language like Rust, would shine in providing a safe tooling, because at least for Emacs, at least 50% of its value comes from its ecosystem (packages/extensions). |
Beta Was this translation helpful? Give feedback.
-
For me the reason I want to implement extensions in Rust is precisely the opposite. REPL driven development look appealing at first but honestly I much prefer the reliability and speed that rust provides over dynamic languages. My experience with dynamically typed languages (python, lisp variants, lua, ...) is that you end up with a ton of bugs that a static type system would prevent unless you use copious amounts of testing (which lets be honest most people won't do for an editor plugin).
Both shared libraries and WASM can be loaded and unloaded at runtime so that would not be a problem.
I think most people here are familiar with emacs. Both Emacs and vim both use However I would hope that for complex plugins, that we are able to leverage existing statically typed programming languages to ensure that helix plugins can be just as reliable as the helix core code |
Beta Was this translation helpful? Give feedback.
-
abi_stable promotes itself doing "load-time type-checking".
Does this imply additional memory safety guarantees compared to a C ABI interface? |
Beta Was this translation helpful? Give feedback.
-
|
I'm Just ordinary user chiming in. From the description from on https://github.com/bytecodealliance/cargo-component , the components model came up because they needed to wrap WebAssembly modules. It seems like there is some need to make a wit-bindgen like tool for the components model, so some solution should arrive soon. Herein, I try collect in one place, summary of sub-discussions for quick reference, with links included inline to the actual sub-discussions. Plugin language options
The case for plugin-provided-feature
The case for in-core-feature
List of candidate plugins
List of conventional-plugin-features gone in-core
Desirable characteristics of a plugin system
Projects that are noteworthy for their plugin system implementation
Other discussions
so... where we at? Q1) Is choosing a plugin language now, just a temporary stop gap for small simple plugins until wasm matures, just to buy time for an eventual later migration? Q2) neovim has both a lua and vimscript engine. Would helix be able to have multiple simultaneous plugin language engines, perhaps aiding transition if the afore-said stop-gap were to be retired? Q3) Would the plugin api be designed to be so that it mimics vscode so that as many plugins from vscode eco-system be easily ported? Q4) If lapce (a gui code-editor like vscode) is going with wasm, wouldn't that also allow easy script sharing and porting with helix (and also kakoune if it gets a wasm plugin engine)? Q5) Would plugin system allow for dynamic load/unload or one time load on startup? notes
Helix holds much promise. thx to the developers P.S.
|
Beta Was this translation helpful? Give feedback.
-
|
You mention JSON API as an option and that it would be slow to convert JSON constant for message processing. What about a binary protocol? The plugins could be applications started by Helix that meet some binary protocol specification on a unix domain socket. Do we think there are interactions that plugins would want that are orders of magnitude more chatty than LSP, which uses a similar approach (and JSON)? I just think it's interesting from a feature isolation standpoint. If a plugin crashes, my editor doesn't freeze or crash (assuming it handles timeouts and read delays with grace). |
Beta Was this translation helpful? Give feedback.
-
|
I say this with zero interest in attempting to rush or apply pressure on any of the maintainers of this project, only as someone who is considering using helix as my primary editor(because its really cool, and of all of the things I've tried is the only one that "just works"): is there any kind of rough estimate on when a plugin system might emerge? I don't mean dates, just the general scope of it. Are we talking weeks/months/years? |
Beta Was this translation helpful? Give feedback.
-
|
For the sake of completeness, I am adding picolisp here. Its extremely small and has a very powerful ffi and can interact with everything. Without knowing much I suspect it would be very easy to embed. |
Beta Was this translation helpful? Give feedback.
-
|
Looks like years is the right answer. |
Beta Was this translation helpful? Give feedback.
-
|
If only we used Cloud Build! Then we'd be up and running in a weekend. |
Beta Was this translation helpful? Give feedback.
-
|
Also: stick with WASM plugins, IMO :) If we want to take Helix with us into the future, we should lean into WASM. |
Beta Was this translation helpful? Give feedback.
-
|
true |
Beta Was this translation helpful? Give feedback.
-
|
100% |
Beta Was this translation helpful? Give feedback.
-
|
... #3806 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
I have a question... I don't have a terribly strong opinion about what language helix ends up using long-term. However, it appears to me that no matter what ends up happening, it will take a significant amount of time. Is there any possibility for any sort of short-term solution? In particular one idea I had was to be able to write something in rust that has access to all of the helix commands as functions, and just be able to compile that to some target and put it in the .config/helix directory. I'm far from an expert on rust development in general or helix's development specifically, but since the codebase is written in rust it doesn't seem like much work would have to be done to expose these functions. No need to write a whole api spec or anything like that. It definitely wouldn't be the best idea in the long run, but the idea would just be to give the users some way to script until a more formal solution comes about. It obviously wouldn't have to be that idea either. I just would like to know if there's any possibility of something to give us the ability to script something until the plugin system is ready. Because it looks like it may be a long time before that happens. |
Beta Was this translation helpful? Give feedback.
-
|
Well, you could always go the Suckless way and write patches to apply after pulling, before compiling. No support from Helix itself needed, but you would obviously have to keep up with internal code changes (but that would be the case with modules in .config too) |
Beta Was this translation helpful? Give feedback.
-
|
This could be a great short-term solution. Xmonad, a window manager written in Haskell, uses essentially this model. The ~ This would get something working right away, and wouldn't require a different language or a VM. |
Beta Was this translation helpful? Give feedback.
-
|
https://bytecodealliance.org/articles/wasmtime-1-0-fast-safe-and-production-ready |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/bytecodealliance/wasmtime/releases/tag/v2.0.0 and now there's 2.0 |
Beta Was this translation helpful? Give feedback.
-
They jumped a whole version number between September and now?? How did that happen? 😅 |
Beta Was this translation helpful? Give feedback.
-
|
The release process is described here. |
Beta Was this translation helpful? Give feedback.
-
|
EDIT: I seem to have come across as trolling or hurtful to some people and for that I apologize. I did not mean to sound overly critical, and should not have implied that overall progress has stagnated, which is obviously wrong: What I meant was that progress on this particular issue seems to be slow, and I can't wait to find out what direction this will take and in what time frame it will happen. First of all: I love the Helix project and wish it nothing but the best. I personally haven't yet invested that much time into learning vim (only dabbled so far), but I really want to learn one modal editor to use for most of/all of my programming in the future. I have been following Helix since the beginning of this year, and really like what it already is! Totally fine if "competing" with Neovim is not a goal of this project, but I would really like to know if that's the case and what the vision for this project really is. If a plugin system is something that's far off, like potentially years away, I would like that to be clarified as that would mean that this is not the right fit for me, personally. ThePrimeagen pointed out in one of his recent videos that one of the great points about learning vim is its ubiquity on all platforms. Of course that grew out of it having been used for decades, and helix might never reach that point – but I'd like to know if helix even aims that high or if this is just kind of an experimental project that's not really intended for mainstream use. Fine either way, but I'd really like to know where the project's core developers stand with regards to this. |
Beta Was this translation helpful? Give feedback.
-
Find ways to make Helix possible. If you don't know how, make sure that you are not standing in the way. |
Beta Was this translation helpful? Give feedback.
-
|
As an ordinary user, in this comment, i just want to
1: I am guessing the comment-OP, and other people like me, who want plugins are not a plugin developer, but a plugin user. Possibility of great plugin system is not the only lure of helix. I want to herein convince reader that its present unavailability of plugin-system is an insignificant negative. When one chooses an editor, one wants to use the same editor everywhere as possible, to get best use of muscle memory. 2: editor evolution 3: no hurry |
Beta Was this translation helpful? Give feedback.
-
|
I think the charm with Helix is that it just works from the box, and I don't need to spend time fixing .init files like with Neovim. So the more we could put in as default behavior in hx without making it bloated, the better. For example, if we had a nice way to just call command line tools and get the output into a temporary shown buffer, maybe even in structured format (similar to nushell or jc), that would work fine for me. Then I could write various custom shell scripts and use those in my environment not putting a burden on custom plugins and a plugin runtime. Git integration could/ should be part of the default hx editor, for example. |
Beta Was this translation helpful? Give feedback.
-
Link to https://github.com/fregante/GhostText/blob/main/PROTOCOL.md an #3478 |
Beta Was this translation helpful? Give feedback.
-
|
While I do think a fully fleshed out plugin system is a fine idea, I also love how much works immediately upon installation, @ksandvik. In fact, think it is probably a good design goal, even if not stated, to have a robust, usable, and feature-rich editing experience out-of-the-box, and reserve plugins for unique or resource intense components. I am excited to see where all this lands and am so pleased that the editor is a perfectly capable joy to use in the meantime. |
Beta Was this translation helpful? Give feedback.
-
|
[Summarizing a conversation from Matrix for informational purposes] Unless I misinterpreted something, it looks like the decision to not use wasm was mainly due to the lack of a standardized interface. Using wasm only opens up the possibility of multi-language support, but it's not automatic due to the lack of the interface types. So I shared what looks like a workaround of this issue that I've seen in the past. graph-node has a wasm runtime. It's not exactly a plugin system, but it does expose a wasm API and runs user wasm code (so sandboxing is a hard requirement). Instead of defining the interface in Rust, they're actually doing it in their target language AssemblyScript here in graph-ts, and in turn use a custom codegen tool to turn it back to Rust for use in the wasm host. This way AssemblyScript gets direct access to the API, and for any other language, they just have to implement another codegen tool for bridging the gap. Generalizing this approach further, we can declare the interface in a format like Obviously with this approach there's gonna be extra effort for supporting more languages. For that, @archseer acknowledged the value of settling on a single plugin language, and I agree. However, wasm has a lot more benefits other than just "multi-language". Performance and sandboxing (and thus a permission system) are extremely important in a plugin system too. We can totally settle on a single "officially supported" plugin language, and only maintain the codegen tool for that language. It's also future-proof if the plugin author community decides that another language is better - no backward compatibility concern at all. |
Beta Was this translation helpful? Give feedback.
-
|
Nice ! |
Beta Was this translation helpful? Give feedback.
-
|
In a discussion below, it seemed like sandboxing, in some implementations, may force byte-array ser-de for call arguments and return values across host-guest interface. It may be that ser-de does not affect performance if if such funtion-calls into host are rare. The general vibe I got while reading all comments is there are those who are enthusiastic about sandboxing, and wasm being a web technology has a selling point of easy to sandbox. But, is Sandboxing necessary? Sandboxing
Sandboxing is important in use-cases like
Editors and IDEs are usually trusted applications that pretty much have access to anything. Users are also economic with plugin choices and they are also trusted, ergo the question: So what if editor plugins do too? Does sandboxing make sense for editor plugins? For completeness of discussion sake, could one try expand on all the future reasons/use-cases sand-boxing brings to editor plugins. Perhaps also list some known cases in other projects where lack of sandboxing caused difficulties. (just trying to learn here) Some reasons I could think of
The above to me seem far fetched and don't feel compelling enough to want them in editor plugins. |
Beta Was this translation helpful? Give feedback.
-
|
I'm going to make a suggestion that maybe Helix doesn't need an extension system at all. The problem with extension systems is straightforward: They reduce velocity because they increase the consequences of breaking backwards compatibility (since existing extensions will be impacted by changes that aren't backwards compatible). In specific terms, they direct text editor developers efforts towards extension API maintenance, and away from feature development. What about the fact that most (all?) successful text editors have extension systems? I've written about the rise of extension-based editors (see "the Text Editor as Platform"). In that piece, I make the case that VS Code has pushed the extension-based editor as far as it can go. In short, as always happens with innovation, the design space of extension-based editors has now been exhausted. An extension-based system in Helix will put it up against VS Code in design approach. This is a match that Helix will always lose, because one, Helix wasn't formed around extensions at its inception like VS Code was (and creations always bear the characteristics of their inception in perpetuity). But also simply because of the constraints of the format itself, since a web-based editor has a lingua franca of GUI customization (HTML/JS) with no equivalent for a TUI editor. It's always a stronger approach, especially at the beginning, to focus on your strengths, over shoring up your weaknesses (a later concern). In my opinion, Helix's strongest advantages today is that it achieves a tremendous quality and power while not yet taking on the baggage of an extension system. Another reason all other successful editors have historically had extension systems is because, until recently, they needed them! There was no consolidation about how languages were supported in an editor. So extension systems needed to exist to implement the glue code between how various linters, autocompletion systems, and other tooling interfaced with the editor. Now that language server protocols have largely consolidated how those features are implemented, the case for an extension system is a lot weaker. When a paradigm shifts arises, in this case, text editors moving beyond the need for extension systems, it's because the technology landscape has changed in a fundamental way. LSP is that change. Beat your enemies, not by playing their game, but by changing the rules of the game (just having fun here, I certainly don't think anyone should think of VS Code as an enemy). In this case that means continuing to build new core features at the same amazing quality level of Helix's existing features. VS Code can't do this themselves, because the VS Code team has to maintain their extension system. Zig when they zag. |
Beta Was this translation helpful? Give feedback.
-
With that said, I do think dedicated VSC integration could be better than
❤️❤️❤️! |
Beta Was this translation helpful? Give feedback.
-
|
I think that Helix really does need a plugin/extension API. There some languages such as Idris and Clojure that are really meant to be written in an interactive manner, and unless it is possible to write extensions to support these languages, Helix will fundamentally lack support for them in a way that no other popular editor does. LSP is delightful, but neither it nor any other protocol will ever be able to support specialized, language-specific needs such as these interactive editing modes. |
Beta Was this translation helpful? Give feedback.
-
|
As more developer tools companies begin introducing VSCode extensions along with their products, as well as some popular libraries used for development, this sentiment can only go so far. Given that Helix decides to not go with a plugin system at all, these new products will have no way of integrating themselves into the editor. Let's consider this: if Vim adopted LSP when it came out, would NeoVim have ever taken off? Would there need to be this whole effort to "port" LSP into Vim just to get it on-par with VSCode? So, I would say that if Helix doesn't build a plugin system, then we are digging our own grave here, IMO. That said, I don't think it needs to be all that crazy. As others have said in this discussion, most editor plugins these days really don't need much: call a CLI tool of some kind, read its response and massage it into the data the editor needs to show feedback. Not really that complicated, not sure if we need an enterprise-scale common runtime system just for that. LSP and code formatters are really good enough for my needs, personally. |
Beta Was this translation helpful? Give feedback.
-
This is exactly what kakoune does, and it doesn't scale well even to simple use cases. All the shelling out is extremely expensive and makes using even a few plugins at the same time extremely sluggish. |
Beta Was this translation helpful? Give feedback.
-
|
Writing a shell script is not a deal breaker, but having to write a shell script in a string literal in a kakoune script was a deal breaker. There is no good way to indent shell script in a string literal in a kakoune script. There is no good way to activate syntax highlighting and automatic indentation for blocks of external programming languages in a kakoune script. Thus, kakoune sucks at scripting. It doesn't have a clean way to deal with external languages. Because kakoune script doesn't have control structures like if statement, it has to embed shell script in a string literal. It is a failed attempt at an orthogonal plugin system. It's also difficult to keep IPC(interprocess communication) fast. Making IPC fast is possible, but it requires a careful design and optimization. I think lua as the official plugin language is a good starting point, but the real deal would be to have one official scripting/plugin language that is run by an interpreter running on sandboxed webassembly runtime. Helix can include the interpreter. Users can configure helix with the official scripting language running on a webassembly interpreter. Then, you would have orthogonality through other compiled webassembly languages. We just need to wait for a good webassembly interpreter. |
Beta Was this translation helpful? Give feedback.
-
|
Has anybody considered Python? I've dabbled with pyO3 and the interop with Rust is fairly painless. It's seamless interop is one of the things I like about Rust. Also, python is more popular than Lua and more mature than wasm. Although caveats of course is you're still limited by the GIL, and it uses your local python interpreter, so varying local python setups might make the user experience inconsistent, but I suspect that can be circumvented somewhat with virtual environments. |
Beta Was this translation helpful? Give feedback.
-
|
Let's not lol. Using a local Python interpreter would introduce all sorts of problem like you already mentioned. No I don't want to set up virtual Python environment just for using some plugins... I need things to just work like they do in say VS Code. What about shipping an interpreter like |
Beta Was this translation helpful? Give feedback.
-
|
Actually, same energy as: #3806 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
Hmm fair points. But I guess the purpose of plugins should be that it's easy to author, and it's pluggable and it's optional to the core experience. Hence a scripting language is often more successful. Agree that RustPython is probably overkill (although TBF I don't really know how big it is, nor the compile time). Throwing ideas out there, how about somehow limiting people to core python3? I don't know enough to comment whether core python would suffice for writing plugins. Cursorily looks like what neovim does for Lua? Although it seems to ship the Lua interpreter. And also is it a reasonable assumption/pre-requisite to make sure people have Python 3.7+ installed on their machines (my initial thoughts are it is, because most distros ship with Python, excluding windows). |
Beta Was this translation helpful? Give feedback.
-
|
I don't think shipping only core will suffice because some plugins like discord presence may need more more than that |
Beta Was this translation helpful? Give feedback.
-
|
My Mac brew environment now has python 3.7, 3.8, 3.9 and even 3.10. It's a pain to keep a stable python environment around as the scripting system for Helix. |
Beta Was this translation helpful? Give feedback.
-
|
I think you meant to reply to a thread above, but anyways: IMO: No plugins but features well integrated ≈ Contained and sandboxed plugin system (wasm) >>> Any other plugin solution |
Beta Was this translation helpful? Give feedback.
-
Cloud Build - pure Rust helix pluginsNot sure if Cloud Build has been suggested yet. Prior artBoth BetaFlight (v4.4) and ExpressLRS (v3.3) have recently introduced Cloud Build. Both projects deal with MCUs:
Their motivation was overcoming flash space limitations on cheaper MCUs, while being able to add features (= more bytes) for top notch hardware as well as support a wider variety of features such as new models of microcontrollers and peripherals, such as gyroscopes, accelerometers and GNSS systems as well as new protocols for communicating between the MCU and the various hardware peripherals. Applicability to helixLike helix plugins, BetaFlight and ExpressLRS features must have the lowest possible latency and be baked into the core software as tightly as possible. In this discussion, many proposals had to be rejected due to size, RPC overhead, serialize/unserialize/marshal/unmarshal costs and the lack of a stable ABI. Other options involve more or less esoteric programming languages, where obviously Rust would be the best choice if just we could circumvent the need for users to have toolchains available and compile helix with their own set of plugins. How it worksIn BetaFlight and ExpressLRS, it works like this:
The CI costs are covered by donations. BootstrappingBetaFlight and ExpressLRS both use a "configurator", which is either an Electron app or a PWA with USB access for flashing hardware. For helix, I would imagine this to be either a plain web app, or (probably not?) part of the helix binary. As a third option, a tiny bootstrapper CLI can be published, which is only used to pick plugins and then download helix for you from Cloud Build. But using a small web app appears to me to be the best solution. It would provide UI for the following steps:
Benefits for helixA Cloud Build solution would allow for helix plugins to be written in pure Rust, without requiring users to have toolchains installed. |
Beta Was this translation helpful? Give feedback.
-
You’ve got the same issue right now with offering precompiled binaries in the github releases. Nothing changes. It’s just the same CI with an added parameter (list of plugins). Tamper-proofing is a solved issue. It is common practice to even distribute binaries through third party mirrors as long as the checksums are published by the primary source.
Of course you need a versioned API! Any possible solution (including lisp) needs a versioned API. Now imagine you could do all that without the need for unsafe external C bridges, in plain safe rust.
The selected answer is outdated; mentioning wasm, an idea which is said to have been discarded. Exactly like what I propose, the selected answer mentions rust-based plugins for performance. You writing "It doesn't really solve anything either" is just rude. It does in fact solve the problem of using rust-based plugins while not relying on wasm, and neither on people having to have a rust toolchain ready.
The primary use case for any plugin system is extensible functionality. No scriptable config. A more powerful scriptable config can only ever be secondary. A scriptable config is a separate issue that should receive a separate solution. That’s a config problem, not an editor functionality problem. It would help seeing those two realms more clearly as separated. If you will, a scriptable config system could be implemented as one of many native rust plugins. |
Beta Was this translation helpful? Give feedback.
-
The use case of any plugin system is that you can plug components in. What you're proposing right now is something where, if I wanted to add a new plugin to test it/see if I like it, I'd need to go to a configurator page, to tick all the boxes I had before plus one, and then wait for a CI process to run and build the editor, and then download it. And do that for each architecture if I use helix on a macbook and a linux desktop for example. I really don't like that UX for a plugin system. |
Beta Was this translation helpful? Give feedback.
-
|
This is absolutely acceptable if you can have both safe and and fast plugins. Choose C bindings and you will have unsafe pointer handling, memory leaks, segfaults, and vulnerability hell in the plugin system. Choose a scripting language (for the plugin functionality, not the configuration, mind you) and you get a soggy laggy editor that calls into foreign scripting runtimes on each keystroke.
Web app: In a web app, this is easily solved with a cookies / local storage. You will never need to tick boxes for plugins you already have installed - this is a typical task computers are supposed to do for us. CLI app: If you have the Cloud Build system API client implemented as CLI tool instead of web app, then it already knows which plugins you use, and it can just add the one you like to try. Same if plugin management was integrated into helix itself. (Just run Build time: If you are not running plugins nobody else uses, then your binaries will already have been built and cached, so there is no waiting time at all. You get your binary directly and can start trying the plugin. On the other hand, if you are experimenting with your own plugin or something new and unknown, then you would need to compile it anyway. Except if, as said above, the plugin system is based on slow scripting runtimes which would make you quit using helix anyway, due to bad runtime UX. The image you are painting is unrealistic: You are not changing your set of plugins constantly. Plugging something in does not mean that you rip it back out and shove it in again multiple times a day. It just means composability and extensibility. It means being able to make a connection, an integration happen. If you require a dynamic system, use something written in a dynamic language, or something static with a stable ABI which is also safe (hint: does not exist). |
Beta Was this translation helpful? Give feedback.
-
Ironically, Rust does not have a stable ABI so we'd have to compile to C ABI. |
Beta Was this translation helpful? Give feedback.
-
|
Ah list of "plugins" to extend the core functionality to be specified at compile time? Sounds like a pending PR rebaser lmao: Welcome to my plugin system :) |
Beta Was this translation helpful? Give feedback.
-
|
Who are your potential plug-in writers? What would be the easiest solution form them? I do not know if a Helix user survey exist, but I suppose most of us have a modal editing background. Very likely we come from Vim, then Neovim and end up with Helix (for good reasons). Probably most of us have some Lua experience we could easily build on. At the same time I think we have very few users coming from Emacs mastering Lisp. Furthermore, I can imagine many of us have some notion of Rust. As these are just my personal assumptions, shouldn't we make a survey what plug-in language our users are actually familiar with? The plug-in extension is planned for some audience; shouldn't we know more about them? Once we know what is the easiest solution for our target audience, the chances are high to motivate more people to write fancy plug-ins. A simple survey could be realized in a "discussion" thread. |
Beta Was this translation helpful? Give feedback.
-
|
Thats the point @pascalkuthe was making. Helix "does not care" about this aspect. It solves problems for a subset of people. If that subset is large (as is today) that is a side-effect, not a main drive/goal. Hence, Helix will for the most part prioritise itself, not what a large(r) user-base would like. If it made technical sense to have an Assembly-based plugin system, that would be the choice, even if the majority of its users would only know Python. The maintainers are already working on this issue anyways, with their own approach. They read opinions, but are not bound by any other than theirs. Afterall, they are responsible for developing, designing and maintaining such a system. They will go with what makes sense and is easy, since they already have a hard, time consuming job as-is. |
Beta Was this translation helpful? Give feedback.
-
|
The art (and pleasure) of our developer job, is to design interfaces, that are easy to consume. This does not question authority in any way. |
Beta Was this translation helpful? Give feedback.
-
|
True, but an open source project also stays alife from how easy it is for others to join. Since this is all done using private time for free, maintainers change since life can change. So if something is a PITA it might not be that good of an idea in regards to the future of the project. That being said, with my comment I am not critizing any position, choosen stack or anything, just a general observation and thought. |
Beta Was this translation helpful? Give feedback.
-
This could argue in favour of Rust as plug-in language. A plug-in writer could easily become a (core) developer. BTW: When the Neovim project chose Lua some years ago, Lua had still a niche status. Nevertheless, it was a very smart choice: Lua is extremely simple and easy to learn, flexible, fast and resource efficient. That said, other (new) languages might have similar properties and should be considered. |
Beta Was this translation helpful? Give feedback.
-
|
BTW: I just learned that Lua was embedded as scripting language into a TeX engine already in 2007!
Modern LaTeX - modern-latex.pdf, page 47. |
Beta Was this translation helpful? Give feedback.
This comment was marked as off-topic.
This comment was marked as off-topic.
-
|
Hi! Did you take a look at what Kirottu/anyrun does for its plugin system? It's quite limited compared to what you'd want to have when builtin plugins for a text editor, rather than an application launcher, but is still quite a good start. This would still assume that most people know how to use Rust extensively, and quite opposed to the idea of @getreu where we could poll potential plugin developers (perhaps we could use FFIs to integrate with more languages like Lua?) |
Beta Was this translation helpful? Give feedback.
-
|
Btw deno is also written in rust and it's ready for embedding in other projects. Plugins in js/ts will be good, soydev friendly addition 🙃 |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
@hemedani I think if more people thought that deno is good choice, then making deno support would be quickly achieved. |
Beta Was this translation helpful? Give feedback.
-
|
I'm agree with you, and I working on several project based on |
Beta Was this translation helpful? Give feedback.
-
|
is nickel a good fit for this use case? |
Beta Was this translation helpful? Give feedback.
-
|
I believe nickel is configuration language and would be close to useless when it comes to making plugins. Non existent ecosystem + language barriers (I don't think it's turing complete) makes it a bad choice here imao. Nickel could be used instead of toml for Helix configuration, but tbh toml is plenty for simple turning options on or off. |
Beta Was this translation helpful? Give feedback.
-
|
Even Nix itself is a programming language. |
Beta Was this translation helpful? Give feedback.
-
|
@CaesarXInsanium yea, it is programming language (and I think I've read that Nix is turing complete), but it's crafted toward solving specific goal (just like nickel). I'm active Nix enjoyer (writing this comment from NixOS) and it would be horrible to do any real programming in nix. |
Beta Was this translation helpful? Give feedback.
-
|
While a lot has been said already, want to throw Mojo on the radar. Coming from neovim, i really value they have added lua as its perfect to be embedded and has a jit to be fast. Mojo wants to address pythons performance , while being a superset of python. Read more here. |
Beta Was this translation helpful? Give feedback.
-
|
it looks like Mojo can be targeted to WASM, so supporting WASM would pave the way for Mojo support. |
Beta Was this translation helpful? Give feedback.
-
|
W3C recently released their draft for Wasm 2.0. Could be worth taking another look into |
Beta Was this translation helpful? Give feedback.
-
|
its seem no one suggest scripting language like dascript or quirrel (the fork squirrel) both are made by a company who uses it in their products, not some random guy writing scripting language in the weekends and no lua please, i have a nightmare writing neovim plugin because 1 based array indexing. |
Beta Was this translation helpful? Give feedback.
-
|
I think there are 3 'good' ways maintainers can go forward with plugin system.
Anyway I think that dascript and quirrel are such a niche languages, that picking them would lower the value of plugin system. Despite it's flaws lua is lightweight and relatively easy to use. I'd much rather use 'bad' widely used language such as Lua than learn dascript/quirrel. Of course it's just my opinion and there's a chance that maintainers are big fans of those languages in which case 1) applies. |
Beta Was this translation helpful? Give feedback.
-
|
WASM is there, WASI is not there with all system-related api's. Good solution for this kind of things might be WASIX. |
Beta Was this translation helpful? Give feedback.
-
|
Can we use this / learn from it? Plugin User Experience (UX) | IntelliJ Platform Plugin (Link corrected). |
Beta Was this translation helpful? Give feedback.
-
|
I think those are based on the JVM, something helix probably doesn't want to include. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, IntelliJ Platform Plugin System includes a large number of JVM features. So does Jenkins. |
Beta Was this translation helpful? Give feedback.
-
|
I posted the wrong link, sorry. It should be Plugin User Experience (UX) | IntelliJ Platform Plugin. The document describes general UX requirements for plugin developers. I also find the pages Plugin Structure and Plugin Dependencies interesting. It categorizes use cases. |
Beta Was this translation helpful? Give feedback.
-
|
Might I ask, what sort of interface would these hypothetical plugin system use? If we assume that the language is chosen, what API would it be able to access? Should plugins be for language specific code, things that the helix maintainer do not want to provide themselves? Would it allow creation of custom UI's
I really like this editor. If it had parinfer or something like it, I would be using it now instead of Neovim. These are just some ideas. Personally I would like to use Helix to edit some lisp code, as I am reading SICP |
Beta Was this translation helpful? Give feedback.
-
|
IMO plugins systems are tarpit. Have people considered just "write more Rust, recompile the app?" as an approach? I.e. think of the editor as a library, combine with other libraries, and then just slap on a very small boilerplate |
Beta Was this translation helpful? Give feedback.
-
|
And if breaking changes in the Rust are an issue, could go full mono-repo and have a |
Beta Was this translation helpful? Give feedback.
-
I've been thinking about using this method, not to write helix plugins, but to reference helix as part of a zsh plugin (or maybe a fork). The goal would be that unlike the existing vi and emacs modes, it's not a partial implementation--it's the real thing, and it respects your existing config. Does anybody know of projects that already interface with helix in this way? I can perhaps use them as a guide. |
Beta Was this translation helpful? Give feedback.
-
|
@Ericson2314 totally - would love to see some API / crates / docs for a compile-time extension approach first - then if folks want higher-level extension options like WASM or scheme, they could always be implemented on top, potentially out-of-tree. |
Beta Was this translation helpful? Give feedback.
-
|
Yeh I think this is probably the best bet. This would be the fastest way to explore the concept and figure out a plugin API. It could be extended to support WASM once a concrete API is determined. You could also load the plugins as dynamic libs. I don't think the complexity of rust or language choice is much of issue when it is confined to a Plugin API. |
Beta Was this translation helpful? Give feedback.
-
|
IMHO, continue discussing on this thread will not have a consensus. Why not just take the python community routine, Benevolent Dictator For Life, the helix creators decide what to use. If the scheme is the one to go, then that's it. In the future API design, expose the |
Beta Was this translation helpful? Give feedback.
-
this is exactly what's happening. @archseer is our BDFL. As we said often in the matrix we already decided on using a scheme. Its also why we marked this discussion as anwsersd (and posted exactly that information in the response). People just keep necro-bumping it |
Beta Was this translation helpful? Give feedback.
-
|
Maybe the title should be changed to "We chose Scheme as the plugin language!" so people won't overlook it anymore, :P |
Beta Was this translation helpful? Give feedback.
-
|
What about using wasm/wasi/wasix, it have good build-in support and for other languages. For things like js/lua/python it would be possible to just embed runtimes like deno into plugin using wasix (so for example all scripts written in js would require a plugin to run and provide access to helix apis). Also |
Beta Was this translation helpful? Give feedback.
-
|
Nvm it looks like there is already selected similar approach, the only thing that I think is needed is the wasi/wasix support to provide access to the system apis that aren't present in normal wasm. |
Beta Was this translation helpful? Give feedback.
-
|
Hell no |
Beta Was this translation helpful? Give feedback.
-
|
As I said in the earlier thread, I think the implementation details(language/runtime) of the plugin system are not important as this stage. First we need to figure out what sort of features we would like to enable with this plugin system, then define a plugin API that allows these features to be implemented and then finally the implementation details(language/runtime etc) can be considered. A plugin system is a huge undertaking which is why I think it is best to pick one feature, for example source control, and implement a plugin system just for that feature OR pick an existing feature and make it modifiable using a plugin. I feel like both of the plugin threads have been unintentionally derailed by people listing their favourite scripting languages and/or runtimes. The difficult part is going to be deciding the scope and capabilities of the plugin system and how it will affect the vibe of the editor. A lot of people have praised helix for its ease-of-setup compared to neovim or emacs. Something to keep in mind. |
Beta Was this translation helpful? Give feedback.
-
|
This thread has devolved into language popularity discussions and most options have been mentioned multiple times already (and no, we won't pick XYZ because it's your favourite language). I've kept the discussion open because there's occasionally been an interesting comment or project mention. But as the discussion grew older ideas would get buried in the backlog and brought up again several times. Since each post on this thread pings a couple hundred of us subscribed to the discussion I'm going to lock the issue for the time being. If you'd like to have a technical discussion on the implementation we're always available on Matrix. I plan to write a longer post that discusses alternatives we've considered (including WASM) and the design constraints chosen but we're currently leaning towards a Scheme-like implementation (in fact @mattwparas in #3806 (comment) has already implemented a nice prototype that even covers a vim-dirvish-like file tree, the furthest any of the plugin system attempts has gotten).
Helix is a pragmatic editor: it should behave as you'd expect out of the box, and I don't expect users to write significant amounts of Scheme unless they're plugin authors (yak shaving over thousands of lines of config files is precisely why the average user has switched to helix!). In fact the core has been extensible enough that we haven't seriously needed a plugin system yet. But I'd still like to provide a plugin system flexible enough to extend the editor for less common usecases Why not WASM: WebAssembly is popular but it's not the magic solution to every problem. We need to expose a very large ABI to the language and since there isn't a cross language compatible memory layout we'd still end up being locked into a single language running on top of WASM (or the maintenance burden of multiple shims to support other languages). We'd only be getting the benefits of WASM's VM and we'd be importing projects orders of magnitude bigger than the editor itself. It's not clear to me if the tradeoff would be worth it. Even if we were able to support multiple languages via WASM, it seems better to me to focus on a single language so the ecosystem doesn't fragment into smaller parts where each plugin I'd be running could be in a different language, making contribution a lot harder. Why not compiled plugins: The scripting language should also be used for configuration. While there's software that does this (e.g. |
Beta Was this translation helpful? Give feedback.
and others
This thread has devolved into language popularity discussions and most options have been mentioned multiple times already (and no, we won't pick XYZ because it's your favourite language).
I've kept the discussion open because there's occasionally been an interesting comment or project mention. But as the discussion grew older ideas would get buried in the backlog and brought up again several times. Since each post on this thread pings a couple hundred of us subscribed to the discussion I'm going to lock the issue for the time being. If you'd like to have a technical discussion on the implementation we're always available on Matrix.
I plan to write a longer post that discusses alternative…