+1

For record keeping, can you just list some of the use cases that you envision for having the API running in workers?

@marcoscaceres at least this one cwilso/WebMIDIAPIShim#33

Hi @marcoscaceres, this would be really helpful for my use case. I'm generating MIDI events in Javascript, and GUI/window resize events can really affect the timing.

I've moved the scheduling code into a web worker, which works well; on average, events inside the worker fire within an average of 1.5-1.7ms of their intended time, tested over a period of 5 minutes and more.

However after that, the scheduler thread has to send the outgoing MIDI messages back via the main thread, at which point they can still be interrupted. If the web worker could send the MIDI messages directly, this wouldn't happen, and the timing could be much more reliable.

@marcoscaceres @igorclark
Hi, my AssistantPerformer (https://github.com/notator/assistant-performer)
also generates MIDI events in Javascript, and wants to do that over arbitrarily long periods of time without being disturbed by interruptions in the main thread. So +1 to igorclark -- especially his last paragraph.
I'm also planning a polyphonic version of this application. This would be like playing a prepared piano. Obviously, the more integration with workers, the better.

Thanks everyone! These are really helpful.

Sounds great to me!

This would be really nice :)

+1 on the feature, for sure. But perhaps this isn't critical for release, given that events in a sequence can be assigned exact timestamps and that the main thread need not schedule events in tight proximity to their physical output.

+1 I think @igorclark nailed the use case that all of us developing sequencers have encountered. This seems like a rather essential feature in my opinion.

I also think this is an important feature. Once basic part of the spec is fixed, I'll work on this.

It's sounding like people want this in v1. (It's pretty straightforward to do; in addition to

partial interface Navigator {
    Promise<MIDIAccess> requestMIDIAccess (optional MIDIOptions options);
};

we need

partial interface WorkerNavigator {
    Promise<MIDIAccess> requestMIDIAccess (optional MIDIOptions options);
};

and (I think) that's pretty much it.) Additional cost on implementations, of course.

This is critical for acceptable timing. How can I help move this forward?

What happened to this? The spec talks about Launchpads and DJ controllers, but you can't build anything serious around MIDI controllers with main-thread latencies. In a pro-audio context, that would be super nasty.

I think AudioWorkletGlobalScope might be the better fit for this purpose. Then you can process MIDI data right inside of the audio rendering thread.

I don't think the audio thread should be handling MIDI (especially outbound MIDI messages for controlling LEDs etc). It's not what worklets were designed for. Besides, a regular worker can use a shared array buffer (or Wasm memory) to write directly to the memory that the audio thread is rendering from, without extra latency or blocking the audio thread.

Just nudging this thread, hoping to get a status update. A few noteworthy points...

There's an interesting proposal for exposing regular input (basically keyboard and pointer events) in workers. That proposal contains some use cases and general observations that are relevant to controlling realtime audio.

Chromium et al automatically use high priority threads for audio worklets now, and that has substantially expanded the range of serious audio applications that are possible on the platform. The only major limitation that remains (on Chromium, at least) is the inability to handle low-latency input (WebUSB is threadable, but does not expose (class-compliant) HID or MIDI devices).

In short, if we could (directly) handle keyboard, touch and MIDI events in workers, we will have all of the essential primitives for serious realtime audio programming in the browser. Given that this thread is just about updating the spec, I cannot see any reason to hesitate any longer.

@hoch and @padenot should weigh in on the feasibility, but I'd hesitate to just put it in the spec without some confidence that it's possible and likely to implement.

In terms of the feasibility, I believe @toyoshim can provide us with the better answer.

I am also aware that FireFox has a working implementation - so If both implementors are positive about the feasibility, the spec change doesn't seem controversial.

@padenot is Firefox's implementation exposed on Worker?

We implement the spec as it is today, so no.

I don't think it would be particularly hard to do so, though, modulo the permission aspect, that is tied to navigator at the minute.

I do think it would be a good idea, and in fact, necessary, for any software that does anything non-trivial on the main thread.

I experimentally created a POC patch to support Web MIDI in dedicated workers.
https://chromium-review.googlesource.com/c/chromium/src/+/3439238

We have a few code that depends on information bound to DOMWindow, but we can fix it easily. So, what we really need is just to expose the requestMIDIAccess interface to the WorkerNavigator. It should not be difficult to brash up this CL to be production ready.

Can we move this forwards, please guys, and get the specification updated? I believe the reasons for holding off on this have been addressed now. It's just the API that needs finalizing (re. navigator).
Thanks.

I'd really appreciate if the Web MIDI API would be accessible from Web Workers.
When implementing a MIDI player that needs to be stepped forward at 60 FPS, the only way to do it right now seems to be either setInterval (bad performance & jitter) or requestAnimationFrame (which doesn't work while the window is not focused, so the MIDI playback pauses when the user switches from the browser window to the DAW window).
Or did I miss another way? I'm curious if there is one. If I run the midi player in a worker but use message passing to send the midi events to the main thread before sending them out to the midi port, do you think that would work without added latency/jitter or would it be suboptimal to setInterval/requestAnimationFrame?

Anyhow, if the MIDI API was accessible from web workers, a loop could be run in a worker thread for the MIDI playback.

any updates on this? is this that hard to implement.I'm using exhaustively webmidi in my app and the dom is drastically slowing down some processes. Thanks in advance

I believe everyone in the WG understood and agreed with the need of this change, but the group needs to come up with the spec text first. Sorry for the delay, but I am cautiously looking at 2023.

Thanks, @hoch. I appreciate the update. It's nice to know things are moving along, even if it'll take a while.

If this spec is committed to threaded MIDI, then the Input for Workers Proposal should drop their (stalled) effort to do the same thing. IMO, the scope of that proposal should have always been limited to keyboard and pointer events (which are tied to the DOM), and never addressed MIDI events (which have always belonged in the MIDI spec).

I'm not sure if the Input for Workers proposal is actively maintained, but if so, somebody should let them know to remove MIDI events.

I forgot that I tried to contact the Input for Workers people six months ago, just asking for a sign of life, and got nowhere. I've updated them anyway.

As mentioned briefly before, while I'd like to see MIDI in workers, I don't personally think MIDI should be exposed to audio worklets.

Worklets are very specialized. I don't think it's appropriate to handle MIDI on the browser's audio thread.

Note: Apps that are sensitive to latency can implement their MIDI interface in a regular worker, then use a shared array buffer to write directly to the memory that the audio worklet is rendering from.

I'm not certain, but I don't think any of the other input APIs (WebHID, WebUSB, WebBluetooth etc) are planning to make their APIs available in worklets (of any kind).

Note: Apps that are sensitive to latency can implement their MIDI interface in a regular worker, then use a shared array buffer to write directly to the memory that the audio worklet is rendering from.

Or a MessageChannel to communicate directly between the MIDI worker thread and the audio worklet/rendering thread, avoiding the main thread.

There are no plans to expose MIDI to Audio Worklets. I also don't know of other APIs being exposed in Worklets.

Communicating from a regular Web Worker to an AudioWorkletGlobaleScope using SharedArrayBuffer, via an SPSC wait-free queue, is the way to go.

postMessage(...); will cause problems, this blog post explains why, with benchmarks.

Audio Working Group 2023-10-05 meeting conclusions:

• We will pursue this for V1
• Two things to do to resolve this:
  • Make MIDIAccess Transferable
  • Expose MIDIAccess in WorkletGlobalScope

I wrote "WorkletGlobalScope" but it looks like we will expose this in WorkerGlobalScope; please see discussion in #256 if interested.