Display Mumble Users in OBS Studio for Twitch Stream

[kingtigermusic]

Hello, my wife and I recently started using Mumble for gaming together, and we use it on Twitch streams so we can not only talk to each other but also be heard by our viewers. We use Mumble for this rather than, say, Discord, because of its low latency and light footprint.

I'm looking for a way to display channel members as an overlay in OBS Studio, so that me & my wife's avatars (and those of anyone else who participates in the call during stream, such as our teenage kid or our roommate) are displayed on stream and light up when we're speaking. As an example of what I'm looking for, Discord has their StreamKit Overlay, and there's also a 3rd-party overlay developed by Fugitech.

I understand that the Mumble overlay only hooks into Direct3D or OpenGL applications, so that's not an option, as many of the games we play don't use either of them, and we need to be able to display our avatars separately from our games.

Does anyone have any suggestions? Maybe there's a plugin I missed, or a channel viewer, or some way to display the avatars in the main client window so I can capture that window instead?

Many thanks.

[Hartmnt]

There is the "Talking UI" in the Menu Bar -> Configure tab. That opens a new minimal window showing the talking user(s). Furthermore it can be configured in the settings under user interface.

[kingtigermusic]

That's almost perfect - I can't find a way to configure the Talking UI to display avatars.

[Hartmnt]

I do not think it is currently possible to display avatars in the Talking UI... You could open a feature request for that, and maybe someone might work on it. But this would be a long-term solution.

[Hartmnt]

Another idea would be the Mumble channel view protocol: https://www.mumble.info/documentation/developer/channel-viewer-protocol/

You would have to find an application or write one yourself which uses that protocol to gather server-side information and display it on your screen (either as a website or a dedicated app).
I have no idea if this allows you to display, if a user is currently actively talking though.

[kingtigermusic]

I'll take a look at the docs. Thanks

[kingtigermusic]

Looking in the JSON and XML examples, the "users" field and the <user> element don't include any sort of "talking" element, so looks like CVP is out.

[Hartmnt]

The last possibility is to write your own client side Mumble plugin using the Mumble plugin API. This would require the most work from your side, though. And I can not tell off the top of my head, if you would be able to get avatars via the plugin API. So in the worst case, you would have to write the plugin AND extend the Mumble plugin API

[kingtigermusic]

I was thinking this might be what I'll have to do. I'll look through the API docs and see what it's currently capable of

[kingtigermusic]

The plugin API does include a function for a user's talking state:

/**
 * This enum's values represent talking states a user can be in when using Mumble.
 */
enum Mumble_TalkingState {
	MUMBLE_TS_INVALID = -1,
	MUMBLE_TS_PASSIVE = 0,
	MUMBLE_TS_TALKING,
	MUMBLE_TS_WHISPERING,
	MUMBLE_TS_SHOUTING,
	MUMBLE_TS_TALKING_MUTED,
};

but nothing to pull the actual avatars. However... that shouldn't be a major issue, as any users that would be included in the stream could simply provide me with their avatars.

According to the docs, the only languages that work with the plugin API are C, C++, and Rust. Currently the only programming language I'm remotely comfortable with is Python, so if I'm going to develop a plugin, I'm either going to need an excessive amount of help, or some time to learn C (or some of both) 😸 I may enlist the assistance of a friend of mine.

[Krzmbrzl]

Extending the plugin API to expose avatars in some shape or form should also be possible. Given that the information has to exist somewhere in the Mumble client, we can probably figure out a way to channel this data to plugins, if they request it.

[FilipStadler]

Just for the funn I have OBS running i kasem (browser controlled) and it does stream and can be controlled via the browser but in a scene it pull audio from opensource radio station and if mumble users is speaking auto docking low the music on the stream. Mumble users dont hear the music ;) I can also use the https://github.com/johni0702/mumble-web as replacement for the mumble client but it does not provide the nice Talking UI but a way that can be used in OBS also.