A little bit of guidance for adapting the connectivity.c example to real world use

[thomashenry79]

Hello, and thank you for making this library that looks very helpful. I'm trying to adapt the connectivity.c example to use in the real world, but am not 100% how things are meant to work. We currently have our own custom server that allows clients to find each other, and choose to establish a connection. When this happens, the server informs both clients of each other's public IP address, and they attempt to establish a bidirectional UDP 'connection', which works 80% of the time in practise. Rather than this 'ad-hoc' approach to hole-punching, we want to use a full ICE protocol with a TURN server. From my understanding, what should happen is this:

1. Using our existing server, upon agreeing to connect, each client should create a juice agent, configured with STUN and TURN server addresses/credentials. (OR - does only 1 client need the TURN credentials, like in the example?)

2. Each client generates a local description, and then sends to the other client via our exisiting server. Upon receipt of the other client's description, each client calls juice_set_remote_description.

3. Each client then calls juice_gather_candidates. In every on_candidate callback, the client should send the sdp description to the other via our existing server. Upon recepit of such a message, each client will call juice_add_remote_candidate.

4. When the on_gathering_done callback happens, the client signals this to the other via our existing server.

5. Both clients then wait for on_state_changed callback to signal JUICE_STATE_CONNECTED/JUICE_STATE_COMPLETED, or JUICE_STATE_FAILED. I'm not sure of the difference betweeen JUICE_STATE_CONNECTED and JUICE_STATE_COMPLETED... ?

6. If the state changes to JUICE_STATE_CONNECTED/JUICE_STATE_COMPLETED, then each client can start sending UDP packets to the other using the remote address for juice_get_selected_addresses. Do I need to use the juice_send commands for this, or can I just use my existing socket communication?

7. The agent created by each client can now be destroyed via juice_destroy? Or does it need to be kept alive for the duration of the UDP session?

Do I have this more or less correct?
Many thanks
Tom

[paullouisageneau]

Hello, thank you for your interest in libjuice. Here are some details or corrections:

1. Using our existing server, upon agreeing to connect, each client should create a juice agent, configured with STUN and TURN server addresses/credentials. (OR - does only 1 client need the TURN credentials, like in the example?)

Each client must have a juice agent configured with a STUN server. One client having a TURN server should be sufficient, but it's good practice to configure both clients with a TURN server. STUN and TURN servers may be different, they don't need to be the same.

2. Each client generates a local description, and then sends to the other client via our exisiting server. Upon receipt of the other client's description, each client calls juice_set_remote_description.

This would work but both agents will assume that they take the lead, therefore there will need to be a tie-break to solve the role conflict, generating a warning and introducing a slight delay.

The normal use case is that one agent should call the other. The agents should also trickle candidates to allow them to connect faster (this is not mandatory, see the notrickle test):

• Agent A generates a local description and sends it to agent B through your signaling server.
• Agent A start gathering candidates and sending them to agent B.
• Agent B receives the description and sets it as remote description (agent B could also refuse the connection if your signaling protocol allows it), agent B will also receive remote candidates asynchronously and set them
• Agent B generates a local description and sends it to agent A through your signaling server.
• Agent B start gathering candidates and sending them to agent A.
• Agent A receives the description and sets it as remote description, agent A will also receive remote candidates asynchronously and set them

3. Each client then calls juice_gather_candidates. In every on_candidate callback, the client should send the sdp description to the other via our existing server. Upon recepit of such a message, each client will call juice_add_remote_candidate.

Yes that's right, but juice_gather_candidates() should actually be called immediately after sending the local description.

4. When the  on_gathering_done callback happens, the client signals this to the other via our existing server.

Yes, only this mechanism is optional. If you don't signal that gathering is done to the remote agent, it will timeout by itself after waiting for other candidates. Signaling that gathering is done only allows the remote agent to time out faster if the connection fails, it doesn't make a difference if the connection succeeds.

5. Both clients then wait for on_state_changed callback to signal JUICE_STATE_CONNECTED/JUICE_STATE_COMPLETED, or JUICE_STATE_FAILED. I'm not sure of the difference betweeen JUICE_STATE_CONNECTED and JUICE_STATE_COMPLETED... ?

The difference between CONNECTED and COMPLETED is an ICE concept. CONNECTED means connectivity has been established for a candidate pair, but connectivity checks are still performed to find a better one, while COMPLETED means connectivity has been established and no more checks are in progress.

Basically it means that CONNECTED and COMPLETED allow you to send, but the selected candidates (and therefore selected addresses and actual network path) might still change if not COMPLETED. For instance, the state would be CONNECTED if connection has been established through the relay but checks are still in progress to attempt a direct connection via hole punching.

6. If the state changes to JUICE_STATE_CONNECTED/JUICE_STATE_COMPLETED, then each client can start sending UDP packets to the other using the remote address for juice_get_selected_addresses. Do I need to use the juice_send commands for this, or can I just use my existing socket communication?

You must use juice_send(). The connection is only established for the local UDP port the agent is bound to, and the library needs to handle stuff internally like keepalives and tunneling in case the traffic is relayed through a TURN server. If the connection is direct, there won't be any header added to the packets, juice_send() sends the packet as UDP, so the only network overhead will be from keepalive packets.

7. The agent created by each client can now be destroyed via juice_destroy? Or does it need to be kept alive for the duration of the UDP session?

The agent must be kept alive for the duration of the session.

[rcvangemert]

Hi Paul-Louis,

Your work has been helpfull in understanding the subject.

I am trying to use libjuce in my free android app. I have multiple devices local descriptions in a firebase real-time database and am moving to the connect phase. The local descriptions have user and password attributes, is it a problem to reuse the local description of a remote device in later connection sessions. New sessions will have a different password etc. Or is the password only related to the inital connection to the stun server?

Thanks,
Robert

[rcvangemert]

more questions while I am working on this!

does juice_get_local_description just return a string of the local description? I am assuming juice_gather_candidates does the actual work?

Do I need to call juice_gather_candidates twice, the first time to get local descriptions and the second time after juice_set_remote_description with the sdp coming from the online database

Thanks again.

[paullouisageneau]

I am trying to use libjuce in my free android app. I have multiple devices local descriptions in a firebase real-time database and am moving to the connect phase. The local descriptions have user and password attributes, is it a problem to reuse the local description of a remote device in later connection sessions. New sessions will have a different password etc. Or is the password only related to the inital connection to the stun server?

You should not store SDP descriptions (or candidates). They are specific to a connection attempt, they have limited validity, and must never be reused. You should just forward each description to the remote peer as soon as possible when opening a connection. The username and password mechanism allows each agent to authenticate the remote agent when connecting.

does juice_get_local_description just return a string of the local description? I am assuming juice_gather_candidates does the actual work?

They do different things: juice_get_local_description() indeed gets the current local description while juice_gather_candidates() asynchronously populates it with local network candidates. There are two modes of operation here:

• Trickling: First get and send the local description (which won't have candidates) to the remote peer, then gather the candidates and send the local candidates to the remote peer as they arrive in the candidate callback.
• No trickling: First gather the candidates, and when gathering is finished (using the gathering done callback), get and send the local description (which will have candidates) to the remote peer (This is simpler but might introduce connection delays, see the notrickle test for code).

Do I need to call juice_gather_candidates twice, the first time to get local descriptions and the second time after juice_set_remote_description with the sdp coming from the online database

No, each peer must call it only once. The normal process is that the calling peer gathers candidates immediately when opening the connection (it sets the remote description after when it gets it) while the answering peer gathers candidates only after setting the remote description.

[rcvangemert]

OK, so how do I communicate with the remote agent to allow this process to occur? I need to know its address and port. Should I have s signaling server on both agents and interate over the local candidates held in dbf. Hopfully we will establish ccommunication and pass the required info.

I am now a little confused , the purpose of all this is to set up a communication path that is optimised for the network but we need to know the address/port to communicate on. We do not even know if the signaling server ports a reachable.

What am I missing?

Thanks for your help.

[paullouisageneau]

OK, so how do I communicate with the remote agent to allow this process to occur? I need to know its address and port. Should I have s signaling server on both agents and interate over the local candidates held in dbf. Hopfully we will establish ccommunication and pass the required info.

I am now a little confused , the purpose of all this is to set up a communication path that is optimised for the network but we need to know the address/port to communicate on. We do not even know if the signaling server ports a reachable.

The signaling server must of course not be at the remote agent, instead signaling must be done through an existing channel, typically a server acting as meeting point and able to relay the messages. Such a server may be part of an applicative backend also handling peer discovery (with contact lists, game rooms, etc). In practice, any protocol allowing bidirectional communication works (for instance WebSocket, HTTP, XMPP, but bare TCP would work too).

[thomashenry79]

Ok, many thanks. I guess this is actually going to be quite a lot harder than I thought then, as I would need to rework the transport layer of our existing libraries which currently use sockets, and somehow either adapt them to use the juice send/recieve.

[rcvangemert]

Thanks for your interest and libjuice, still pursuing using a firebase real-time database for signalling. Also trying to use libjuce in one direction only. I think this may work if I use juice_set_remote_description only on the send side. The process is as follows: Both users start the android app. Both sides generate a sdp using juice_get_local_description (like your gathering example). Both sides save there sdp's in a dbf using a device ID as a key. The sending side reads the remote sdp from the dbf using its device id and calls juice_set_remote_description with the remote sdp (like your notrickle example). It's assumed both users do not restart their apps so passwords and user names do no change. I am now getting a connected state on the sending app but the message is not received. To be continued...

…

On Fri, 7 Apr 2023, 00:38 Paul-Louis Ageneau, ***@***.***> wrote: OK, so how do I communicate with the remote agent to allow this process to occur? I need to know its address and port. Should I have s signaling server on both agents and interate over the local candidates held in dbf. Hopfully we will establish ccommunication and pass the required info. I am now a little confused , the purpose of all this is to set up a communication path that is optimised for the network but we need to know the address/port to communicate on. We do not even know if the signaling server ports a reachable. The signaling server must of course not be at the remote agent, instead signaling must be done through an existing channel, typically a server acting as meeting point and able to relay the messages. Such a server may be part of an applicative backend also handling peer discovery (with contact lists, game rooms, etc). In practice, any protocol allowing bidirectional communication works (for instance WebSocket, HTTP, XMPP, but bare TCP would work too). — Reply to this email directly, view it on GitHub <#192 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGSW3AIUXBQ3MRBFKGV32LW73IMRANCNFSM6AAAAAAWDS5ZG4> . You are receiving this because you commented.Message ID: ***@***.*** com>

[rcvangemert]

this is great, I am now receiving my messages and both ends can join in their own time. If I use my phone for connection (not wifi) it also works.

the next problem to sort out is when one of the agents disconnects I do not get a state change. I suspect the states are ice things as the state moves from connecting to connected and then to completed. Also if both agents don't connect I get a failure status but one-way communication continues ok.

any suggestions on this?

[rcvangemert]

I am also seeing STUN server communication (housekeeping?) after juice_set_remote_description. I thought STUN was only used during the gathering phase and your ICE code works out the path. I can send messages (WIFI) while the STUN server communication occurs. Communication by mobile data now does not work.

[paullouisageneau]

You are not using the library correctly. Both agents need to connect in parallel, that's how the ICE protocol works. You must forward the descriptions in both directions under a short enough time duration (ideally under 10s). If you don't do that, you will encounter connection failures depending on the presence of firewalls or NATs.

the next problem to sort out is when one of the agents disconnects I do not get a state change.

The library detects connection timeouts after 30s starting from version 1.2. Previous versions did not detect timeout, they only handled keeping the connection alive. In practice, what you get is raw UDP communication, so you should detect packet and connectivity loss at application level.

I am also seeing STUN server communication (housekeeping?) after juice_set_remote_description. I thought STUN was only used during the gathering phase and your ICE code works out the path.

The whole ICE protocol is actually based on STUN, the STUN server is only a small part of it, so you'll see STUN traffic between peers.

[rcvangemert]

Yes you are right, I'll re think my approach. Thanks.

…

On Mon, 10 Apr 2023, 20:27 Paul-Louis Ageneau, ***@***.***> wrote: You are not using the library correctly. Both agents need to connect in parallel, that's how the ICE protocol works. You must forward the descriptions in both directions under a short enough time duration (ideally under 10s). If you don't do that, you will encounter connection failures depending on the presence of firewalls or NATs. the next problem to sort out is when one of the agents disconnects I do not get a state change. The library detects connection timeouts after 30s starting from version 1.2. Previous versions did not detect timeout, they only handled keeping the connection alive. In practice, what you get is raw UDP communication, so you should detect packet and connectivity loss at application level. I am also seeing STUN server communication (housekeeping?) after juice_set_remote_description. I thought STUN was only used during the gathering phase and your ICE code works out the path. The whole ICE protocol is actually based on STUN, the STUN server is only a small part of it, so you'll see STUN traffic between peers. — Reply to this email directly, view it on GitHub <#192 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGSW3F5RUCGAVHT2OFTOLLXAPOBRANCNFSM6AAAAAAWDS5ZG4> . You are receiving this because you commented.Message ID: ***@***.*** com>

[rcvangemert]

hi, finally got back to libjuice and now have consistent behavior. Firebase real-time database works well for signaling and once I sorted out the different threads on the Ndk side it all started working. I have tested on Android phone data and wifi. Also tested it on Chromebook Android and ok. If you're an Android man and want the code as far just ask. Any suggestions on just setting up one-way communication with libjuice? Thanks.

[rcvangemert]

Hi again, On the assumption that juice_destroy(local_agent) will stop communication and clear all resources, I call juice_destroy(local_agent) when we do not need to communicate and recreate an agent when we want to re-establish another or the same connection. The same process is used for the Android pause/resume lifecycle.

Well, I get a thread error (ndk), and the app crashes. If I remove the juice_destroy(local_agent) and recreate the agent the fault does not occur. Will this process leak memory?

What is the recommended way to start/stop/restart juice?

[paullouisageneau]

Hi again, On the assumption that juice_destroy(local_agent) will stop communication and clear all resources, I call juice_destroy(local_agent) when we do not need to communicate and recreate an agent when we want to re-establish another or the same connection. The same process is used for the Android pause/resume lifecycle.

Your understanding is correct.

Well, I get a thread error (ndk), and the app crashes. If I remove the juice_destroy(local_agent) and recreate the agent the fault does not occur. Will this process leak memory?

Yes it will, as the agents will never be deallocated. If juice_destroy() crashes consistently, something is wrong. Do you have more information about the crash? By any chance, are you attempting to call it from one of libjuice's callbacks? You must not do that.

[rcvangemert]

I have restructured a few things and no longer have the crash. Sorry about this.
thanks for lib juice.

[jazzcake]

Hi, I'm developing a server application with webrtc, is it possible to connect to multiple clients using libjuice? I've prototyped a peer to peer connection with my own signaling server first, but I don't know how to handle multiple clients.

[paullouisageneau]

Yes it is possible, simply create a new agent for each client with juice_create(), then signal each agent independently.

[jazzcake]

Hello I have another question, in datachannel's UDP seem not have any implementation for reliable data transfer. Do I have to handle UDP packet for reliabilty if i need it?

[paullouisageneau]

libjuice only offers raw UDP transport. For reliability, congestion/flow control, or encryption, you need to add the proper layers on top. For instance libdatachannel does it for you by implementing WebRTC Data Channels.

[jazzcake]

Thank you and lib juice. :)