eCAL to work with multiple NIC on same host?

[chengguizi]

Hi eCAL team,

I recently find out that it is not trivial to setup a machine that has two NIC, say eth0 and eth1 and configure to cloud mode and receive data from both interfaces.

The reason being that, the standard instruction of setting 239.0.0.0/24 route will make the routing table only route to 1 NIC. There seems no good way in linux to make a route to send multicast to multiple NIC at the same multicast address.

For now, a workaround we found is that, to start ecal node on host using 239.0.0.0 cloud mode. Then modify the ecal.ini, change the route to 239.0.1.0. Then start the second ecal node.

On the other ends of the NICs, the ecal nodes are configured with 239.0.0.0 and 239.0.1.0 respectively.

Is there a better way to do this?

It would be great if eCAL ini can configure multicast address per NIC. So to receive from either NIC.

[chengguizi]

A related question. Is it possible to let ecal node to use a custom eacl.ini, instead of the system path one?

[FlorianReimold]

eCAL is socket-based, not NIC-based

eCAL is socket-based, i.e. it uses the socket API provided by the operating system. eCAL has no idea about what the operating system does with the data after it was passed to the socket. So you will have to achieve your goal by means of the operating system.

different ecal.ini files

Yes, you can use individual ecal.ini files for individual applications by putting them in separate directories and starting your eCAL applications with differen $ECAL_DATA variables. You can read more about that in the documentation:
https://eclipse-ecal.github.io/ecal/configuration/options.html

But to summarize it: eCAL will load the $ECAL_DATA/ecal.ini file, if the $ECAL_DATA variable ist set.

Using a network bridge

To me it sounds like you actually want to have a proper network with a central switch.

• If you can incorporate a physical switch in your network, this should be easy. You can then set your multicast route to only one NIC and let the switch relay the data to all connected machines.
• If you cannot connect everything to a physical switch, Linux has utilities to create a virtual switch (-> a network bridge). With this technique you can create a new virtual adapter (often named br0) and set the multicast route to that bridge. You can then assign the bridge to bridge between eth0 and eth1. With this technique, both of your connected devices can also see each other and communicate via eCAL, as the br0 bridges between both physical NICs. If that is not desired, you can apply package filters and e.g. drop all traffic that would be routed from eth0->eth1 and eth1->eth0.
  If this is your desired strategy, you should check out the brctl manpage: https://linux.die.net/man/8/brctl

[chengguizi]

Thanks for the excellent and quick answer! The virtual switch / network bridge seems the right abstration to go. Will try that next.

Interested to understand more about eCAL being socket-based. I understand at large eCAL has monitoring layer which is UDP socket based, and data layer which can be UDP or TCP. Is the data layer in UDP mode, uses exactly the same OS API to the monitoring layer? I do experience sometime that i could see the message in eCAL monitor, but UDP receiver shows nothing. (probably a one-way comm is block?)

In cloud mode, multicast group is expected to be setup, and eCAL has to be aware of the multicast group in its eCAL.ini . How to interpret the fact that eCAL is socket based only? Is it saying that the multicast is just the mean in practice to get the UDP packets to arrive at the correct port on the local machine, from the network?
Then why does eCAL has to know the multicast group? It is to filter out the message on that port? Or maybe when we open a UDP socket, the OS's API asks explicitly if the program wants to join a specific multicast?

[FlorianReimold]

Is the data layer in UDP mode, uses exactly the same OS API to the monitoring layer?

Monitoring & Registration Layer:

• UDP Multicast by default
• SHM when specifically enabled in the ecal.ini.

Data Layer:

• Network communicaiton:
  • UDP Multicast by default
  • TCP if enabled (in ecal.ini or via C++ API)
• Local communication:
  • SHM by default
  • UDP or TCP if enabled (in ecal.ini or via C++ API)

As the UDP monitoring/registration layer and the UDP data layer both use the same technique, the both use the same API. We are using asio (boost::asio, but we use it standalone without boost) as an API abstraction for all network communication.

I do experience sometime that i could see the message in eCAL monitor, but UDP receiver shows nothing. (probably a one-way comm is block?)

Yes, that probably is due to one-way communication. For a successful data transmission, the communcation must work both ways (the subscriber has to send to the publisher that it is interested in the data!)

How to interpret the fact that eCAL is socket based only?

I am not sure what your question is about. But sockets are a way ob abstracting from the physical layers and the device drivers. Sockets exist in basically all operating systems since the 90ties. On Windows it's Winsocks2, on Linux it is a POSIX Socket API. What else would one use for network communication?

Is it saying that the multicast is just the mean in practice to get the UDP packets to arrive at the correct port on the local machine, from the network?

UDP Multicast is a technique to send a datagram to multiple receivers at once. The data is only sent once and physically replicated by routers and switches for every receiver. If an application wants to receive multicast data, it has to create a socket and join the appropriate multicast group.

Then why does eCAL has to know the multicast group? It is to filter out the message on that port?

eCAL has to know where to send the data to. It also has to know the multicast group that other applications will send their data to, in order to join that group and receive the data.
The network stack of the operating system takes care of which data goes to which socket.

Or maybe when we open a UDP socket, the OS's API asks explicitly if the program wants to join a specific multicast?

The OS API doesn't "ask" that, no. It is an API function that the application needs to call itself.

[chengguizi]

Very insightful. Yes, the two way comm is broken on a simple ip route append attempt. We will try to solve it better next time, probably using bridge is the easiest and cleanest. Thanks!