SendAsync Behaviour

[g-tejas]

Problem: I am trying to use CUDPSender::SendAsync instead of Send. However, there's a few obstacles I've ran into and I want to ask if there's anything I'm missing.

In snd_sample.cpp, I noticed that TransmitToUDP is hardcoded to Send and not SendAsync so I had a naive modification of just changing it to SendAsync, however SendAsync does not return anything, so for now it just returns len_. This probably will not work.

size_t TransmitToUDP(const void* buf_, const size_t len_, const std::shared_ptr<eCAL::CUDPSender>& sample_sender_, const std::string& mcast_address_)
  {
    //return (sample_sender_->Send(buf_, len_, mcast_address_.c_str()));
    sample_sender_->SendAsync(buf_, len_, mcast_address_.c_str());
    std::cout << "TransmitToUDP() - returning len_ = " << len_ << std::endl;
    return (len_);
  }

I also noticed that the member variable m_payload in SampleSender is the buffer being sent, so I was wondering what's the behaviour if there were back to back SendAsync calls, wouldn't the second async call overwrite the m_payload from the "CreateSampleBuffer" invocation? If that's the case, doesn't boost asio require the caller of async_send_to to ensure that this buffer remains valid till it's actually sent, so the second async call would invalidate the first one if it was before it was sent.

I also have another question regarding the iocontext, from what I understand, this recursively calls the SendAsync till the msg is fully sent, but wouldn't this call iocontext.stop multiple times? And iocontext.run() is only invoked once (within the SendAsync with 3 params).

void CUDPSenderImpl::OnSendAsync(asio::error_code ec_, std::size_t bytes_transferred_, asio::const_buffer buf_, const char* ipaddr_)
  {
    if (!ec_)
    {
      // bytes_transferred_ amount of data was successfully sent
      auto bytes_left = buf_ + bytes_transferred_;
      if (bytes_left.size())
      {
        // send "rest data" in the next SendAsync operation
        SendAsync(bytes_left, ipaddr_);
      }
      m_iocontext.stop();
    }
    else
    {
      std::cout << "CUDPSender::OnSend failed with: \'" << ec_.message() << "\'" << std::endl;
    }
  };

Our goal is to send async and want to know what's the proper way to go about implementing this for UDP?

[FlorianReimold]

Hi @g-tejas,

That function is not part of the public API, it is part of the internal implementation of the eCAL UDP Layer. Currently, it is not used. I don't see a way how you can access it. I also don't really see a reason why you would want to access it.

Have you tried the basic eCAL Samples, that demonstrate how to transmit data via eCAL? https://eclipse-ecal.github.io/ecal/getting_started/hello_world.html

If so, what exactly are you trying to achieve?

[g-tejas]

I am building eCAL from source, to use with our driver. The reason why is because we are running into blocking issues with the Send (non-async version) when transmitting images over wifi. We prefer to use UDP multicast than tcp. When there's a subscriber on a different host which triggers udp mc, we profiled our code and noticed that the send blocks for like 100ms, which then causes further problems w our driver. And looking through the code we noticed that the send was synchronous and blocks.

[rex-schilasky]

Hi @g-tejas,

the async functionality was probably forgotten to remove and comes from an early prototype. It's not tested and we will remove this next time. Currently all eCAL transport layer work synchronously for good reasons. We want to ensure that if a publisher send call returns all data are transferred to the connected layers (shm, udp, tcp).
If your special use case has some specific issues with the udp socket you try to get the root cause of this behavior. And if you REALLY want to do asynchronous sending you could place your send logic in it's own thread.

What do you mean ?

[rex-schilasky]

For udp testing you may also use this small test repository https://github.com/rex-schilasky/udp_test

[g-tejas]

Ah, thank you for the detailed reply. I'll use the method you suggested (send thread) for now, it seems to be working.

[FlorianReimold]

OK, now I understand what you are trying to achieve.

So my first question would be: Is it your goal to work on eCAL and improve it? I am asking because you dug up that internal function and tried to use it.

I usually assume that you are a user of eCAL. So let's see how your actual problem can be solved.

• You can simply implement your own send queue. So you have 1 sender thread that gets woken up (e.g. via a std::condition_variable) whenever another thread has put something in its queue. It send sends it out to eCAL. Putting data in that queue will be very fast and the data is then sent in the background
• You really should reconsider your preference for UDP multicast. You are putting the layer in the worst possible environment that is imaginable. We already know that it will probably perform badly:
  • You already saw one issue: the blocking behavior
  • Big messages (and images usually qualify as big) lead to a high fragmentation. If one fragment is lost, the entire message is lost
  • WiFi connections are never perfect. You will occasionally lose a package here and there. You will always have a hard time transmitting big UDP multicast packages over WiFi
  • The TCP layer on the other hand:
    • has a non-blocking behavior
    • has congestion avoidance by design (-> TCP)
    • retransmits fragments by design (-> TCP), so losing some fragments isn't an issue at all

[g-tejas]

Thank you for the detailed reply! I've gone ahead and implemented the method you suggested, and it seems to be working now! As for the UDP multicast advice, that makes sense, our main use case is not over wifi but rather ethernet, so i guess this works for now, i shall look into using TCP instead, thanks!