How can I get Peer Connection's packet loss information?

[DevRockstarZ]

I looked up all of the code, in webrtc/stats.go
I found

	// PacketsLost is the total number of RTP packets lost for this SSRC. Note that
	// because of how this is estimated, it can be negative if more packets are received than sent.
	PacketsLost int32 `json:"packetsLost"`

this code.
When I call peerconnection.GetStats(), but no information about packet loss.
How can I get packet loss information?
Thanks.

[r-novel]

Hello @DevRockstarZ!
Unfortunately, this functionality has not implemented at the moment. I guess it will be implemented in 3.1.0 version. (See roadmap).
You could implement some RTCP reader and extract packet loss info from Recv/Send Reports.
Also, you may create your own simple jitter buffer that provides calculate packet loss, parse RTCP NACK packets, and retransmit missed RTP packets.
You could explore some examples using these links:
https://github.com/pion/ion/blob/master/pkg/rtc/plugins/buffer.go
https://github.com/pion/ion/blob/master/pkg/rtc/plugins/jitterbuffer.go

[DevRockstarZ]

@r-novel Thanks for your great comment!
I'll check examples and find the way to get information about packet losses.
Thanks!

[DevRockstarZ]

@r-novel Sorry for re-comment in the closed issue.
I looked https://github.com/pion/ion/blob/master/pkg/rtc/plugins/buffer.go code,
but I only need Opus audio packet loss information. There's only for vp8,vp9 and h264 support.
Do I have to implement by myself? Or, audio packet loss doesn't need to be calculated?
Thanks.

[r-novel]

@DevRockstarZ Hello!
Sure! You can do that. You may change clock rate from 90000 to 48000. ( RTP Opus rfc )
Calculate/retransmit logic audio packets has the same logic that video.

[DevRockstarZ]

@r-novel Thanks for your great comment!
This is my final question, when I use RTP,buffer.go implement will be works fine?
And buffer / jitter buffer has a lot of differences when I want to know packet losses?
Thanks a lot and have a nice day! ;D

[r-novel]

@DevRockstarZ Hello! Yeah, buffer.go works with RTP packets. You could read data from pc.track, unmarshaling to pion/rtp Packet, and send to buffer.
You may define counter variable and increment by lostPkt here for example.

Generally, the algorithm looks like this:

1. read data or RTPpacket;
2. push to buffer;
3. calculate NACK and lostPkt every NACKWindowSize (17)
4. optional counting PktRecv and calculate packet loss percent: (lostPkt / RecvPkt + lostPkt)
5. Sent NACK RTCP message to the sender (for example browser or another Pion-based app)

Also, you can look at amazing Pion-based app Peer-Calls:
I hope, it will help you.

[DevRockstarZ]

@r-novel Thanks for your comment!
And this is my real last question, can I use jitter buffer(buffer.go) which you linked (Peer-Calls) when I use RTP packet only?
Sorry for many questions. Thanks!

[r-novel]

@DevRockstarZ Hello! Sure! You could use only RTP packets for counting packet lost; You may do a few changes for removing rtcp.Nack create and send logic from your jitter buffer implementation.
RTCP NACK needs if you want to retransmit lost packets.

If you are interesting you could read here more about RTCP NACK feedback and packet retransmit.

[DevRockstarZ]

@r-novel Thanks for your fast comment.
It really helped me a lot!
Have a nice day! ;D

[r-novel]

@DevRockstarZ I'm really happy if my comments were useful.
Good luck and have a nice day! 😃

[uharband]

@r-novel hi and sorry if i'm repeating parts of the question above.. i'm new to pion. trying to achieve something similar to the rtwatch app: stream an existing file / other source via pion to browser. now, i see on the browser side, using the chrome://webrtc-internals, that i have packets lost over time. also, during loses video freezes shortly. usually, in a full webrtc browser-to-browser the packets will get retransmitted using the NACKs or other mechanisms. I do see there is a RTCPFeedback struct but doesn't look like media engine or any codec uses this. are you saying the basic retransmit of lost packets is not implemented yet but planned for 3.1.0? if yes, is there any other error correction which is implemented at this point which i can 'turn on' to avoid packet loss?
thanks a lot!

[r-novel]

@uharband Hello! If I understand correctly, you streaming from static file (from you FS) through Pion app to Browser?
You could try to implement a jitter buffer where you RTP packets can be stored before transmitting to the browser. When your browser recognizes packet loss it sends NACK packet to Pion-based server.
You need to intercept these incoming RTCP packets, parse nack messages, find packets into jitter-buffer and retransmit them.

Simple algo below:

1. read_data_from_file() -> packetize() -> push_to_buffer(rtp) -> pop_from_buffer() *rtp -> sendToBrowser(rtp)
2. rtcpRead() -> is_nack_found -> parseNack() -> lookup_rtp_pkts() *[]rtp -> retransmit([]rtp)

If you cast video via GStreamer or FFmpeg you may try to tune some settings.
Once, I remember, long ago, there was something similar: I've used FFmpeg for streaming and was getting many freezes and packet loss.

What about RTCPFeedback:
You could use it before data exchange, during first register codecs in your application.

For example:

rtcpFb := []webrtc.RTCPFeedback{
     webrtc.RTCPFeedback{
	Type: webrtc.TypeRTCPFBGoogREMB,
     },
     webrtc.RTCPFeedback{
	Type:      webrtc.TypeRTCPFBNACK,
        Parameter: "pli",
     },
      webrtc.RTCPFeedback{
	Type: webrtc.TypeRTCPFBNACK,
      },
}
codec = webrtc.NewRTPVP8CodecExt(pt, 90000, rtcpFb , "")
mediaEngine.RegisterCodec(codec)

I hope my answer will be useful to you.

Best regards, Roman.

[uharband]

hi @r-novel
thanks so much for the lead.
i'm not finding how to read the rtcp packets coming in from the other peer.. (the other peer is a simple browser). i see the peerconnection has a WriteRTCP method, no ReadRTCP.. how do i connect to the incoming rtcp stream?
does that somehow have to do with the RTCFeedback?
thanks again,
Uzi

[uharband]

for the sake of future reference to getting nack packets. assuming you have one sender
i was looking to get feedback on the sender channel (from pion to browser). the RTCP packets are in the sender, not the receiver

sender := peerConnection.GetSenders()[0]  
packets, _ := sender.ReadRTCP()

// make sure this packet is a nack one  
nackPacket, ok := packet.(*rtcp.TransportLayerNack)  

if ok{  
// nack logic  
}