Audio Delay Offset for Recordings

[PablaV]

First off want to say thanks for Frigate, it is truly a remarkable NVR and the continued work on it is very much appreciated! I am running it with my Reolink Doorbell POE and like others, have noticed the audio in recordings is about 3 seconds behind the video. The audio is in sync when viewing the live stream though. I was wondering if a configuration variable could be added to offset the audio for recordings.

[NickM-27]

this is something that is handled by ffmpeg, so you can try to configure it already if you want to, but I don't see this issue with my reolink doorbell so I would guess it is something with the way the doorbell is configured

[PablaV]

Would any changes made also affect live view audio? My live view audio is in sync just not recordings

[NickM-27]

not necessarily, live and recordings are saved / viewed differently

[PablaV]

Before I go down the rabbit hole of figuring out what ffmpeg args to use do you mind checking out my config and seeing if anything in there may be adding the audio delay since you aren't having one?

mqtt:
  host: 192.168.8.85
  port: 1883
  user: xxxx
  password: xxxx
cameras:
  front_doorbell:
    ffmpeg:
      inputs:
      - path: rtsp://127.0.0.1:8554/front_doorbell?video=copy&audio=aac
        input_args: preset-rtsp-restream
        roles:
        - record
      - path: rtsp://127.0.0.1:8554/front_doorbell_sub?video=copy
        input_args: preset-rtsp-restream
        roles:
        - detect
    motion:
      mask:
      - 640,0,640,29,322,27,322,0
      - 475,0,539,53,466,146,132,140,155,0
go2rtc:
  streams:
    front_doorbell:
    - ffmpeg:http://192.168.8.186/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=xxxx&password=xxxx#video=copy#audio=copy#audio=opus
    - rtsp://xxxx:xxxx@192.168.8.186:554/h264Preview_01_sub
    front_doorbell_sub:
    - ffmpeg:http://192.168.8.186/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=xxxx&password=xxxx

  webrtc:
    candidates:
    - 192.168.8.85:8700
    - stun:8555
  ffmpeg:
    bin: ffmpeg
    volume: "-af \"volume=30dB\""
objects:
  track:
  - person
  - car
  - dog
  - cat
  filters:
    car:
      min_area: 6000
    person:
      min_area: 2600
record:
  enabled: true
  events:
    pre_capture: 30
    post_capture: 30
    retain:
      default: 3
      mode: active_objects
motion:
  mask: 640,0,640,29,322,27,322,0
detect:
  max_disappeared: 200
  # width: 640
  # height: 480
detectors:
  coral:
    type: edgetpu
    device: usb
  # cpu1:
  #   type: cpu
  #   num_threads: 4
ffmpeg:
  output_args:
    record: preset-record-generic-audio-aac

[conorlap]

I also experience this with my Reolink doorbell!

[nwish]

I’m experiencing an audio delay as well with my Reolink POE doorbell, but mine seems pretty hard to pin down. I’m running go2rtc and connecting to the RTSP stream only, which for me seems the most stable.

I have found that the audio delay sometimes appears during live view on the HA Frigate card, the Frigate interface, recordings, and the Reolink app itself. Sometimes one of them is off, sometimes they all are. I did just get a new router so maybe that’ll help.

Also, a reboot of the doorbell seems to bring everything back into sync for at least a few days. I would make sure you have the latest firmware as well!

Anyway, I’ll report back if I find anything else!

[PablaV]

Sounds like we have multiple people having this issue. I have 3 separate installations of HA+Frigate+Reolink Doorbell one is POE the other two are wifi models. All three are having the same issues with audio delays. They're also complete separate installations ie different houses with different networking gear.

I still am looking into how to tweak this with FFMPEG.

[ourkid]

I also have a Reolink PoE Video Doorbell and experience audio delays of about 1.5 seconds when viewing recordings of events. I am using go2rtc to proxy the http stream as recommended in the camera specific guide. If I view the Reolink Doorbell camera stream directly in VLC, there is no delay in the audio. I have the latest firmware and it is the latest hardware version as well.

[NickM-27]

https://www.reddit.com/r/reolinkcam/comments/15h2kar/comment/jx3202g/?utm_source=share&utm_medium=mweb3x&utm_name=mweb3xcss&utm_term=1&utm_content=share_button

[ourkid]

Thanks for the link. I have installed the August firmware, but it did not make any difference. Do you have any other suggestions that I can try?

[ourkid]

The new Reolink firmware (Feb '24) seems to improve the lag by quite a bit

[NickM-27]

Unfortunately it breaks two way talk

[PablaV]

Yeaup new FW breaks 2 way and doesn't fix lag would not recommend upgrading

[kanemari]

this issue does exist with more than just the Reolink. I have the same issue with a hard wired intercom (Hikvision). Its not out of sync when using the hikvision app, and I would expect the video to be slightly out of sync using go2rtc (which it is) but the audio is way out of sync, like 3-5 seconds. I am copying the original and re-encoding 2 types of audio (aac + opus) to try and increase compatibility with 3rd party viewers (mainly google home, which will display a live preview but wont connect when you click on the camera - but I digress) - but my CPU usage on the HA VM as a whole never goes over 50% and the disk IO delay doesnt go over 1-2% either. I did notice that HAOS seems to kill ffmpeg quite often due to memory constraints - but the VM never gets close to using all its memory either so maybe its a docker limitation?

anyway main thing, its not just Reolink so pls dont dismiss it as that.

[kanemari]

after a bit of tinkering Im starting to think this is a problem with the ffmpeg module in go2rtc. I initially had to use the ffmpeg module for my Wyze cams because of audio issues, but since I started using docker-wyze-bridge I was able to use the rtsp stream directly and the audio delay disappeared. That prompted me to mess around with the config of my other cams to change the native audio options and not rely on ffmpeg, which removed the audio delay from them as well. so in my case at least, the ffmpeg module doesnt just cause latency, it also causes massive audio delay.

[christopherapostoli]

kanemari, can you please share your frigate config file code? I have Wyze cams with Frigate and would like to compare. Thanks!

[kanemari]

@christopherapostoli - I changed it again since the last post because my v2 cam died during a firmware upgrade - so in the process of replacing it with a v3 I moved back to wyze-mini-hacks from docker-wyze-bridge. go2rtc runs on the cams themselves outputting aac audio into frigate, which saves me needing to reencode the audio to record clips. its not perfect, but its a relatively stable sub 1 second audio delay. its only really obvious when its a sharp sound like a car door closing

relevant frigate config is below:-

go2rtc:
  streams:
    garage:
      - rtsp://xxx:xxx@192.168.20.xx:8554/1080p?mp4
      - "ffmpeg:garage#audio=opus#audio=copy"
---
cameras:
 garage:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage
          input_args: preset-rtsp-restream-low-latency
---
ffmpeg:
  input_args: preset-rtsp-restream-low-latency
  output_args:
    detect: -avioflags direct -threads 1 -fps_mode vfr -muxdelay 0 -flags low_delay -flush_packets 1 -f rawvideo -pix_fmt yuv420p
    record: preset-record-generic-audio-copy

[ourkid]

In the end I removed the Reolink recording stream from go2rtc completely, using the rtsp stream instead, and the audio delay has disappeared. I know that the camera specific guide recommends the http stream for stability (via go2rtc) but it has been working fine for me for about 2 weeks now. Perhaps Reolink have cleared up their rtsp stream issues (for the PoE Doorbell at least). I still use the sub-stream via go2rtrc for detection.

[christopherapostoli]

Thank you! This was very helpful :)

[PablaV]

In the end I removed the Reolink recording stream from go2rtc completely, using the rtsp stream instead, and the audio delay has disappeared. I know that the camera specific guide recommends the http stream for stability (via go2rtc) but it has been working fine for me for about 2 weeks now. Perhaps Reolink have cleared up their rtsp stream issues (for the PoE Doorbell at least). I still use the sub-stream via go2rtrc for detection.

Would you mind sharing your config? I tried using the RTSP stream and it still has lag

[ourkid]

Note that I am using the latest firmware as well

go2rtc:
  streams:
    frontdoor_sub: 
      - "ffmpeg:http://192.168.0.zzz/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=uuuu&password=pppp"

cameras:
  frontdoor:
    birdseye:
      enabled: true
      mode: continuous
      order: 0
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      hwaccel_args: preset-vaapi
      inputs:
        - path: rtsp://uuuu:pppp@192.168.0.zzz:554/h264Preview_01_main
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/frontdoor_sub?video=copy
          input_args: preset-rtsp-restream
          roles:
            - detect