RTMP Resolution Probe Issue

[jspizziri]

This is a follow-up to the several comments I left here.

It appears that there might be an issue with the _probe function regarding resolution.

1. I'm publishing an RTMP stream from OBS to an Nginx proxy.
2. I'm attempting have shaka-streamer transcode the RTMP stream.
3. I'm using the exact input config specified here with the exception of the actual URL to the RTMP stream.

Following these steps results in the following error:

root@2e5132cbe21d:/usr/src/app# python3 shaka-streamer -i config_files/input_rtmp_config.yaml -p config_files/pipeline_live_config.yaml -o /usr/src/app/output
+ /usr/local/lib/python3.9/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=index -of compact=p=0:nk=1
+ /usr/local/lib/python3.9/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=field_order -of compact=p=0:nk=1
+ /usr/local/lib/python3.9/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=avg_frame_rate -of compact=p=0:nk=1
+ /usr/local/lib/python3.9/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=width,height -of compact=p=0:nk=1
Fatal error:
  Input is missing a required field: resolution, a VideoResolution name (one of '144p', '240p', '360p', '480p', '576p', '720p', '720p-hfr', '1080p', '1080p-hfr', '1440p', '1440p-hfr', '4k', '4k-hfr', '8k', '8k-hfr')

The above error is the result of a failed attempt to autodetect the resolution. The ffprobe command that's being logged is the following:

/usr/local/lib/python3.9/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=width,height -of compact=p=0:nk=1

So presumeably this command is not returning the width and height of the stream. I did some debugging, and python is returning None for the actual result of the probe call. However, if you run that ffprobe command directly on the stream you do in fact get a correct result:

root@2e5132cbe21d:/usr/src/app# /usr/local/lib/python3.9/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=width,height -of compact=p=0:nk=1
ffprobe version n4.4 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --pkg-config-flags=--static --disable-ffplay --enable-libvpx --enable-libaom --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-runtime-cpudetect --enable-gpl --enable-version3 --enable-static --enable-nvenc --enable-vdpau
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, flv, from 'rtmp://nginx-rtmp/live/obs_stream':
  Metadata:
    |RtmpSampleAccess: true
    Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
    displayWidth    : 1280
    displayHeight   : 720
    fps             : 30
    profile         :
    level           :
  Duration: 00:00:00.00, start: 1225.943000, bitrate: N/A
  Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 163 kb/s
  Stream #0:1: Video: h264 (High), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 2560 kb/s, 30 fps, 30 tbr, 1k tbn, 60 tbc
1280|720

To further confirm that this was the problem I hardcoded the resolution in my input yaml file and my RTMP stream started transcoding the way I expected (given I had bypassed the resolution autodetect).

inputs:
  - name: rtmp://nginx-rtmp/live/obs_stream
    media_type: video
    # hardcode resolution to avoid autodetection
    resolution: 720p

  - name: rtmp://nginx-rtmp/live/obs_stream
    media_type: audio

I've done some debugging and my guess is that the issue is related to how the ffprobe command is actually invoked here.

Going to do some more digging but any guidance would be helpful!

[joeyparrish]

Well done tracking this down! The probe could fail if the process exit code is non-zero, or if the string output can't be parsed. I would investigate those two things next.

[jspizziri]

@joeyparrish thanks for the input.

TL;DR;

I've done some more digging and it looks like this is related to having too short of an analyzeduration on ffprobe (which defaults to 5s). Passing a higher threshold for the analyzeduration solves the issue.

How would you like to approach this solution? Perhaps provide an option on the input to control the analyzeduration? Or detect if it's an RTMP stream and increase it?

Investigation

While digging into it I noticed that back-to-back calls to ffprobe to evaluate the resolution would randomly succeed or fail. Below are the results of two such probe calls that yielded different results:

Proper Result

root@578709af1cc9:/usr/src/app# result=$(/usr/local/lib/python3.11/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=width,height -of compact=p=0:nk=1)
ffprobe version n4.4 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --pkg-config-flags=--static --disable-ffplay --enable-libvpx --enable-libaom --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-runtime-cpudetect --enable-gpl --enable-version3 --enable-static --enable-nvenc --enable-vdpau
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, flv, from 'rtmp://nginx-rtmp/live/obs_stream':
  Metadata:
    |RtmpSampleAccess: true
    Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
    displayWidth    : 1280
    displayHeight   : 720
    fps             : 30
    profile         :
    level           :
  Duration: 00:00:00.00, start: 686.722000, bitrate: N/A
  Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 163 kb/s
  Stream #0:1: Video: h264 (High), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 2560 kb/s, 30 fps, 30 tbr, 1k tbn, 60 tbc
root@578709af1cc9:/usr/src/app# echo $result
1280|720

⚠️ Note the Stream #0:1: Video: h264 ... 1280x720 ...

No Result

root@578709af1cc9:/usr/src/app# result=$(/usr/local/lib/python3.11/site-packages/streamer_binaries/ffprobe-linux-x64 rtmp://nginx-rtmp/live/obs_stream -select_streams v:0 -show_entries stream=width,height -of compact=p=0:nk=1)
ffprobe version n4.4 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --pkg-config-flags=--static --disable-ffplay --enable-libvpx --enable-libaom --enable-libx264 --enable-libx265 --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-runtime-cpudetect --enable-gpl --enable-version3 --enable-static --enable-nvenc --enable-vdpau
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, flv, from 'rtmp://nginx-rtmp/live/obs_stream':
  Metadata:
    |RtmpSampleAccess: true
    Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
    displayWidth    : 1280
    displayHeight   : 720
    fps             : 30
    profile         :
    level           :
  Duration: 00:00:00.00, start: 717.292000, bitrate: N/A
  Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 163 kb/s
root@578709af1cc9:/usr/src/app# echo $result

⚠️ Note the echo $result command returned nothing and the absence of the Stream #0:1: Video: h264 ... 1280x720 ....

To me it's still quite strange that these calls failed given that the Metadata properties logged still displays displayWidth : 1280 & displayHeight : 720 in all the calls I make regardless of the analyzeduration. So something is clearly aware of the stream being 720p.

[jspizziri]

@joeyparrish friendly ping on this. Happy to submit a PR, just need to know which option you'd want to go with (see above).

[joeyparrish]

Thanks for the ping!

Is there a down side to passing a 5s analyzeduration by default?

If you think we might need a configurable duration, could we have a default that works for both RTMP and regular files?

You have spent more time on this than me by roughly infinity percent, so I'm happy to review a PR with whatever solution you feel is best. Just explain the choice in the PR description, for posterity and for ease of review.

Thanks!

[jspizziri]

@joeyparrish the current default is 5s so we'd need to set it to something like 10s.

The potential downsides to just increasing the default value:

1. It would increase the overall transcode time for all cases, but a few seconds is probably a drop in the bucket, so maybe not a big deal.
2. I'm not exactly sure if there's a "one-size-fits-all" default for RTMP. Its possible that different streams would require a different amount of time, and I'm not sure what that maximum for all cases value would be. I'm pretty new to RTMP (as in figuring it out actively), but this to me might be the bigger of the two problems.

I think I'd lean towards having a default that could be overridden via pipeline config. I'll send a PR to that effect unless you respond with more thoughts on the matter.

Thanks!

[joeyparrish]

Sounds good. My philosophy is generally to offer a config where it's needed, and to try to set a default that will work for most people most of the time, so that it isn't strictly required to set it.