ffmpeg.md

Using VMAF with FFmpeg

After installing libvmaf, you can use it with FFmpeg. Under the FFmpeg directory, configure, build and install FFmpeg with:

./configure --enable-libvmaf
make -j4
make install

Using FFmpeg+libvmaf is very powerful, as you can create complex filters to calculate VMAF directly on videos of different encoding formats and resolutions. For the best practices of computing VMAF at the right resolution, refer to our tech blog.

We provide a few examples how you can construct the FFmpeg command line and use VMAF as a filter. Note that you may need to download the test videos from vmaf_resource.

Below is an example on how you can run FFmpeg+libvmaf on a pair of YUV files. First, download the reference video src01_hrc00_576x324.yuv and the distorted video src01_hrc01_576x324.yuv. -r 24 sets the frame rate (note that it needs to be before -i), and PTS-STARTPTS synchronizes the PTS (presentation timestamp) of the two videos (this is crucial if one of your videos does not start at PTS 0, for example, if you cut your video out of a long video stream). It is important to set the frame rate and the PTS right, since FFmpeg filters synchronize based on timestamps instead of frames.

The log_path is set to standard output /dev/stdout. It uses the model_path at location /usr/local/share/model/vmaf_float_v0.6.1.json (which is the default and can be omitted).

ffmpeg -video_size 576x324 -r 24 -pixel_format yuv420p -i src01_hrc00_576x324.yuv \
    -video_size 576x324 -r 24 -pixel_format yuv420p -i src01_hrc01_576x324.yuv \
    -lavfi "[0:v]setpts=PTS-STARTPTS[reference]; \
            [1:v]setpts=PTS-STARTPTS[distorted]; \
            [distorted][reference]libvmaf=log_fmt=xml:log_path=/dev/stdout:model_path={your_vmaf_dir}/model/vmaf_v0.6.1.json:n_threads=4" \
    -f null -

The expected output is:

[libvmaf @ 0x7fcfa3403980] VMAF score: 76.668905

Below is a more complicated example where the inputs are packaged .mp4 files. It takes in 1) a reference video Seeking_30_480_1050.mp4 of 480p and 2) a distorted video Seeking_10_288_375.mp4 of 288p upsampled to 720x480 using bicubic, and compute VMAF on the two 480p videos. Bicubic is used as the recommended upsampling method (also see the techblog for more details).

ffmpeg \
    -r 24 -i Seeking_30_480_1050.mp4 \
    -r 24 -i Seeking_10_288_375.mp4 \
    -lavfi "[0:v]setpts=PTS-STARTPTS[reference]; \
            [1:v]scale=720:480:flags=bicubic,setpts=PTS-STARTPTS[distorted]; \
            [distorted][reference]libvmaf=log_fmt=xml:log_path=/dev/stdout:model_path={your_vmaf_dir}/model/vmaf_v0.6.1.json:n_threads=4" \
    -f null -

The expected output is:

[libvmaf @ 0x7fb5b672bc00] VMAF score: 51.017497

See the FFmpeg's guide to libvmaf, the FFmpeg Filtering Guide for more examples of complex filters, and the Scaling Guide for information about scaling and using different scaling algorithms.

Note about the model path on Windows

Due to Windows not having a good default for where to pull the VMAF model from, you will always need to specify model_path when calling libvmaf through ffmpeg. However, you will need to be careful about the path you pass to model_path.

If you are using a relative path for your model_path, you can completely ignore this whole section, else if you are trying to use an absolute Windows path (D:\mypath\vmaf_v0.6.1.json) for your model_path argument, you will need to be careful so ffmpeg passes the right path to libvmaf.

The final command line will depend on what shell you are running ffmpeg through, so you will need to go through the following steps to make sure your path is okay.

1. Convert all of the backslashes \ to forward slashes / (D:/mypath/vmaf_v0.6.1.json)

2. Escape the colon : character by using a backslash \ (D\:/mypath/vmaf_v0.6.1.json)

3. Then escape that backslash with another backslash (D\\:/mypath/vmaf_v0.6.1.json)

4. The next step will depend on the shell that will run ffmpeg:

   • For PowerShell and Command Prompt, this will be enough and your final ffmpeg command line will look something like

   ./ffmpeg.exe -i dist.y4m -i ref.y4m \
       -lavfi libvmaf=model_path="D\\:/mypath/vmaf_v0.6.1.json" \
       -f null -

   Note: I only quoted the path part for trivial reasons and in this specific case, it can be unquoted or you can quote the whole part after lavfi starting from libvmaf to json and it should give the same result due to neither shell treating the \ as a special character

   • For bash or specifically msys2 bash, it has some additional considerations. The first thing to know is that bash treats the backslash character \ a bit special in that it's an escape character normally when not put inside single quotes. The second thing to know is that msys2's bash attempts convert a posix-like path (/mingw64/share/model/vmaf_v0.6.1.json) to a Windows mixed path (D:/msys2/mingw64/share/model/vmaf_v0.6.1.json) when passing arguments to a program. Normally, this would be fine, however, in our case, this works against us since we cannot allow it to convert the path to a normal path with an un-escaped colon. For this, we will need to not only escape the escaped backslash, but we will also need to pass the MSYS2_ARG_CONV_EXCL environment variable with the value of * to make sure it doesn't apply that special conversion on any of the arguments

   MSYS2_ARG_CONV_EXCL="*" \
       ./ffmpeg.exe -i dist.y4m -i ref.y4m -lavfi \
       libvmaf=model_path="D\\\:/mypath/vmaf_v0.6.1.json" -f null -

   Note: in this case, the quotes are not as trivial as the PowerShell/cmd version, as removing the quotes entirely will require you to re-escape the backslash resulting in 4 total backslashes, but quoting the whole argument will be fine.

   Second Note: if you use single quotes around the path, it will be fine as well and the final command line would look like

   MSYS2_ARG_CONV_EXCL="*" \
       ./ffmpeg.exe -i dist.y4m -i ref.y4m -lavfi \
       libvmaf=model_path='D\\:/mypath/vmaf_v0.6.1.json' -f null -

   with only a double backslash instead of a triple.

External resources

Refer to this page for a list of FFmpeg-based 3rd-party tools.