VideoMetricsLab v2.0: Major performance update, Vulkan VMAF v1 and v0.6.1, metrics and UI run several times faster
v2.0 is a major performance update. VMAF, VMAF NEG and VMAF v1 now run on
any GPU, the other metrics and Video Compare are several times faster, and the
app uses far less memory.
This will likely be the last major release of VideoMetricsLab. While it was a lot of fun to work on this, it has consumed far too much of my life in the past month and I need to back off on this a lot to have a healthier life. I will still do little things here and there, but releases will be much slower going forward.
VideoMetricsLab is a free Windows app for measuring and comparing video quality. Add a source and any number of encodes, and it calculates VMAF, VMAF v1, SSIMULACRA2, Butteraugli, ColorVideo VDP, PSNR, SSIM and XPSNR in one run, accelerated on NVIDIA, AMD and Intel GPUs. Per-frame graphs show exactly where each encode falls behind, and frame-exact playback switches instantly between the source and each encode, so you can see the differences the scores point to. It’s built for anyone tuning encoders, comparing codecs or settings, or checking a release before it ships.
Let me know what you think, have any issues, or have feature requests.
v2.0 changelog
Metric performance
- Added Vulkan support for VMAF v1, so it now runs on any GPU (NVIDIA, AMD or Intel), up to 17x faster than on the CPU.
- Added Vulkan support for VMAF v0.6.1 and VMAF NEG on AMD and Intel GPUs, 2.3x-2.7x faster than on the CPU.
- PSNR, SSIM and XPSNR are now calculated in the app with libvmaf-fast, 2.7x-11x faster than with FFmpeg.
- Added software decoding in the app for videos the GPU can't decode, such as VVC, making VMAF 1.8x-2.7x faster for them.
- SSIMULACRA2 and Butteraugli on the CPU are now 7x faster.
System performance
- Video Compare now plays 4K HDR at 120 frames a second instead of 14, using a fifteenth of the CPU.
- Greatly reduced memory use: the app now idles at 180 MB instead of 840 MB, and VMAF v1 at 4K uses 2.6 GB instead of 13.8 GB.
- Metric Graphs, Video Compare and Settings now open instantly, and the window stays responsive with hundreds of videos.
- Video Compare no longer uses any CPU while idle.
- Detected black bars are now remembered between sessions.
New features
- Added a command line,
VideoMetricsLab-cli, for running comparisons from scripts. - Video Compare can now play HDR video in Display P3 colours on the GPU.
- Added the ability to zoom in Video Compare.
- Added an option to highlight each metric's best and worst results in the Videos tab.
- Added a dark theme, which follows Windows or can be chosen in Settings.
- Added drag and drop for videos, pasting a reference path, reordering rows by dragging, and sorting by any column.
- File dialogs now open in the folder your videos last came from, and the app can reopen your last session's videos.
Features:
Metrics
- VMAF v0.6.1, VMAF NEG, VMAF v1, PSNR, SSIM, XPSNR, SSIMULACRA2, Butteraugli and ColorVideo VDP, for many test videos at once.
- VMAF models for standard, phone, 4K and HFR viewing.
Fast on any GPU
- VMAF, VMAF NEG and VMAF v1 on NVIDIA, AMD and Intel GPUs with libvmaf-fast: VMAF v1 up to 17x faster.
- SSIMULACRA2, Butteraugli and ColorVideo VDP on the GPU with Vship (CUDA, HIP or Vulkan); libjxl on the CPU when no GPU can be used, for SSIMULACRA2 and Butteraugli.
- PSNR, SSIM and XPSNR 2.7x-11x faster, calculated in the app.
- GPU video decoding, with software decoding for formats the GPU can't handle, such as VVC.
Compare and inspect
- Frame-exact playback that switches instantly between the source and each encode (hold S), with zoom, at up to 120 fps in 4K HDR.
- Metric curves, statistics and per-frame scores (per second for ColorVideo VDP), with each metric's best and worst results highlighted.
- Bitrate by frame, second or GOP.
Workflow
- Black bars and resolution mismatches handled automatically.
- Results saved and reloaded automatically.
- Drag and drop videos, sort and reorder the list, and a dark theme.
- A command line,
VideoMetricsLab-cli, for scripts. - Two test videos calculated in parallel on many-core CPUs.
The app can be downloaded below.
