FFEncoder Version 2.5.0

What's New

This is a pre-release for an upcoming revision to the script which contains a new encoding report format, DoVi processing updates, and bug fixes. Functions for processing HDR/DoVi metadata which were previously private are now exported for standalone use outside of script execution.

All new features and fixes have been tested, but more is required before adding changes upstream.

Updates

• New, beautiful HTML encoding report format option with better data presentation, but also includes the raw encoding summary at the bottom for reference. A new parameter is also available to specify the report format, -ReportType or its alias -ReportFormat. Options are html or text
• Improved lossless audio detection which is used to warn users when attempting to transcode from one lossy codec to another for all audio options (including DEE).
  • The following formats are considered lossless and will not raise a warning when transcoding to another codec:
    • DTS-HD MA / DTS XLL standard and object-based (DTS-X)
    • TrueHD / MLP FBA standard and object-based (TrueHD Atmos)
    • FLAC
    • Linear PCM variants
• Removed multiplexing requirement when encoding audio with DEE. The script allows deew select the first track automatically, which should be the primary lossless audio. If it is not lossless, the function will exit without transcoding. This removes significant complexity that was necessary in the early days of the script's implementation, reducing the amount of files output by the script and improving performance (yay!)
• Implemented Get-MediaInfo module to replace longer running tasks previously done using ffmpeg/ffprobe (credit to stax76). This provides large speed enhancements and reduces the chance for errors
• Updated HDR detection to be more accurate and extensible, particularly for non-standard sources in resolutions other than 2160p
• Updated Edit-RPU function to allow for standalone use
• Added Get-HDRMetadata and Edit-RPU to module exports for standalone use
• Added new options for DoVi encoding. Now supports profile 8.4 and FEL mapping to profile 8.1 (requires additional processing in VapourSynth/AviSynth+ before script can use it). These values can be set via the -DolbyVisionMode parameter
• More .NET optimization enhancements

Fixes

• Fixed summary display for DoVi encodes at end of script. Previously, 2 lines were being excluded
• Fixed Deblock parameter parsing from config file when negative values are passed
• Added remove_mapping to DoVi RPU editor to prevent image alteration on some sources
• Removed presets from DoVi RPU editor unless active offset is 0 to prevent conflicts with variable aspect ratio sources
• (Windows) Fixed VMAF statistics when UNC paths are used. Previously, UNC paths would be processed but the log file would not save properly, resulting in no output. Now, the log file saves to the user's Videos directory (determined dynamically via .NET API call)
• Added better error handling and reporting for Measure-CropDimensions function
• Improved the implementation of the -RemoveFiles switch parameter. Now, all generated files will be deleted and done so in a more efficient manner

encoder.ini Parser Fix

What's New

A minor fix to encoder.ini parsing when non-parameter options are specified within the file.

Config File Updates, VMAF Overhaul

What's New

This release comes with new options for configuration files as well as a major overhaul to VMAF. Also included are several bug fixes and/or enhancements for other features.

Updates

• Added MKVTagGenerator section to script.ini
• Added VMAF section to script.ini
• Added new VMAF parameter -VMAFResizeKernel (alias -VMAFKernel) to specify which kernel to use for upscaling/downscaling the encode back to source resolution. Default is bicubic
• Updated text output for VMAF with conditional color formatting based on score. Removed noisy ffmpeg output
• Calculate VMAF, SSIM, and PSNR based on harmonic mean instead of arithmetic mean for a more realistic score
• Removed git requirement when updating FFEncoder - REST API used to download archive instead

Fixes

• Fixed config file overrides for a few missing script parameters
• Removed leading - requirement for ffmpeg config options
• Fixed potential key collision error when a setting exists in a config file and in -FFMegExtra or -EncoderExtra
• Fixed error handling when parsing config files. If an error occurs while loading one configuration file it won't impact the others
• -GenerateMKVTagFile will now raise an exception if no API key is passed
• Removed requirement check for mkvpropedit in main script logic. If it is not found, the file will be created but not muxed
• Renamed parameter -Scale to -ScaleKernel to be more descriptive
• Fixed SSIM and PSNR calculations during VMAF (previously using old, incorrect syntax from ffmpeg filter documentation)
• Fixed bug with DoVi processing of audio/subs
• Fixed broken path for DoVi RPU when edited file already exists
• Fixed bug with preset values set in config file being duplicated or causing a crash
• Replaced deprecated --y4m argument when piping output from vspipe to x265
• Config file parser will automatically detect key collisions and fix them without raising an exception

VapourSynth Mux Fix

What's New

Several bug fixes, including a major fix for VapourSynth input muxing, and a new config file for script switch parameters.

Updates

• A new config file is available for script switch parameters. Config options can be set in the file instead of having to pass a parameter each time. If the parameter is passed, it overrides the configuration file
• More .NET performance enhancements
• Updated deew default bitrates
• Removed git requirement when updating FFEncoder

Bug Fixes

• Fixed VapourSynth script muxing for audio, subs, and chapters. These were not being copied before
• Fixed additional string parsing bugs from PowerShell 7.3 update
• Extract chapters using ffmpeg during DoVi encodes if mkvextract isn't available
• Fixed lossy -> lossy audio encoding warning for some DTS-HD MA tracks
• Fixed issue where parameter options were ignored in the encoder config file. If a script parameter is passed, it overrides the config

Auto-Edit DoVi RPUs & Dynamic Metadata Updates

What's New

This release comes with a few big updates for handling dynamic metadata, as well as some bug fixes.

Automatic RPU Editing

The script will now edit generated Dolby Vision RPU files to ensure the metadata is accurate. I started to notice that RPU's did not always contain the correct metadata even when extracted from a profile 7 source, particularly base HDR10 metadata. This is now corrected automatically.

HDR10+ Handling

If the -SkipHDR10Plus parameter is used, SEI messages are automatically dropped from DoVi compatible sources to prevent conflict.

A new parameter has been added to correct HDR10+ metadata ordering called -HDR10PlusSkipReorder. In some cases, HDR10+ metadata is copied in its normal decoding order which can break when B frames are introduced into the stream. It is important to note that this isn't always required, and you must validate yourself whether this will be necessary or not.

Binary Updates

All binaries have been updated to their latest version.

Bug Fixes

• The core will be extracted from DTS-HD MA streams automatically when the DTS audio option is selected instead of forcing a re-encode
• Audio track naming has been fixed
• Start time for audio in DoVi test encodes has been fixed based on test parameters passed. Previously it was hard coded to begin at 00:01:30
• Fixed audio parsing bug for DoVi encodes on PowerShell 7.3
• Fixed other string parsing bugs related to the PowerShell 7.3 update

Binary & PowerShell 7.3 Updates

What's New

This release contains updates for all associated binaries, as well as breakfix changes for PowerShell 7.3 users. A new parameter has arrived, too.

Binary Updates

All binaries distributed with the project have been updated. Deew was recompiled on all platforms with new updates from both the original author and me. These updates fix a few minor issues and pave the way for future enhancements.

PowerShell 7.3 Updates

Parameter quoting and parsing for third party binaries changed dramatically in PowerShell 7.3 and caused much of the script to break. These conflicts have been fixed and tested on PowerShell 7.2 and 7.3.

-ConstantQP Parameter

A new rate control parameter that can be used as an alternative to CRF and ABR (1/2 pass encodes). This parameter is useful for testing (particularly when comparing x264 vs. x265) but is, in general, not recommended for production use unless you have a good use case.

You may also use the alias -QP.

Bug Fixes

• As mentioned, escaped quoting has been removed so that the script will work with PowerShell 7.3
  • Tested on PowerShell 7.2 and 7.3, but not 7.1 and 7.0 (should work, let me know if it doesn't)
• Fixed script pause for stereo encoder background jobs (when the encode finishes before the audio)
• Fixed track naming for stereo audio when no background jobs are needed
• Fixed proper termination of script with x265 when Ctrl+C is detected. This requires forcibly killing the x265 process before shutting down
  • Function will search for an x265 job generated from cmd.exe (this is how the script launches the process due to memory leaks in PowerShell)
• Fixed bug with psy-rd parsing in x264, which caused the PsyRdoq/PsyTrellis parameter to be ignored when psy-rdo wasn't explicitly set

Bug Fixes for Progress Bar

What's New

This is a small release meant to fix all weird edge errors with the progress bar...hopefully for good.

Fixes

• Fixed null indexing error with the progress bar on pass 2/2 of a 2-pass encode
• Added error handling surrounding each call to the progress bar function. If it throws an error for whatever reason, they will be caught by the caller and the encode will resume without the bar
  • If it's a 2-pass encode, the script has the opportunity to recover progress for the second pass
• Frame counts used by the progress bar are stored in a module variable on the first pass, negating the need to re-calculate on the second pass

Config Files & Bug Fixes

What's New

The few remaining hard-coded options are gone, and I've added config files for both encoders and ffmpeg. There is one major bug fix included and a few minor ones, too.

Configuration Files

In the main project folder, there is a new directory config that houses two configuration files, encoders.ini and ffmpeg.ini. Here, you can set any other option that is not covered by the script. This is convenient for things you use a lot, but no parameter is available for it other than the -*Extra parameters. I still find those more convenient for lesser-used settings.

The structure is simple, using standard key-value pairs in encoders.ini:

; encoders.ini

[x264]
open-gop=0
merange=64

[x265]
open-gop=0
limit-refs=2

ffmpeg.ini is slightly different, with one section for settings which take an argument, i.e. loglevel=warning, and another for no argument settings, like shortest:

; ffmpeg.ini

[Arguments]
; Settings that take arguments. Do not quote arguments:
-loglevel=panic
-cpucount=10

[NoArguments]
; Settings that take no arguments:
-hide_banner
-shortest

I coded it so that it expects the preceding - for each setting, I'm not sure why, but be sure to put it there or it won't work. I'll make it optional in the next release.

Bug Fixes

• I did some unintentional refactoring during the last release, and accidentally pushed it with the code changes. It caused a terminating error on any non-DV encode due to an erroneously missing if statement, and has been fixed.
• I also fixed a bug in DV where some settings were being tripled in the final command line. This wasn't causing any critical errors, but probably would at some point (not to mention performance).

VapourSynth, Unsharp Filter & VMAF Updates

What's New

This release comes with a few new exciting features, several updates to existing features, and quite a few bug fixes as well, some of which were unintentionally added in the last release.

New Features

Vapoursynth Integration

I'm a big fan of Vapoursynth, so this is something I've wanted to implement for a while now. It was frustrating to have two separate workflows, one for ffmpeg and one for VapourSynth, so I went about adding it to the script.

ffmpeg can be compiled with native VapourSynth demuxing support through the --enable-vapoursynth compilation flag, and this is a requirement for this feature to work on all non Dolby Vision sources (more on that in a second). This allows you to feed a VapourSynth script directly to ffmpeg without using VSPipe, thus reducing overhead and complexity.

However, because Dolby Vision encoding still uses x265, I had to implement a VSPipe workaround as ffmpeg still isn't handling RPU files properly as of version 5. To summarize:

• All non Dolby Vision encodes will use the native VapourSynth demuxer compiled with ffmpeg
• Dolby Vision encodes will use VSPipe instead, which must be available via PATH. The VapourSynth script is then piped directly to x265

Some Notes on Usage

• The -InputPath parameter is still required, even when using a VapourSynth script; it is used to discover properties used throughout the script, and is also used to encode/copy audio and subtitles (if selected)
  • The source must be present in the VapourSynth script, too, as it handles the filtering
• Video filter-related parameters will be ignored when a VapourSynth script is used - all filtering must be done in the VapourSynth script, including cropping

See the wiki for usage details.

Sharpen / Blur Filter

I've added two new parameters for applying sharpening and blur filters to the video using unsharp. The new parameters are -Unsharp and -UnsharpStrength, and each parameter comes with several presets I've created to make things easier. Each parameter includes tab-complete support, too, so you can tab through and see your options.

For -Unsharp, the preset options sets the type of filtering (luma, chroma, yuv) with a postfix denoting the search area (small, medium, large). You also have the option to pass a custom filter string if you don't want to use a preset, and the string will be passed to ffmpeg - the only string manipulation done is replacing custom= with unsharp= if it isn't already present.

For -UnsharpStrength, the prefix denotes whether the filter should sharpen or blur, while the postfix controls the strength. If a custom filter string is passed via -Unsharp, this parameter is ignored.

Options for -Unsharp

• luma_small
• luma_medium
• luma_large
• chroma_small
• chroma_medium
• chroma_large
• yuv_small
• yuv_medium
• yuv_large
• custom=<custom_filter_string>

Options for -UnsharpStrength

• sharpen_mild
• sharpen_medium
• sharpen_strong
• blur_mild
• blur_medium
• blur_strong

Updates

This release comes with updates to several existing features.

Background Audio Jobs Update

Previously, if a background audio job didn't complete before the video did, the script would continue on and prompt you to add the audio manually. I realized this was kind of stupid, so the script will now wait for the background job to complete before proceeding.

VMAF Updates

By default, libvmaf v.2.0.0+ only uses one thread, which can be horribly slow depending on the comparison. I've updated the code to query the number of logical processors on the system (using native commands for each supported Operating System) and use ~ 50% of the resources during comparisons, which speeds things up a lot.

FPS Counter in Progress Bar

You can now see your current FPS directly in the progress bar. Sweet.

Other Updates

• A few subtitle and audio options have new banner wording to clear up some confusing verbiage
• More code optimizations to improve performance using native .NET calls wherever possible

Bug Fixes

A number of bugs have been fixed in this release:

• Fixed resource lockout on the encoding log when running multiple encodes of the same source simultaneously. The script will now check if there is an existing encode locking the log file, and create a new log if necessary
• Fixed track name field duplication in mkvmerge. Somehow the same line of code was repeated 3 times, causing issues while naming audio tracks
• VMAF bug fixes
  • Removed extra whitespace in the resolution check function, which was added by my autoformatter and caused ffmpeg to freak out
  • Fixed regex match failure while checking resolution. This is definitely a bug with PowerShell's $Matches variable and not the script, and the functionality has been replaced with a .NET method call
  • Fixed model and log file path logic on Windows
• Fixed hard-coded, non-overriding values for keyint / min-keyint when performing a Dolby Vision encode

PsyRd Bug Fix with Dolby Vision

What's New

This release is primarily a breakfix for the PsyRd parameter when running a Dolby Vision encode. If you did not set it explicitly via parameter, it was defaulting to to 0.00. This is a result of changes made in 2.0.0 for x264 and has been fixed.

I also updated some of the console output to be consistent with the new formatting and removed legacy code in the audio functions.