xeve app base/main fps fraction support

[Selur]

like mentioned in #109 with examples current xeve base/main app does not support fractional fps
Would be nice if this could be supported.

[i990049]

https://github.com/mpeg5/xeve/releases
Try version 0.5.1
don't know if setting 23.976 or 59.94 works.

[Selur]

Using xeveb_app-windows from https://github.com/mpeg5/xeve/actions/runs/10227644170 and:
f:\Hybrid\64bit\Vapoursynth\VSPipe.exe "c:\Users\Selur\Desktop\test.vpy" - -c y4m | f:\Hybrid\64bit\xeveb_app.exe --verbose 2 --input stdin --input-depth 10 --input-csp 1 --fps 24000/1001 --codec-bit-depth 10 --rc-type 0 --qp 32 --profile baseline --level-idc 0 --preset medium --info 1 --aq-mode 0 --range limited --matrix-coefficients bt470bg --sar 1:1 --output "G:\Output\test_fraction.evc" (the Vapoursynth script used returns 1000 frames of 640x360 YUV420P10 content @23.976)
encodes:

Output 1000 frames in 57.11 seconds (17.51 fps)1 kbps ] [  0h  0m 56s ]
=== Summary ====================================================================
Bitrate                           = 63.8800 kbps
Encoded frame count               = 1000
Total encoding time               = 57928.000 msec, 57.928 sec
Average encoding time for a frame = 57.928 msec
Average encoding speed            = 17.263 frames/sec
=============================================================================

according to ffmpeg the output is 25fps:

Input #0, evc, from 'G:\Output\test_fraction.evc':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: evc (Baseline), none, 640x360, 25 fps, 25 tbr, 1200k tbn

Using:
f:\Hybrid\64bit\Vapoursynth\VSPipe.exe "c:\Users\Selur\Desktop\test.vpy" - -c y4m | f:\Hybrid\64bit\xeveb_app.exe --verbose 2 --input stdin --input-depth 10 --input-csp 1 --fps 23.976 --codec-bit-depth 10 --rc-type 0 --qp 32 --profile baseline --level-idc 0 --preset medium --info 1 --aq-mode 0 --range limited --matrix-coefficients bt470bg --sar 1:1 --output "G:\Output\test_decimal.evc"
encodes:

=== Summary ====================================================================
Bitrate                           = 63.8800 kbps
Encoded frame count               = 1000
Total encoding time               = 58017.000 msec, 58.017 sec
Average encoding time for a frame = 58.017 msec
Average encoding speed            = 17.236 frames/sec
================================================================================

and ffmpeg reports:

Input #0, evc, from 'G:\Output\test_decimal.evc':
  Duration: N/A, bitrate: N/A
  Stream #0:0: Video: evc (Baseline), none, 640x360, 25 fps, 25 tbr, 1200k tbn

At least to me, it does not look like the xeve app does support decimal or fractions as frame rate atm.

[kpchoi]

Hi @Selur,
"*.evc" file is just elementray bitsteams unlike mp4 or mkv.
evc file doesn't have frame rate information internally.
Even though you set the frame rate properly in encoding time by command-line, EVC encoder doesn't store the information into the encoded evc bitstream because EVC standard doesn't have such syntax.
The frame rate (fps) information is used for bitrate control only in encoding time.
So, ffmpeg cannot understand frame rate (fps) of the evc file.
I guess the ffmpeg is just showing default(?) frame rate like 25fps.

@dkozinski, it would be helpful if you could check why ffmpeg shows 25fps in this case.

[Selur]

EVC encoder doesn't store the information into the encoded evc bitstream because EVC standard doesn't have such syntax.

bummer.

Using:

ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "G:\TestClips&Co\files\i.mkv" -map 0:0 -an -sn -vf yadif=0:1 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -color_range tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -f yuv4mpegpipe - | xeve-base --verbose 2 --input stdin --input-depth 10 --input-csp 1 --fps 29.97 --codec-bit-depth 10  --rc-type 0 --qp 32 --profile baseline --level-idc 0 --preset medium --info 1 --aq-mode 0 --range limited --matrix-coefficients bt709 --sar 1:1 --output "J:\tmp\i_1_2024-08-08@16_05_42_5110_01.evc"

for encoding and

ffmpeg -y -loglevel debug  -r 30000/1001 -i "G:\Output\i_1_2024-08-08@16_07_31_4310_01.evc" -metadata encoding_tool="Hybrid 2024.08.08.1" -c:v
copy -map 0:0   -aspect 1920:1080 -r 30000/1001 "G:\Output\i.mp4"
ffmpeg version N-116436-gbedd4ba26c-g719e46f54c+1 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 14.1.0 (Rev3, Built by MSYS2 project)
  configuration:  --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --ld='ccache g++' --extra-cxxflags=-fpermissive --extra-cflags=-Wno-int-conversion --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --disable-doc --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-librav1e --enable-libsrt --enable-libgsm --enable-libvmaf --enable-libsvtav1 --enable-gnutls --extra-cflags=-DLIBTWOLAME_STATIC --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads
  libavutil      59. 29.100 / 59. 29.100
  libavcodec     61. 10.100 / 61. 10.100
  libavformat    61.  5.101 / 61.  5.101
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.102 / 10.  2.102
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-r' ... matched as option 'r' (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument '30000/1001'.
Reading option '-i' ... matched as input url with argument 'G:\Output\i_1_2024-08-08@16_07_31_4310_01.evc'.
Reading option '-metadata' ... matched as option 'metadata' (add metadata) with argument 'encoding_tool=Hybrid 2024.08.08.1'.
Reading option '-c:v' ... matched as option 'c' (select encoder/decoder ('copy' to copy stream without reencoding)) with argument 'copy'.
Reading option '-map' ... matched as option 'map' (set input stream mapping) with argument '0:0'.
Reading option '-aspect' ... matched as option 'aspect' (set aspect ratio (4:3, 16:9 or 1.3333, 1.7777)) with argument '1920:1080'.
Reading option '-r' ... matched as option 'r' (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument '30000/1001'.
Reading option 'G:\Output\i.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url G:\Output\i_1_2024-08-08@16_07_31_4310_01.evc.
Applying option r (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument 30000/1001.
Successfully parsed a group of options.
Opening an input file: G:\Output\i_1_2024-08-08@16_07_31_4310_01.evc.
[AVFormatContext @ 0000029324e0f180] Opening 'G:\Output\i_1_2024-08-08@16_07_31_4310_01.evc' for reading
[file @ 0000029324e0f800] Setting default whitelist 'file,crypto,data'
[evc @ 0000029324e0f180] Format evc probed with size=16384 and score=51
[evc @ 0000029324e0f180] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[evc @ 0000029324e0f180] All info found
[evc @ 0000029324e0f180] After avformat_find_stream_info() pos: 45948 bytes read:65536 seeks:0 frames:22
Input #0, evc, from 'G:\Output\i_1_2024-08-08@16_07_31_4310_01.evc':
  Duration: N/A, bitrate: N/A
  Stream #0:0, 22, 1/1200000: Video: evc (Baseline), 1 reference frame, none, 1920x1080, 0/1, 25 fps, 25 tbr, 1200k tbn
Successfully opened the file.
Parsing a group of options: output url G:\Output\i.mp4.
Applying option metadata (add metadata) with argument encoding_tool=Hybrid 2024.08.08.1.
Applying option c:v (select encoder/decoder ('copy' to copy stream without reencoding)) with argument copy.
Applying option map (set input stream mapping) with argument 0:0.
Applying option aspect (set aspect ratio (4:3, 16:9 or 1.3333, 1.7777)) with argument 1920:1080.
Applying option r (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument 30000/1001.
Successfully parsed a group of options.
Opening an output file: G:\Output\i.mp4.
[out#0/mp4 @ 0000029324e0ec40] Adding streams from explicit maps...
[vost#0:0/copy @ 0000029324e33f80] Created video stream from input stream 0:0
[vost#0:0/copy @ 0000029324e33f80] Overriding aspect ratio with stream copy may produce invalid files
[file @ 0000029324e12a80] Setting default whitelist 'file,crypto,data'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Output #0, mp4, to 'G:\Output\i.mp4':
  Metadata:
    encoding_tool   : Hybrid 2024.08.08.1
    encoder         : Lavf61.5.101
  Stream #0:0, 0, 1/30000: Video: evc (Baseline), 1 reference frame (evc1 / 0x31637665), none, 1920x1080 [SAR 1:1 DAR 16:9], 0/1, q=2-31, 25 fps, 25 tbr, 30k tbn
[out#0/mp4 @ 0000029324e0ec40] Starting thread...
[in#0/evc @ 0000029324e0ed80] Starting thread...
Press [q] to stop, [?] for help
[in#0/evc @ 0000029324e0ed80] EOF while reading input
[in#0/evc @ 0000029324e0ed80] Terminating thread with return code 0 (success)
[out#0/mp4 @ 0000029324e0ec40] All streams finished
[out#0/mp4 @ 0000029324e0ec40] Terminating thread with return code 0 (success)
[AVIOContext @ 0000029324e79040] Statistics: 92861 bytes written, 2 seeks, 4 writeouts
[out#0/mp4 @ 0000029324e0ec40] Output file #0 (G:\Output\i.mp4):
[out#0/mp4 @ 0000029324e0ec40]   Output stream #0:0 (video): 60 packets muxed (90462 bytes);
[out#0/mp4 @ 0000029324e0ec40]   Total: 60 packets (90462 bytes) muxed
[out#0/mp4 @ 0000029324e0ec40] video:88KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 2.647521%
size=      91KiB time=00:00:02.40 bitrate= 309.5kbits/s speed=1.62e+03x
[in#0/evc @ 0000029324e0ed80] Input file #0 (G:\Output\i_1_2024-08-08@16_07_31_4310_01.evc):
[in#0/evc @ 0000029324e0ed80]   Input stream #0:0 (video): 60 packets read (90462 bytes);
[in#0/evc @ 0000029324e0ed80]   Total: 60 packets (90462 bytes) demuxed
[AVIOContext @ 0000029324e1a0c0] Statistics: 90462 bytes read, 0 seeks

for muxing, I still see:

MediaInfo.exe --full g:\Output\i.mp4
General
Count                                    : 349
Count of stream of this kind             : 1
Kind of stream                           : General
Kind of stream                           : General
Stream identifier                        : 0
Count of video streams                   : 1
Video_Format_List                        : evc1
Video_Format_WithHint_List               : evc1
Codecs Video                             : evc1
Complete name                            : g:\Output\i.mp4
Folder name                              : g:\Output
File name extension                      : i.mp4
File name                                : i
File extension                           : mp4
Format                                   : MPEG-4
Format                                   : MPEG-4
Format/Extensions usually used           : braw mov mp4 m4v m4a m4b m4p m4r 3ga 3gpa 3gpp 3gp 3gpp2 3g2 k3g jpm jpx mqv ismv isma ismt f4a f4b f4v
Commercial name                          : MPEG-4
Format profile                           : Base Media
Internet media type                      : video/mp4
Codec ID                                 : isom
Codec ID                                 : isom (isom/iso2/mp41)
Codec ID/Url                             : http://www.apple.com/quicktime/download/standalone.html
CodecID_Compatible                       : isom/iso2/mp41
File size                                : 92857
File size                                : 90.7 KiB
File size                                : 91 KiB
File size                                : 91 KiB
File size                                : 90.7 KiB
File size                                : 90.68 KiB
Duration                                 : 2400
Duration                                 : 2 s 400 ms
Duration                                 : 2 s 400 ms
Duration                                 : 2 s 400 ms
Duration                                 : 00:00:02.400
Duration                                 : 00:00:02:10
Duration                                 : 00:00:02.400 (00:00:02:10)
Overall bit rate                         : 309523
Overall bit rate                         : 310 kb/s
Frame rate                               : 25.000
Frame rate                               : 25.000 FPS
Frame count                              : 60
Stream size                              : 2395
Stream size                              : 2.34 KiB (3%)
Stream size                              : 2 KiB
Stream size                              : 2.3 KiB
Stream size                              : 2.34 KiB
Stream size                              : 2.339 KiB
Stream size                              : 2.34 KiB (3%)
Proportion of this stream                : 0.02579
HeaderSize                               : 36
DataSize                                 : 90470
FooterSize                               : 2351
IsStreamable                             : No
File creation date                       : 2024-08-08 14:08:07.184 UTC
File creation date (local)               : 2024-08-08 16:08:07.184
File last modification date              : 2024-08-08 14:08:07.184 UTC
File last modification date (local)      : 2024-08-08 16:08:07.184
Writing application                      : Hybrid 2024.08.08.1
Writing application                      : Hybrid 2024.08.08.1

Video
Count                                    : 390
Count of stream of this kind             : 1
Kind of stream                           : Video
Kind of stream                           : Video
Stream identifier                        : 0
StreamOrder                              : 0
ID                                       : 1
ID                                       : 1
Format                                   : evc1
Format                                   : evc1
Commercial name                          : evc1
Codec ID                                 : evc1
Duration                                 : 2400
Duration                                 : 2 s 400 ms
Duration                                 : 2 s 400 ms
Duration                                 : 2 s 400 ms
Duration                                 : 00:00:02.400
Duration                                 : 00:00:02:10
Duration                                 : 00:00:02.400 (00:00:02:10)
Bit rate                                 : 301540
Bit rate                                 : 302 kb/s
Width                                    : 1920
Width                                    : 1 920 pixels
Height                                   : 1080
Height                                   : 1 080 pixels
Pixel aspect ratio                       : 1.000
Display aspect ratio                     : 1.778
Display aspect ratio                     : 16:9
Rotation                                 : 0.000
Frame rate mode                          : CFR
Frame rate mode                          : Constant
Frame rate                               : 25.000
Frame rate                               : 25.000 FPS
FrameRate_Num                            : 25
FrameRate_Den                            : 1
Frame count                              : 60
Bits/(Pixel*Frame)                       : 0.006
Stream size                              : 90462
Stream size                              : 88.3 KiB (97%)
Stream size                              : 88 KiB
Stream size                              : 88 KiB
Stream size                              : 88.3 KiB
Stream size                              : 88.34 KiB
Stream size                              : 88.3 KiB (97%)
Proportion of this stream                : 0.97421

ffmpeg version was build at 2024.07.28 with MBAS.

[rwillenbacher]

@kpchoi Timing Information, like FPS and fixed Framerate or variable Frame, in EVC is stored in the VUI parameters. These are stored in the SPS. So EVC does contain Timing Information, just like AVC, HEVC, VVC ...

[Selur]

in xeve_set_vui
vui->num_units_in_tick = ctx->param.num_units_in_tick;
vui->time_scale = ctx->param.time_scale;
should set these, I suspect they are not properly based on the command line parameters,..

[mpeg5]

Hi @rwillenbacher and @Selur ,
I remember that timing information in VUI is not mandatory field and it is difficult to say the information is important in video codec.
We could implement the syntax in YUI, but sometime it has been mis-leading information due to mismatch between VUI information and file format information after video editing.
Could you suggest great example to show reason why it is required?

[Selur]

What is wrong with the need to signal the frame rate properly?

[kpchoi]

@Selur , I think that it is not a problem relating wrong or correct.
It is recommended to use frame rate information of file format standard rather than using VUI information.
I also found that there was mismatching issue between file format's frame rate and VUI's frame rate after video editing.

[Selur]

Okay, so the conclusion is one can't really use xeveb_app, but should use ffmpeg?
(Assuming that ffmpeg properly marks the stream and the container both for example with 24000/1001.)