AEAD payload length check

[maxsharabayko]

If AES-GCM is enabled, 16 bytes of the payload are occupied for the authentication tag.
Hence the payload size must be properly validated when submitted to the SND buffer.

Once a connection is established and the MTU value is negotiated (minimum of the two), this value is used to determine the maximum allowed payload size. Thus any payload exceeding the value is not allowed to be sent.

Also fixes #2589.

Also fixes CSndBuffer::addBufferFromFile(..): wrong usage of iPktLen.

Example Test Case 01

 ./srt-xtransmit.exe generate "srt://127.0.0.1:4200?cryptomode=2&passphrase=abcdefghijk&payloadsize=1440" --sendrate 1Mbps --msgsize 1440 --enable-metrics -v

./srt-xtransmit.exe receive "srt://:4200?cryptomode=2&passphrase=abcdefghijk" --enable-metrics --msgsize 1456 -v

[maxsharabayko]

A question is what to do with the SRTO_PAYLOADSIZE max value. The SRTO_CRYPTOMODE is set to auto by default, thus allowing GCM connections. Then 1456 is no longer allowed unless an AES-CTR connection is established.

[ RUN      ] TestSocketOptions.MaxVals
15:10:51.963229/test-srt*E:SRT.sm: SRTO_GROUPMINSTABLETIMEO set 5000
Skipping SRTO_KMPREANNOUNCE
Skipping SRTO_KMREFRESHRATE
15:10:51.963401/test-srt*E:SRT.ac: SRTO_PAYLOADSIZE: value exceeds 1456 bytes decreased by 16 required for AES-GCM.
/home/runner/work/srt/srt/test/test_socket_options.cpp:264: Failure
Expected equality of these values:
  srt_setsockopt(sock, 0, entry.optid, &opt_val, opt_len)
    Which is: -1
  expect_return
    Which is: 0
Setting SRTO_PAYLOADSIZE to 1456 must succeed
/home/runner/work/srt/srt/test/test_socket_options.cpp:242: Failure
Expected equality of these values:
  opt_val
    Which is: 1316
  value
    Which is: 1456
[Caller, max value]: Wrong SRTO_PAYLOADSIZE value 1316
[  FAILED  ] TestSocketOptions.MaxVals (1 ms)

[ethouris]

There was generally a controversy around the SRTO_PAYLOADSIZE option. This option had to actually set the maximum size of the payload mainly for stats, but then it's useful for setting the maximum size for a single call to srt_send, and the minimum buffer size for srt_recv. There are two problems with it though:

1. This value isn't negotiated. You can set this to 1456 on the sender side and the receiver side has still 1316 and is allowed to pass a too-small buffer.
2. The feature to enforce the buffer size is desired, but SRTO_PAYLOADSIZE is for the single packet only.

Therefore I think it should be possible to kinda repurpose this option for something else, and this way:

• In Live mode, it is always limited to 1456, minus possibly reserved space for FEC or AEAD
• In Message mode, it is theoretically unlimited, but shall not exceed the flight flag size and sender buffer size
• In Stream mode, it is always 0 and cannot be modified.

[ethouris]

BTW test_socket_options.cpp should be fixed accordingly.

[codecov-commenter]

Codecov Report

Merging #2591 (9c35a6c) into master (98b1b00) will increase coverage by 0.01%.
The diff coverage is 63.88%.

@@            Coverage Diff             @@
##           master    #2591      +/-   ##
==========================================
+ Coverage   66.35%   66.36%   +0.01%     
==========================================
  Files          99       99              
  Lines       19822    19826       +4     
==========================================
+ Hits        13153    13158       +5     
+ Misses       6669     6668       -1     

Impacted Files	Coverage Δ
srtcore/buffer_snd.h	50.00% <ø> (ø)
srtcore/strerror_defs.cpp	75.00% <ø> (ø)
srtcore/socketconfig.cpp	69.49% <50.00%> (-0.51%)	⬇️
srtcore/core.cpp	60.96% <60.00%> (+0.42%)	⬆️
srtcore/buffer_snd.cpp	68.70% <70.00%> (+0.95%)	⬆️
srtcore/api.cpp	53.28% <0.00%> (-1.76%)	⬇️
srtcore/group.cpp	37.76% <0.00%> (+0.05%)	⬆️
srtcore/list.cpp	81.70% <0.00%> (+0.25%)	⬆️
srtcore/utilities.h	63.63% <0.00%> (+0.44%)	⬆️
... and 5 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more