[BUG] Invalid reset packet (incorrect ack sequence number) when connection backlog is full

[t123yh]

Describe the bug
If you set connection backlog to a low limit, and connection count starts to reach the limit so new connections cannot be handled, it is expected that FreeRTOS+TCP will send RST packet upon receiving a SYN request it cannot handle (see FreeRTOS_TCP_State_Handling.c#L963).

However, the RST packets sent here have a wrong ACK value, as you can see in the following screenshot. 192.168.2.102 (client, Linux) is trying to connect to 192.168.1.115 (server running FreeRTOS+TCP). The client sent a SYN packet with sequence number 615815458, and server replied the same ACK number.

This behavior will make TCP stack of the client ignore this RST packet, because as per RFC793, Page 69, when a segment arrives for a connection in a state other than LISTEN, it is at first checked whether the sequence number falls into the receiver window, and if it doesn't, it's dropped (ref). When client sent SYN with seq=n, it is expected that the RST packet comes back with ack=n+1 (you can check this behavior on any normal Linux PC). In the screenshot below, 192.168.1.1 is a normal Linux server:

Target

• Development board: rp2040+w5500
• Instruction Set Architecture: ARM Cortex m0+
• IDE and version: CLion
• Toolchain and version: arm-none-eabi-gcc (GNU Arm Embedded Toolchain 10.3-2021.10) 10.3.1

Host

• Host OS: Linux
• Version: Debian 11

To Reproduce

• Create a simple TCP server with FreeRTOS+TCP that listens but never accepts connections. Set the backlog to 3.
• Connect to this server with TCP clients on your PC. Make 4 connections simultaneously.

Expected behavior
An RST packet with correct ACK number is sent from FreeRTOS+TCP. The fourth TCP client reports Connection refused.

** Actual behaviour **
The RST packet with wrong ACK number is ignored, and the fourth TCP client keeps waiting.

Screenshots
See above.

[htibosch]

Hello @t123yh , I was aware of the rule in RFC793 ( page 69 ), but I had never realised that the when replying to a SYN, the seq must be incremented with 1 also. I will prepare and test a PR for this.
Thank you very much for taking the effort to report this.

[htibosch]

Hello @t123yh, I just made the required change in PR #724.

I tested it on an embedded device:

Now the DUT responds with Seq=1, in stead of Seq=0.

Thanks

[AniruddhaKanhere]

Thank you for bringing this bug to our attention @t123yh and thank you for fixing the said bug @htibosch.

Since the PR is merged, I will go ahead and close the issue now.