large File Uplink fails due to PacketOutOfOrder

[sheershak-agarwal]

F´ Version	v2.0.0 and later
Affected Component	FileUplink

---

Problem Description

When uplinking a big file(>1MB), f' throws PacketOutOfOrder.

Example Events:
022-02-08T03:58:03.286Z | 0x7ff | fileUplink.PacketOutOfOrder | WARNING_HI | Received packet 49 after packet 34
2022-02-08T03:58:03.304Z | 0x7ff | fileUplink.PacketOutOfOrder | WARNING_HI | Received packet 54 after packet 49

How to Reproduce

1. Changed Topology.cpp QUEUE_SIZE so that f` can receive large files.

enum
{
  UPLINK_BUFFER_STORE_SIZE = 3000,
  UPLINK_BUFFER_QUEUE_SIZE = 25000,
  UPLINK_BUFFER_MGR_ID = 200
};

2. Try uplinking large files over the network.

Expected Behavior

FileUplink should be able to upload files without losing packets.
Maybe put a delay in GDS while sending files or make changes in fprime to request for missed packets or account for out of order packets.

Related to nasa/fprime-gds#43

[timcanham]

You mentioned there were also asserts. Can you post them as well? I'm guessing there was a queue overflow.

[sheershak-agarwal]

Assert file "/home/sheershak/F-Prime/App/build-fprime-automatic-raspberrypi/F-Prime/Svc/FileUplink/FileUplinkComponentAc.cpp": Line: 1757 8
Assert file "/home/sheershak/F-Prime/App/build-fprime-automatic-raspberrypi/F-Prime/Svc/FileUplink/FileUplinkComponentAc.cpp": Line: 1757 8
Assert file "/home/sheershak/F-Prime/App/build-fprime-automatic-raspberrypi/F-Prime/Svc/FileUplink/FileUplinkComponentAc.cpp": Line: 1757 8
Assert file "/home/sheershak/F-Prime/App/build-fprime-automatic-raspberrypi/F-Prime/Svc/FileUplink/FileUplinkComponentAc.cpp": Line: 1757 8

[LeStarch]

May need to add a throttling mechanism in the GDS as this is likely an overflow of incoming packets.

[timcanham]

At one point we had discussed the idea of flow control on interfaces that could support it, e.g. network. The network "radio" ACKs each uplinked file packet, and the ground won't send the next until it sees the ACK.

[LeStarch]

This problem seems to be driven by a queue overflowing (error 8 above). The best fix would be to increase the file uplink queue size. We could also make a discussion to talk about what to do when the radio is substantially faster than file uplink.