Certain truncated files can trigger Python MemoryError

[michaeltandy]

Description

• Version: MCAP libraries for Python 1.1.1 with mcap-ros2-support 0.5.3
• Platform:

Steps To Reproduce

Load a truncated file, such as this one: minimal-mcap-to-trigger-memory-error.mcap.gz

Using for example this code:

from mcap_ros2.reader import read_ros2_messages
for msg in read_ros2_messages('minimal-mcap-to-trigger-memory-error.mcap'):
    print(msg.ros_msg)
    break

Observed behaviour

(venv) mtandy@mtandy-tr:/tmp/mcap-test$ python3 basictest.py 
Traceback (most recent call last):
  File "/tmp/mcap-test/basictest.py", line 4, in <module>
    for msg in read_ros2_messages('demo-memoryerror.mcap'):
  File "/tmp/venv/lib/python3.12/site-packages/mcap_ros2/reader.py", line 74, in read_ros2_messages
    for schema, channel, message, ros2_msg in reader.iter_decoded_messages(
  File "/tmp/venv/lib/python3.12/site-packages/mcap/reader.py", line 191, in iter_decoded_messages
    for schema, channel, message in message_iterator:
  File "/tmp/venv/lib/python3.12/site-packages/mcap/reader.py", line 278, in iter_messages
    summary = self.get_summary()
              ^^^^^^^^^^^^^^^^^^
  File "/tmp/venv/lib/python3.12/site-packages/mcap/reader.py", line 338, in get_summary
    footer = next(StreamReader(self._stream, skip_magic=True).records)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/venv/lib/python3.12/site-packages/mcap/stream_reader.py", line 120, in records
    record = self._read_record(opcode, length)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/venv/lib/python3.12/site-packages/mcap/stream_reader.py", line 205, in _read_record
    self._stream.read(length - 9)
  File "/tmp/venv/lib/python3.12/site-packages/mcap/data_stream.py", line 29, in read
    data = self._stream.read(length)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
MemoryError

• Looking for a footer, reader.py seeks to 37 bytes before the end of the file.
• StreamReader reads an opcode and length. The length is 4049071657446291757 bytes, having read the ASCII 69048118 and the opcode is 51 "unknown"
• To skip the unknown record, a read the length of the record is requested, which would read four exabytes of data.
• A MemoryError is understandably raised.

Expected Behavior
mcap.exceptions.EndOfFile or similar

[linear]

FG-8575 Certain truncated files can trigger Python MemoryError