Implement native stream recording into the server

[aler9]

Describe the feature

Implement native stream recording into the server in order to replace ffmpeg and provide a complete NVR solution.

[Tabarane]

Support it 1000000%

[realflight1]

Thanks for rtsp-simple-server / MediaMTX! This is currently the only feature I am missing from SRS (they call it DVR). I wanted to describe my use case and discuss why the current recommended solution (run ffmpeg in runOnReady) is not optimal.

My use case - besides live viewing I also need to record 10-20 video streams in "sessions" of 1-2 hours and later play them more or less synchronized (1-2 seconds difference is fine). I started with the following: when a session starts, launch an external ffmpeg process for each stream and save RTMP/RTSP to file. This works surprisingly well, as long as the streams are uninterrupted. As soon as they are interrupted I need to restart ffmpeg and later fill the missing gaps (e.g. with blank video), and it's hard to keep the recordings synchronized. Using runOnReady with ffmpeg is less optimal for this because the ffmpeg segment feature time format does not support milliseconds resolution (at least not on Windows). So I am currently using SRS's DVR feature which can create files with milliseconds resolution in their timestamps.

Anyway, thanks again for this project!

[Tabarane]

Hello @realflight1,

I recently came across SRS through your comment. However, in my experience, MediaMTX is a much more advanced and robust solution compared to SRS. Generally, DVRs are used for analog cameras while NVRs are utilized for digital cameras or video feeds. In my enterprise work, any CCTV application or video processing utilizes NVR as the Network Video Recorder.

I am facing a similar issue as you, where video streams are coming from drones and mobile video transmission units that are up for a certain amount of time and then go down, and then up again, repeatedly. Unfortunately, there is no viable way to record this scenario using any solutions I have tried, except for MileStone or Genatic, which are enterprise solutions for CCTV. However, these solutions require payment for any changes or additions.

Incorporating the recorder feature in MediaMTX will enable it to be a superior and comprehensive enterprise-level solution compared to SRS.

[realflight1]

Hi @Tabarane, thanks for your comment. To be clear I prefer MediaMTX and not advocating for SRS - this is why I am posting here! With regards to DVR/NVR, sure, the terminology might not be 100% correct, I was just referring to the feature name in SRS if anyone wants to read their docs or test it. You may replace "DVR" or "NVR" with "save digital video streams to local disk" which is the feature being discussed here. I agree with everything else you mentioned.

[aler9]

Hello everyone. Open source is not about competition, There is no money involved and there's not a prize for the better solution (also because there isn't a common indicator of "goodness" and there will never be).

Money is involved at a higher level, since nowadays every single company, big or small, makes use of open source, but that's another argument.

Open source by itself is about other things, in particular is about finding solutions to common problems. Projects are conceived because they solve specific problems that are not fully solved until that moment.

SRS for instance is particularly good for implementing a distributed SRT-based system (while most SRT media servers are proprietary), and i can name dozens of other software that are good in tasks that are similar or overlap with the ones of this project. If tomorrow my company tasks me with routing video with SRT, i'd use SRS. This project on the other hand was conceived in order to allow everyone, with every hardware, from Raspberry Pis to enterprise servers, to handle and process video streams. If my company tasks me with routing 2000 video streams from a city and showing them on a web page, i know that this project can handle them and i use it.

Furthermore, the advantage of open source is that if you find a missing feature, you can develop and contribute it.

Back to this task, the main difficult consists into choosing a container format that is capable of handling most of the codecs that can be routed through the server. MPEG-TS is one, the other candidate is Matroska. If you want to speed up the development of this task, you can

• investigate whether MPEG-TS and Matroska can handle all codecs listed in the README
• find pure-Go implementation of Both
• build mock-ups or even hack the server in order to add needed parameters to the configuration file and implement the recording system

That's all.

[Tabarane]

Codecs supported by MPEG-TS:

1. MPEG-1 Video
2. MPEG-2 Video
3. H.264/AVC (Advanced Video Coding)
4. H.265/HEVC (High-Efficiency Video Coding)
5. MPEG-4 Visual (Part 2)
6. VC-1 (Video Codec 1)
7. VP8
8. VP9
9. AV1
10. MPEG-1 Audio Layer I/II
11. MPEG-1 Audio Layer III (MP3)
12. MPEG-2 Audio Layer II
13. MPEG-2 Audio Layer III
14. Dolby Digital (AC-3)
15. Digital Theater Systems (DTS)
16. PCM (Pulse Code Modulation)
17. Adaptive Multi-Rate (AMR)

Codecs supported by Matroska:

1. MPEG-1 Video
2. MPEG-2 Video
3. H.264/AVC (Advanced Video Coding)
4. H.265/HEVC (High-Efficiency Video Coding)
5. MPEG-4 Visual (Part 2)
6. VP8
7. VP9
8. AV1
9. MPEG-1 Audio Layer I/II
10. MPEG-1 Audio Layer III (MP3)
11. MPEG-2 Audio Layer II
12. MPEG-2 Audio Layer III
13. Advanced Audio Coding (AAC)
14. Dolby Digital (AC-3)
15. Digital Theater Systems (DTS)
16. Vorbis
17. Opus
18. PCM (Pulse Code Modulation)
19. FLAC (Free Lossless Audio Codec)
20. ATRAC (Adaptive TRansform Acoustic Coding)
21. Monkey's Audio (APE)
22. Musepack
23. Speex
24. WavPack

• Pure-Go
  https://github.com/Comcast/gots
  https://github.com/asticode/go-astits
  https://github.com/at-wat/ebml-go

as for the last part im too noob of developer in order to implement such thing :))))

[realflight1]

@aler9 thanks for your comment! I definitely understand why you would prefer to implement this once in one of the container formats you mentioned. But after looking at existing golang libraries I do not believe it is going to be easy without pulling in a dependency like go-gst (with CGO). I am not even sure there are existing AV1 over MPEG-TS implementations.

What do you think about going for the low hanging fruit first based on the existing HLS implementation? Instead of writing the data to HTTP it can be written to the filesystem. Yes, I realize it limits what codecs are supported, but it might be a good start. A relevant part of the configuration file could look like this:

paths:
  mypath:
    # record to disk, disabled by default
    record: yes
    # record format: mpegts, fmp4
    recordFormat: mpegts
    # disk location for saving the files - variable names are just for the purpose of discussion, or maybe allow more control with strftime style variables including milliseconds
    recordPath: /path/to/recordings/$RTSP_PATH/$TIMESTAMP_WITH_MSECS.ts

(Related discussion: #991)

[aler9]

What do you think about going for the low hanging fruit first based on the existing HLS implementation?

That's exactly the way to go, the only required improvement consists into hacking the current MPEG-TS implementation in order to detect more codecs.

[pikachu937]

Hi all. I am happy to participate in the discussion. if you will, I have a suggestion for adding options:
1 - balance recording between drives (if you have 2 or more drives or network storages)
2 - the ability to turn off the audio track (for example, ffmpeg has the "-an" parameter), not everyone needs to write audio
3 - a folder with a new date is created for each day and the numbering of files starts from the beginning (for example, 001_seg.ts).
4 - in addition to the previous point, you can add unix timestamp so that when you restart the stream, already created files are not overwritten (for example, 001_seg_$timestamp.ts)
5 - duration segment in seconds
6 - checking the need to write for a specific stream if many different streams are published on the server (path: all), for example: by default, all(ten or hundred or thousand) streams are written, but on three or ten or hundred you need to turn off the recording

paths:
  mypath:
    # record to disk, disabled by default
    record: yes
    # record format: mpegts, fmp4
    recordFormat: mpegts
    # enable or disable audio track
    enableaudio: false
    # list mount point 
    drivers: /mnt/rec1, /mnt/rec2, /mnt/recN
    # duration segment
    segmentDuration: 30s
# disk location for saving the files - variable names are just for the purpose of discussion, or maybe allow more control with strftime style variables including milliseconds
    recordPath: $drivers/$RTSP_PATH/$(date +%Y-%m-%d)/$num_seg_$TIMESTAMP_WITH_MSECS.ts

as well as adding saving metadata in the database 1681

[wycrystal]

Support it 1000000%

[aler9]

This feature has been implemented and is in beta here: #2255

[aler9]

added in v1.1.0

[klk2]

Is it possible to add recording of external streams, such as：from external rtsp/rtmp server

[aler9]

@klk2 this is already possible:

record: yes
paths:
  my_external_stream:
    source: rtsp://my-external-server

[brunobaraobr]

Hi, First, I would like to thank you for the excellent project, updates and implementations that were made. I have a question, is it possible to set a different recording path per camera through the api?

[pikachu937]

Hi, First, I would like to thank you for the excellent project, updates and implementations that were made. I have a question, is it possible to set a different recording path per camera through the api?

Hello, yes over api or config you can set path, for example:

paths:
  test1:
    source: rtsp://url/path
    record: yes
    recordPath: ./recordings/%path/%Y-%m-%d_%H-%M-%S-%f
    recordPartDuration: 100ms
    recordSegmentDuration: 1h
    …

[aler9]

is it possible to set a different recording path per camera through the api?

This is currently not possible, recording path was centralized in order to centralize the record cleaning system, that runs once for every path.

[aler9]

I realized that i made a mistake by making record settings global, instead of path-based. It's too limiting.

Recording settings will be moved to the paths section.

Furthermore, in order to simplify the configuration of paths, i'll also add a templating system that allows to reuse the same configuration for multiple paths, overriding fields on demand.

Feature request is here: #2410

[github-actions]

This issue is being locked automatically because it has been closed for more than 6 months.
Please open a new issue in case you encounter a similar problem.