Configuration specification for MCAP CLI

[maximeguy]

Hi and thanks for your efforts developping MCAP.
I'd like to do several test of the compression algorithm and compression levels, I'm struggling to find information about how the .mcap.yaml configuration should be formed.
I'm using the McapWriterOptions formatted as yaml or as a string after the --config flag with no impact.

Example .mcap.yaml :
`

.mcap.yaml

noChunkCRC: false
noAttachmentCRC: false
enableDataCRC: false
noSummaryCRC: false

noChunking: false

chunkSize: 4194304

compressionFormat: "zstd"
compressionLevel: "slowest"
forceCompression: false

profile: "ros2"

noRepeatedSchemas: false
noRepeatedChannels: false

noMessageIndex: false
noAttachmentIndex: false
noMetadataIndex: false
noChunkIndex: false

noSummaryOffsets: false
`

The config file is used with this command :
`
$ mcap compress input.mcap -o output.mcap --config ~/.mcap.yaml --pprof-profile

Using config file: /home/maxime/.mcap.yaml
Wrote profiles to mcap-cpu.prof, mcap-mem.prof, and mcap-block.pprof
`
Compression is successful but with the default parameters only, the --compression and --chunk-size have an impact but not the config parameters.

Especially, the top command of the pprof cpu shows me that the same compression level is used even with different compression level settings.
1790ms 37.53% 37.53% 2680ms 56.18% github.com/klauspost/compress/zstd.(*doubleFastEncoder).Encode

Also tried using a string after the --config flag to set parameters with different formatting with no success.

Would there be a documentation or example config file that I missed ?

[bryfox]

I believe the CLI's --config provides you an alternate way to specify configuration for the specific command option. In the case of compress that would allow you to set --chunk-size and --compression ("zstd", "lz4", or "none") — but it doesn't provide access to all of the writer options. You could write your own code to do this — CPP you linked above; Go (options below); or one of the other libraries.

mcap/go/mcap/writer.go

Line 781 in 00eddd3

type WriterOptions struct {

[james-rms]

One way to do this is with ros2 bag convert

You can:

1. create output_options.yaml

output_bags:
- uri: output_path  # required
   all_topics: true
   storage_config_uri: "storage_config.yaml"

2. Fill storage_config.yaml with your McapWriterOptions as you have done for .mcap.yaml
3. run ros2 bag convert --input input.mcap --output_options output_options.yaml
4. Look at your new MCAP files in output_path