-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Go writer: support schemas/channels outside chunks #1135
Go writer: support schemas/channels outside chunks #1135
Conversation
Adds a new option, ChannelsOutsideChunks, to the go writer. Today in chunked MCAP files, schemas and channels are inside the chunks. I would like to be able to concatenate two nonoverlapping MCAP files without either decompressing chunks or having duplicate schemas/channels in the output. This option should make it possible to do that.
I think this will require a new conformance test variant.. still studying how to do that. |
IncludeCRC: true, | ||
ChannelsOutsideChunks: true, | ||
}) | ||
require.NoError(t, err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is this require business?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I learned about this recently because my linter started complaining about usage of assert in dp3. Probably when you guys upgrade you'll get warnings too. I don't mind changing this to assert though - will do that.
As far as I understand it this causes the test to bail right away instead of plowing through, but I haven't actually verified the behavior, just that the linter was happy.
@@ -827,6 +829,12 @@ type WriterOptions struct { | |||
// Compressor is a custom compressor. If supplied it will take precedence | |||
// over the built-in ones. | |||
Compressor CustomCompressor | |||
|
|||
// ChannelsOutsideChunks causes channels and schemas to be written outside |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how do you feel about ChunkMessagesOnly
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that's fine. Having difficulty coming up with alternatives.
Closing for lack of activity. If it becomes important we will re-visit. |
Adds a new option, ChannelsOutsideChunks, to the go writer.
Today in chunked MCAP files, schemas and channels are inside the chunks. I would like to be able to concatenate two nonoverlapping MCAP files without either decompressing chunks or having duplicate schemas/channels in the output. This option should make it possible to do that.