Go writer: support schemas/channels outside chunks #1135
Conversation
Adds a new option, ChannelsOutsideChunks, to the go writer. Today in chunked MCAP files, schemas and channels are inside the chunks. I would like to be able to concatenate two nonoverlapping MCAP files without either decompressing chunks or having duplicate schemas/channels in the output. This option should make it possible to do that.
|
I think this will require a new conformance test variant.. still studying how to do that. |
| IncludeCRC: true, | ||
| ChannelsOutsideChunks: true, | ||
| }) | ||
| require.NoError(t, err) |
There was a problem hiding this comment.
what is this require business?
There was a problem hiding this comment.
I learned about this recently because my linter started complaining about usage of assert in dp3. Probably when you guys upgrade you'll get warnings too. I don't mind changing this to assert though - will do that.
As far as I understand it this causes the test to bail right away instead of plowing through, but I haven't actually verified the behavior, just that the linter was happy.
| @@ -827,6 +829,12 @@ type WriterOptions struct { | |||
| // Compressor is a custom compressor. If supplied it will take precedence | |||
| // over the built-in ones. | |||
| Compressor CustomCompressor | |||
|
|
|||
| // ChannelsOutsideChunks causes channels and schemas to be written outside | |||
There was a problem hiding this comment.
how do you feel about ChunkMessagesOnly?
There was a problem hiding this comment.
I think that's fine. Having difficulty coming up with alternatives.
Adds a new option, ChannelsOutsideChunks, to the go writer.
Today in chunked MCAP files, schemas and channels are inside the chunks. I would like to be able to concatenate two nonoverlapping MCAP files without either decompressing chunks or having duplicate schemas/channels in the output. This option should make it possible to do that.