Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: EncodingType #78

Closed
bartelink opened this issue Jun 2, 2022 · 1 comment · Fixed by #80
Closed

Feature: EncodingType #78

bartelink opened this issue Jun 2, 2022 · 1 comment · Fixed by #80

Comments

@bartelink
Copy link
Collaborator

bartelink commented Jun 2, 2022

If the IEventData had an encoding: byte that could be used to push the compression in CosmosStore and DynamoStore outward

e.g. 0 can be Raw, 1 can be Deflated for common UTF8 JSON cases

but probably pluggable to allow it to identify an encoding version or maybe flag json vs protobuf etc

ITimelineEvent and IEventCodec would use it to layer compression/decompression and/or custom decoding

With a default that does the conditional compression when encoding (and/or some other way to preserve the existing DX of it being defaulted on for Equinox.DynamoStore)

Equinox.EventStoreDb can map 'Encoding to octet-stream, octet-steam;deflate, and/or probably also a JSON content-encoding

For CodecJsonElement, NodeType.string vs object is used to infer whether a decompression step is required (currently Equinox.CosmosStore has a flag at category level and supports compression/decompression only for unfolds)

eqx dump can then work off it to default to dumping json

Prompted by jet/equinox#331

@bartelink
Copy link
Collaborator Author

The solution in #80 does not go down this road - FsCodec.ITimelineEvent does not surface the encoding type as envisaged above.

The current status is:

  • DynamoStore: round-trips an encoding : int value. This works directly with the FsCodec EncodeWithTryDeflate/EncodeWithoutCompression helpers
  • CosmosStore: could extract the current compression logic (which is only available only for Unfolds) into a generic EncodeWithTryDeflate extension in FsCodec
  • EventStoreDb - could apply similar scheme to that used in DynamoStore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant