feat: Add option to configure NATS max_payload in JetStream eventbus

[daniel-codefresh]

Solves #2163

[whynowy]

@daniel-codefresh - Sorry, I might have clicked it too fast, max_payload should not be exposed with this approach, instead, it should be done at https://github.com/argoproj/argo-events/blob/master/eventbus/jetstream/base/jetstream.go#L131.

Could you make the change?

[daniel-codefresh]

@daniel-codefresh - Sorry, I might have clicked it too fast, max_payload should not be exposed with this approach, instead, it should be done at https://github.com/argoproj/argo-events/blob/master/eventbus/jetstream/base/jetstream.go#L131.

Could you make the change?

@whynowy not sure I follow - do you mean that max_payload should be passed as part of streamConfig ? if so then I'm not sure it can be done because max_payload should be part of the nats server config and not part of the stream config..
Of course we can create a new serverConfig field to allow users to pass a yaml similarly to how its done with the stream config or with the jetstream settings if thats what you mean (nats.conf):

[whynowy]

A final control is the Maximum Size any single message may have. NATS have it's own limit for maximum size (1 MiB by default), but you can say a Stream will only accept messages up to 1024 bytes using MaxMsgSize.

https://docs.nats.io/using-nats/developer/develop_jetstream/model_deep_dive

Max message size can be configured when creating a stream, see the source code below.

https://github.com/nats-io/nats.go/blob/main/jsm.go#L104

[daniel-codefresh]

A final control is the Maximum Size any single message may have. NATS have it's own limit for maximum size (1 MiB by default), but you can say a Stream will only accept messages up to 1024 bytes using MaxMsgSize.

https://docs.nats.io/using-nats/developer/develop_jetstream/model_deep_dive

Max message size can be configured when creating a stream, see the source code below.

https://github.com/nats-io/nats.go/blob/main/jsm.go#L104

@whynowy maxMsgSize in stream config is respected only if it's smaller than max_payload in NATS config, otherwise max_payload has an higher priority. For instance, if you define a maxMsgSize of 2MB in the stream config and you use the default value of max_payload which is 1MB, then every message above 1MB would still get rejected.