batch.max_bytes incompatible with Prometheus remote write sink

[a26nine]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

The Prometheus remote write sink document says that batch.max_bytes works with the sink, but it seems incompatible:

2023-10-25T17:00:37.231123Z ERROR vector::topology: Configuration error. error=Sink "ne_sink_vm_prod": This sink does not allow setting `max_bytes`

Configuration

sinks:
  ne_sink_vm_prod:
    type: prometheus_remote_write
    healthcheck: false
    endpoint: xxx
    auth:
      strategy: basic
      user: xxx
      password: xxx
    batch:
      max_bytes: 5242880
      timeout_secs: 3

Version

0.33.0

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

[neuronull]

Hello and thanks for filing this report.

Would you mind including your configuration for the sink?

[a26nine]

Hello and thanks for filing this report.

Would you mind including your configuration for the sink?

Added.

sinks:
  ne_sink_vm_prod:
    type: prometheus_remote_write
    healthcheck: false
    endpoint: xxx
    auth:
      strategy: basic
      user: xxx
      password: xxx
    batch:
      max_bytes: 5242880
      timeout_secs: 3

[neuronull]

👍 thanks for adding that.

Ok so I reproduced this on v0.33.0 , but on latest from master the issue does not reproduce.

It appears during a very recent change to this sink (#18676) , we fixed this issue.

In the interim you should be able to use the latest nightly build and get past this issue until v0.34.0 is released.