fix: update elastic.events.queued with the correct number of queued events

[kruskall]

Because of document retries the number of queued events after a flush request can be >0 if the returned 429.
To fix the issue we only remove (failed+indexed) from the queued event metric after a flush request.

Closes #150

[marclop]

Thanks for the fix. Can you please add a test to verify this works as expected?

[kruskall]

Thanks for the fix. Can you please add a test to verify this works as expected?

I tried but it seems to require non-trivial work. I'd prefer to open a followup issue for that without bocking this PR.

[marclop]

I'm not convinced about pushing out tests for a bugfix. What's non-trivial about the test that needs to be added?

[simitt]

@kruskall could you follow up on the last comment regarding the complexity of adding a test? I'm keen on getting this in, but right now the PR is stalling without additional tests.

[kruskall]

I've added a test.

We have no test to validate metrics when document retries are enabled and can't really control the flush/retry behaviour so I had to "cheat":

• only use 1 max request
• use an explicit flush bytes to control the flush behaviour (this is fragile and can break really easily if we change something)

[kruskall]

Before the fix the test fails and returns an Active and queued metric of -1