SQL: Fix bug caused by empty composites

[costin]

When dealing with filtering, a composite aggregation might return empty
buckets (which have been filtered) which gets sent as is to the client.
Unfortunately this interprets the response as no more data instead of
retrying.

This now has changed and the listener keeps retrying until either the
query has ended or data passes the filter.

Fix #30292

[costin]

@nik9000 is this okay for chaining listeners, in particular the return for the current call?

[nik9000]

In general chaining listeners is fine. I don't tend to use the return early style in listeners because I feel like they should look symmetric. They just feel better to me when they do. I use the early return style everywhere else though.

The only concern with chaining listeners like this is that some APIs are synchronous sometimes and if you chain on a synchronous API you'll get a stack overflow. The initial phase of search has this problem. I don't know if you have that problem here but I'd dig through the code to make super sure you don't.

[nik9000]

You shouldn't need to catch (Exception here because the caller already does and will forward any exceptions you throw to onFailure.

[costin]

got it.

[costin]

While updating the code, I was reminded why I added it in the first place - serializeQuery throws an IOException which needs to be caught so it's easy to just pass it to the listener.
I cannot use wrap directly because I need to pass the listener instance to it again...

[nik9000]

I understand with wrap. I think it'd be a little more clear if you just caught the IOException and sent it to onFailure but what you have will work as well.

[nik9000]

In general chaining listeners is fine. I don't tend to use the return early style in listeners because I feel like they should look symmetric. They just feel better to me when they do. I use the early return style everywhere else though.

The only concern with chaining listeners like this is that some APIs are synchronous sometimes and if you chain on a synchronous API you'll get a stack overflow. The initial phase of search has this problem. I don't know if you have that problem here but I'd dig through the code to make super sure you don't.

[nik9000]

Same deal here. Make sure it isn't going to stack overflow. It probably won't be we should make sure.

[costin]

I've ran the test suite with the troublesome seed and without (plus ran the test separately with the page set to 1 - this is what triggered the bug). Anything else I should watch out for?

[nik9000]

You pretty much have to read the code to make sure it won't happen. With search it only came up because dozens of shards for the search were on the local node. With this it'd come up if the composite agg needed to be retried hundreds of times and the API doesn't otherwise go async.

I'd dig through the client implementation. I think this is safe because you are being called on the listener thread pool and the search is running on the search threadpool but it is worth double checking.

[costin]

I like the return in this case because it bails out from the listening method. the if/else might be nested inside another block (like here) and some code might be hanging towards the end.

[costin]

Pinging @elastic/es-search-aggs