Add DynamicFilter#isAwaitable method

[sopel39]

DynamicFilter#isAwaitable method can be used together with
DynamicFilter#isBlocked to wait for narrowed dynamic filters.
Some dynamic filters cannot be waited for (e.g for replicated joins).
Therefore existing DynamicFilter#isComplete method cannot be used
to determine if one can still wait for dynamic filter to be narrowed
(isComplete could return false while isBlocked would return completed
future immediatelly)

[findepi]

in some cases dynamic filtes cannot be waited for (e.g for replicated joins).
Therefore DynamicFilter#isComplete should return "true" for such cases.

Maybe we need a tri-state logic for this?

[sopel39]

Maybe we need a tri-state logic for this?

I thought about it. However, let's say that you obtain info that DF is not complete yet, but you have no means to block while waiting for it. Then the only mean to wait for it would be via active loop.

Additionally, we want/will be supporting laziness for most DFs (replicated, re-partitioned) with exception for large replicated DFs, which should be rarity (and repartition joins too)

[findepi]

I thought about it. However, let's say that you obtain info that DF is not complete yet, but you have no means to block while waiting for it. Then the only mean to wait for it would be via active loop.

That's up to connector to decide what they do.
For example, it may reconsult the DF during split generation (doesn't require busy wait).
Or, it may throttle split generation temporarily, with the hope that split created later will be cheaper to process.

Even if we do not consider these as viable options today, I am generally concerned about removing factually incorrect information from the API.

If you do not like tri-state, maybe let's rename the method isComplete to eg isAwaitable?

[sopel39]

For example, it may reconsult the DF during split generation (doesn't require busy wait).

It can still do that via io.prestosql.spi.connector.DynamicFilter#getCurrentPredicate. However, connector doesn't have knowledge which DFs (column handles) might be collected.

Perhaps, we should give that knowledge to connector, but not part of this PR.

If you do not like tri-state, maybe let's rename the method isComplete to eg isAwaitable?

I could do that. It's a change in relatively fresh SPI though, which i think is fine

[sopel39]

@findepi ac

[findepi]

or !isBlocked.isDone() ?

[sopel39]

isBlocked will unblock when DF is narrowed down partially, but DF is still awaitable

[findepi]

Yes, but then new isBlocked should be created, right?
So, in every moment current value of isBlocked.isBlocked() could be used here -- am i right?

[sopel39]

Yes, but then new isBlocked should be created, right?

Ah, right. Although futuresLeft > 0; seems more explicit

[sopel39]

Thanks, @findepi