[FEA] decide skip agg or not based on not only first batch

[binmahone]

This is a follow up issue on #11712 per requested by #11712 (comment).

(for skip agg logic) :

Right now we check the first batch and decide for the entire task if we want to skip full aggregation or not. I think it would probably be better if we did it for each batch individually. We could even do it for each batch when we merge them too.

This priority of this issue depends on whether we'll need it in real customer cases. For now we just leave it as a TODO

[revans2]

I just noticed that Spark has a very similar feature that is in all versions of Spark that we support.

https://issues.apache.org/jira/browse/SPARK-31973 We may want to go off of this and the corresponding configs as a starting point to evaluate what we really want to do.

[binmahone]

@revans2 https://issues.apache.org/jira/browse/SPARK-31973 looks like in progress, why you mentioned " is in all versions of Spark that we support." ?