[Pivottable]Time column formatter doesn't apply correctly when time grain is added

[junlincc]

In Pivot table v1, when time grain is added, and have the temporal column as row or column, time formatter doesn't work properly.

Actual result - the end date is not formatted.

Expected result - in legacy pivot table, both ends show properly.

@kgabryje

[kgabryje]

apache-superset/superset-ui#1315

[rusackas]

Reopening since the superset-ui changes have not been brought over to superset yet. This will take place in #16350 which should close this issue once and for all.