Enable now() usage in plan-time chunk exclusion

[svenklemm]

This implements an optimization to allow now() expression to be
used during plan time chunk exclusions. Since now() is stable it
would not normally be considered for plan time chunk exclusion.
To enable this behaviour we convert column > now() expressions
into column > const AND column > now(). Assuming that time
always moves forward this is save even for prepared statements.
This optimization works for SELECT, UPDATE and DELETE.
On hypertables with many chunks this can lead to a considerable
speedup for certain queries.

The following expressions are supported:

• column > now()
• column >= now()
• column > now() - Interval
• column > now() + Interval
• column >= now() - Interval
• column >= now() + Interval

Interval must not have a day or month component as those depend
on timezone settings.

Some microbenchmark to show the improvements, I did best of five
for all of the queries.

-- hypertable with 1k chunks
-- with optimization
select * from metrics1k where time > now() - '5m'::interval;
Time: 3.090 ms

-- without optimization
select * from metrics1k where time > now() - '5m'::interval;
Time: 145.640 ms

-- hypertable with 5k chunks
-- with optimization
select * from metrics5k where time > now() - '5m'::interval;
Time: 4.317 ms

-- without optimization
select * from metrics5k where time > now() - '5m'::interval;
Time: 775.259 ms

-- hypertable with 10k chunks
-- with optimization
select * from metrics10k where time > now() - '5m'::interval;
Time: 4.853 ms

-- without optimization
select * from metrics10k where time > now() - '5m'::interval;
Time: 1766.319 ms (00:01.766)

-- hypertable with 20k chunks
-- with optimization
select * from metrics20k where time > now() - '5m'::interval;
Time: 6.141 ms

-- without optimization
select * from metrics20k where time > now() - '5m'::interval;
Time: 3321.968 ms (00:03.322)

Speedup with 1k chunks: 47x
Speedup with 5k chunks: 179x
Speedup with 10k chunks: 363x
Speedup with 20k chunks: 540x

[akuzm]

We already have a more generic variant here: https://github.com/timescale/timescaledb/blob/main/tsl/src/fdw/scan_plan.c#L552-L563
We use it to evaluate stable constraints at the access node before sending them to the data nodes. Can we just use it here?

[akuzm]

That code evaluates many timestamp-related stable functions, not just now(): https://github.com/timescale/timescaledb/blob/main/tsl/src/fdw/deparse.c#L118-L180

[akuzm]

By the way, this is for local hypertables, right? This means we don't need a whitelist and can evaluate all stable functions before performing constraint exclusion. Although I'm not sure if preprocess_queries is the right place to do this, because this happens at planning time, and the stable functions should be evaluated at execution time.

[svenklemm]

This is still a draft so i'll expand the commit message but you cannot just constify now() like that at plan time that would not be safe. What this patch does is transform column > now() - '5m' into column > '2022-05-16 14:49' AND column > now() - '5min'. This allows the constified value to be used during plan time chunk exclusion while still returning correct results even for prepared statements.

[akuzm]

Oh, nice idea!

[codecov]

Codecov Report

Merging #4340 (b8373d3) into main (c6c64c4) will increase coverage by 0.03%.
The diff coverage is 98.19%.

@@            Coverage Diff             @@
##             main    #4340      +/-   ##
==========================================
+ Coverage   90.75%   90.79%   +0.03%     
==========================================
  Files         216      217       +1     
  Lines       39996    40040      +44     
==========================================
+ Hits        36300    36355      +55     
+ Misses       3696     3685      -11     

Impacted Files	Coverage Δ
src/compat/compat.h	94.73% <ø> (ø)
src/copy.c	93.79% <ø> (ø)
src/loader/bgw_launcher.c	91.82% <ø> (ø)
src/nodes/chunk_append/chunk_append.c	97.94% <ø> (ø)
src/planner/add_hashagg.c	52.85% <ø> (ø)
src/planner/agg_bookend.c	92.30% <ø> (ø)
src/planner/expand_hypertable.c	93.94% <ø> (ø)
src/planner/partialize.c	97.91% <ø> (ø)
src/planner/planner.h	100.00% <ø> (ø)
tsl/src/fdw/relinfo.c	96.83% <ø> (-0.02%)	⬇️
... and 17 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 01c6125...b8373d3. Read the comment docs.

[akuzm]

We also have the tsl_override_current_timestamptz thing, need to merge them sometime I guess...

[akuzm]

Tested locally, gives some impressive speedup.

[nikkhils]

LGTM!

[erimatnor]

Do we really need to change the query plan and show the constant expression in the explain to enable this functionality? I worry that it can be confusing?

I am thinking it is enough to add the constant expression to the quals we use to scan for chunks during chunk exclusion. That would make the functionality transparent.

[svenklemm]

Not sure what you are referring to but at the level the optimization is happening there are no scans yet. We need this all to happen way before anything scan related is there because we want to get rid of the planning overhead of excluded chunks.

[svenklemm]

Oh i think I understand what you mean but that would limit the usefulness of the optimization. This optimization works with both our own hypertable expansion and postgres inheritance expansion. That means it applies to SELECT/UPDATE/DELETE. If we would do this only in our own hypertable expansion it would no longer work for UPDATE/DELETE

[erimatnor]

I would have expected tests with prepared statements combined with new data that would affect the plan given that it is important that this optimization works correctly with prepared statements, However, I can't find any such tests.