Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

opt,tree: fix bugs with Next(), Prev(), and histogram calculation for DTimeTZ #74914

Merged
merged 2 commits into from
Jan 19, 2022

Conversation

rytaft
Copy link
Collaborator

@rytaft rytaft commented Jan 17, 2022

sql/sem/tree: fix Next() and Prev() for DTimeTZ

Prior to this commit, the DTimeTZ functions Next() and Prev()
could skip over valid values according to the ordering of DTimeTZ
values in an index (which matches the ordering defined by the
TimeTZ functions After() and Before()).

This commit fixes these functions so that Next() now returns the smallest
valid DTimeTZ that is greater than the receiver, and Prev() returns
the largest valid DTimeTZ that is less than the receiver. This is
an important invariant that the optimizer relies on when building index
constraints.

Fixes #74912

Release note (bug fix): Fixed a bug that could occur when a TIMETZ
column was indexed, and a query predicate constrained that column using
a < or > operator with a timetz constant. If the column contained values
with time zones that did not match the time zone of the timetz constant,
it was possible that not all matching values could be returned by the
query. Specifically, the results may not have included values within one
microsecond of the predicate's absolute time. This bug was introduced
when the timetz datatype was first added in 20.1. It exists on all
versions of 20.1, 20.2, 21.1, and 21.2 prior to this patch.

opt: fix bug in histogram calculation for TimeTZ

This commit fixes a bug in the histogram estimation code for TimeTZ
that made the faulty assumption that TimeTZ values are ordered by
TimeOfDay. This is incorrect since it does not take the OffsetSecs
into account. As a result, it was possible to estimate that the size
of a histogram bucket was negative, which caused problems in the
statistics estimation code. This commit fixes the problem by taking
into account both TimeOfDay and OffsetSecs when estimating the size of
a bucket in a TimeTZ histogram.

Fixes #74667

Release note (bug fix): Fixed an internal error, "estimated row count must
be non-zero", that could occur during planning for queries over a table
with a TimeTZ column. This error was due to a faulty assumption in the
statistics estimation code about ordering of TimeTZ values, which has now
been fixed. The error could occur when TimeTZ values used in the query had
a different time zone offset than the TimeTZ values stored in the table.

@rytaft rytaft requested review from otan, RaduBerinde and a team January 17, 2022 18:45
@rytaft rytaft requested a review from a team as a code owner January 17, 2022 18:45
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@otan
Copy link
Contributor

otan commented Jan 17, 2022

ah a big oopsie :\ looks good to me!

Copy link
Collaborator

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find! :lgtm:

Reviewed 4 of 4 files at r1, 3 of 3 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @RaduBerinde and @rytaft)


pkg/sql/logictest/testdata/logic_test/timetz, line 697 at r1 (raw file):


query I
SELECT count(*) FROM regression_74912 WHERE t2 > '05:00:00';

nit: you could force the index here and below to ensure the test will always test what it's supposed to

Prior to this commit, the DTimeTZ functions Next() and Prev()
could skip over valid values according to the ordering of DTimeTZ
values in an index (which matches the ordering defined by the
TimeTZ functions After() and Before()).

This commit fixes these functions so that Next() now returns smallest
valid DTimeTZ that is greater than the receiver, and Prev() returns
the largest valid DTimeTZ that is less than the receiver. This is
an important invariant that the optimizer relies on when building index
constraints.

Fixes cockroachdb#74912

Release note (bug fix): Fixed a bug that could occur when a TIMETZ
column was indexed, and a query predicate constrained that column using
a < or > operator with a timetz constant. If the column contained values
with time zones that did not match the time zone of the timetz constant,
it was possible that not all matching values could be returned by the
query. Specifically, the results may not have included values within one
microsecond of the predicate's absolute time. This bug was introduced
when the timetz datatype was first added in 20.1. It exists on all
versions of 20.1, 20.2, 21.1, and 21.2 prior to this patch.
Copy link
Collaborator Author

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTRs!

bors r+

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner and @RaduBerinde)


pkg/sql/logictest/testdata/logic_test/timetz, line 697 at r1 (raw file):

Previously, mgartner (Marcus Gartner) wrote…

nit: you could force the index here and below to ensure the test will always test what it's supposed to

There's only one index (the primary index), although I guess maybe I could force using a constrained scan -- Done.

@mgartner
Copy link
Collaborator


pkg/sql/logictest/testdata/logic_test/timetz, line 697 at r1 (raw file):

Previously, rytaft (Rebecca Taft) wrote…

There's only one index (the primary index), although I guess maybe I could force using a constrained scan -- Done.

Oh, oops. Forcing a constrained scan is great!

Copy link
Collaborator

@mgartner mgartner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 5 files at r3, 4 of 4 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @RaduBerinde)

@craig
Copy link
Contributor

craig bot commented Jan 18, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Jan 18, 2022

Build failed (retrying...):

@rytaft
Copy link
Collaborator Author

rytaft commented Jan 18, 2022

bors r-

@craig
Copy link
Contributor

craig bot commented Jan 18, 2022

Canceled.

This commit fixes a bug in the histogram estimation code for TimeTZ
that made the faulty assumption that TimeTZ values are ordered by
TimeOfDay. This is incorrect since it does not take the OffsetSecs
into account. As a result, it was possible to estimate that the size
of a histogram bucket was negative, which caused problems in the
statistics estimation code. This commit fixes the problem by taking
into account both TimeOfDay and OffsetSecs when estimating the size of
a bucket in a TimeTZ histogram.

Fixes cockroachdb#74667

Release note (bug fix): Fixed an internal error, "estimated row count must
be non-zero", that could occur during planning for queries over a table
with a TimeTZ column. This error was due to a faulty assumption in the
statistics estimation code about ordering of TimeTZ values, which has now
been fixed. The error could occur when TimeTZ values used in the query had
a different time zone offset than the TimeTZ values stored in the table.
Copy link
Collaborator Author

@rytaft rytaft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bors r+

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner and @RaduBerinde)

@craig
Copy link
Contributor

craig bot commented Jan 19, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Jan 19, 2022

Build failed:

@rytaft
Copy link
Collaborator Author

rytaft commented Jan 19, 2022

bors r+

@craig
Copy link
Contributor

craig bot commented Jan 19, 2022

Build failed:

@rytaft
Copy link
Collaborator Author

rytaft commented Jan 19, 2022

Another flake

bors r+

@craig
Copy link
Contributor

craig bot commented Jan 19, 2022

Build succeeded:

@craig craig bot merged commit 419167d into cockroachdb:master Jan 19, 2022
@blathers-crl
Copy link

blathers-crl bot commented Jan 19, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from ad7bf04 to blathers/backport-release-21.1-74914: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 21.1.x failed. See errors above.


error creating merge commit from ad7bf04 to blathers/backport-release-21.2-74914: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 21.2.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-21.1.x 21.1 is EOL
Projects
None yet
4 participants