Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sqlutil: remove Query and QueryEx from InternalExecutor interface #60428

Merged
merged 1 commit into from
Feb 10, 2021

Conversation

yuzefovich
Copy link
Member

@yuzefovich yuzefovich commented Feb 10, 2021

All usages of sqlutil.InternalExecutor.Query and
sqlutil.InternalExecutor.QueryEx have been refactored: most of
the changes are taking advantage of the iterator API, in a couple
of places we now use Exec and ExecEx methods since in those
places we only need the number of affected rows.

In a few places where we still need to buffer all rows, the caller
currently performs the buffering. In the follow-up commit
QueryBuffered and QueryBufferedEx will be added into the interface
(essentially renaming Query and QueryEx to make it explicit that the
buffering occurs), and the custom logic will be removed.

Addresses: #48595.

Release note: None

@yuzefovich yuzefovich requested a review from a team as a code owner February 10, 2021 16:08
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@yuzefovich
Copy link
Member Author

@andreimatei or @ajwerner, could one of you please take a look at the first commit?

@miretskiy, could you please take a look at the second commit?

Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: neat

Reviewed 16 of 16 files at r1, 8 of 8 files at r2.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andreimatei and @miretskiy)


pkg/jobs/adopt.go, line 283 at r1 (raw file):
Just rambles for myself:

Quoted 6 lines of code…
		// Note that we have to buffer all rows first - before processing each
		// job - because we have to make sure that the query executes without an
		// error (otherwise, the system.jobs table might diverge from the jobs
		// registry).
		// TODO(yuzefovich): use QueryBufferedEx method once it is added to
		// sqlutil.InternalExecutor interface.

I'm not so sure about this. We could just make it two loops but I guess at that point, it's the same complexity in terms of memory.

Also, we could add a LIMIT here if we wanted without violating correctness.

Copy link
Member Author

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andreimatei and @miretskiy)


pkg/jobs/adopt.go, line 283 at r1 (raw file):

Previously, ajwerner wrote…

Just rambles for myself:

		// Note that we have to buffer all rows first - before processing each
		// job - because we have to make sure that the query executes without an
		// error (otherwise, the system.jobs table might diverge from the jobs
		// registry).
		// TODO(yuzefovich): use QueryBufferedEx method once it is added to
		// sqlutil.InternalExecutor interface.

I'm not so sure about this. We could just make it two loops but I guess at that point, it's the same complexity in terms of memory.

Also, we could add a LIMIT here if we wanted without violating correctness.

I tried avoiding the buffering of rows here, but the CI was failing (I think something about auto stats collection, but don't remember for sure) which I bisected to this spot. The comment is my best explanation for why the buffering is needed.

All usages of `sqlutil.InternalExecutor.Query` and
`sqlutil.InternalExecutor.QueryEx` have been refactored: most of
the changes are taking advantage of the iterator API, in a couple
of places we now use `Exec` and `ExecEx` methods since in those
places we only need the number of affected rows.

In a few places where we still need to buffer all rows, the caller
currently performs the buffering. In the follow-up commit
`QueryBuffered` and `QueryBufferedEx` will be added into the interface
(essentially renaming `Query` and `QueryEx` to make it explicit that the
buffering occurs), and the custom logic will be removed.

Release note: None
@yuzefovich yuzefovich changed the title sqlutil: clean up the InternalExecutor interface sqlutil: remove Query and QueryEx from InternalExecutor interface Feb 10, 2021
@yuzefovich
Copy link
Member Author

I removed the second commit and will merge it in a separate PR.

TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Feb 10, 2021

This PR was included in a batch that was canceled, it will be automatically retried

@craig
Copy link
Contributor

craig bot commented Feb 10, 2021

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Feb 10, 2021

Build succeeded:

@craig craig bot merged commit 0ee3587 into cockroachdb:master Feb 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants