-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
txthrottler: further code cleanup #12902
Merged
harshit-gangal
merged 3 commits into
vitessio:main
from
timvaillancourt:tx-throttler-cleanup-pt2
May 17, 2023
Merged
txthrottler: further code cleanup #12902
harshit-gangal
merged 3 commits into
vitessio:main
from
timvaillancourt:tx-throttler-cleanup-pt2
May 17, 2023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timvaillancourt
requested review from
harshit-gangal,
systay,
shlomi-noach and
rohit-nayak-ps
as code owners
April 15, 2023 22:44
4 tasks
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
If a new flag is being introduced:
If a workflow is added or modified:
Bug fixes
Non-trivial changes
New/Existing features
Backward compatibility
|
harshit-gangal
approved these changes
May 16, 2023
shlomi-noach
approved these changes
May 16, 2023
harshit-gangal
added
Component: Query Serving
and removed
Component: Query Serving
labels
May 16, 2023
@timvaillancourt There is a conflict that needs to be resolved. |
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
timvaillancourt
force-pushed
the
tx-throttler-cleanup-pt2
branch
from
May 16, 2023 20:52
2034d58
to
31f2db4
Compare
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
@harshit-gangal conflict resolved! 🙇 |
timvaillancourt
added a commit
to slackhq/vitess
that referenced
this pull request
Jun 28, 2023
* txthrottler: further code cleanup Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix bad merge resolution Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
4 tasks
timvaillancourt
added a commit
to slackhq/vitess
that referenced
this pull request
Jul 3, 2023
* txthrottler: further code cleanup * Fix bad merge resolution --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
timvaillancourt
added a commit
to slackhq/vitess
that referenced
this pull request
Apr 16, 2024
* txthrottler: further code cleanup Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix bad merge resolution Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
4 tasks
timvaillancourt
added a commit
to slackhq/vitess
that referenced
this pull request
May 16, 2024
* txthrottler: further code cleanup Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix bad merge resolution Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
4 tasks
timvaillancourt
added a commit
to slackhq/vitess
that referenced
this pull request
May 16, 2024
…pt. 2 (#350) * Add priority support to transaction throttler (vitessio#12662) * Add support for criticality query directive, and have TxThrottler respect that Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Remove unused variable Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Fix CI pipeline Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Address PR comments. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Make linter happy & add extra test cases. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Address PR comments. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Fix circular import Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Make linter happy Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Address PR comments: * Invalid criticality in query directive fails the query. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Address PR comments: * Renamed criticality to priority. * Change error handling when parsing the priority from string to integer. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Add missing piece of code that got lost during merge conflict resolution Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Fix vtadmin.js Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Address PR comments Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Fix unit tests (I think) Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Invert polarity of priority values With this change, queries with PRIORITY=0 never get throttled, whereas those with PRIORITY=100 always do (provided there is contention). Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Make linter happy Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Fix flag e2e test Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> --------- Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Add flag to select tx throttler tablet type (vitessio#12174) * Add flag to select tx throttler tablet type Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * REPLICA and/or RDONLY only Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Update flag help msg Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Lowercase types in help/doc Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Help update Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * No underscores in flag Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix merge Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * PR suggestion, consolidate config logic Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Update go/vt/vttablet/tabletserver/tabletenv/config.go Co-authored-by: Andrew Mason <amason@hey.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Use topoproto.TabletTypeListFlag to handle flag Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix unit test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Update go/vt/vttablet/tabletserver/tabletenv/config.go Co-authored-by: Andrew Mason <amason@hey.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * improve test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * pr suggestions Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * go fmt Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Andrew Mason <amason@hey.com> * txthrottler: further code cleanup (vitessio#12902) * txthrottler: further code cleanup Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix bad merge resolution Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * TxThrottler support for transactions outside BEGIN/COMMIT (vitessio#13040) * TxThrottler support for transactions outside BEGIN/COMMIT This change allows the TxThrottler to throttle requests sent outside of explicit transactions (i.e. explicit BEGIN/COMMIT blocks) when configured to do so via a new config flag. Otherwise, it preserves the current/default behavior of only throttling transactions inside BEGIN/COMMIT. In addition, when this flag is passed, and because the call to throttle is done in a context in which the execution plan is already known, this change uses the plan type to make sure that throttling is triggered only when the query being executed is INSERT/UPDATE/DELETE/LOAD, so that SELECTs and others no longer get throttled unnecessarily, as they do not contribute to increasing replication lag, which is what the TxThrottler aims at controlling. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Fix e2e flag tests & TxThrottler unit test Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Throttle auto-commit statements in QueryExecutor instead of TxPool This changes where we call the transaction throttler: 1. Statements in `BEGIN/COMMIT` blocks keep being throttled in `TabletServer.begin()`. 2. Additionally, throttling is added in QueryExecutor.execAutocommit() and `QueryExecutor.execAsTransaction()`. We also change things so that throttling in this new case is not opt-in via configuration flag but instead is the new and only behavior. Finally, we remove some previously added changes to example scripts that had been added with the intention of testing and are not part of the PR. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Adds test cases for QueryExecutor.Execute() with TxThrottle throttling To make unit testing simple here, we separated the interface and implementation of the TxThrottle, and simply used a mock implementation of the interface in the tests. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Add note on new TxThrottler behavior in v17 changelog Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Fix PR number in changelog entry for TxThrottler behavior change. Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Make linter happy Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * Address PR comments Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> --------- Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> * txthrottler: verify config at vttablet startup, consolidate funcs (vitessio#13115) * txthrottler: verify config at vttablet startup, consolidate funcs Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Use explicit dest in prototext.Unmarshal Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Use for loop for TestVerifyTxThrottlerConfig Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Cleanup test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Fix go vet complaint Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Add back synonym flag Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Update go/vt/vttablet/tabletserver/txthrottler/tx_throttler.go Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Address staticcheck linter error Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * gofumpt Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com> Co-authored-by: Andrew Mason <amason@hey.com> Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Building on top of #12901, this PR cleans up
go/vt/vttablet/tabletserver/txthrottler
by:struct
s based on @deepthi's recommendation in Cleanup panics intxthrottler
, reorder for readability #12901:*topo.Server
fromtxThrottlerConfig
struct because*topo.Server
is not really "configuration"*topo.Server
toTxThrottler
struct and passed down insteadfunc init()
to be after thefunc
it calls:func resetTxThrottlerFactories()
Related Issue(s)
#12901
Checklist
Deployment Notes