Merge master beyond 8.4 without some tests #399
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ikv#17121)" (tikv#17275) close tikv#17258 This reverts commit 273ec86. The previous commit introduced a bug that could result in data loss. Specifically, the region worker might delete data from a range where a snapshot has just been applied. Signed-off-by: Bisheng Huang <hbisheng@gmail.com>
…17279) ref tikv#16265 Fix incorrect format unit of the `grpc source requests' duration` panel. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
…ikv#17279)" (tikv#17280) ref tikv#16265 Revert duplicated changes. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
close tikv#17265 Signed-off-by: Wenqi Mou <wenqimou@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17291 Ignore RUSTSEC-2024-0357 (unsound `openssl::bio::MemBio::get_buf`) as we have banned relevant APIs in the TiKV codebase. Also, TiKV temporarily pins OpenSSL 1.1.1 because OpenSSL 3.0.0 has performance regressions. See details in deny.toml. Signed-off-by: Neil Shen <overvenus@gmail.com>
close tikv#17282 fix gc may filter keys of onging snapshot Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
…ange (tikv#17261) close tikv#17259 fix repeatable schedule delete-range for the same range Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16890 Signed-off-by: purelind <purelind@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17294 Fix too large slow log Signed-off-by: Connor1996 <zbk602423539@gmail.com>
…ikv#17029) ref tikv#16907 Introduce a simple algorithm to make the batching mechanism on WriteBatch larger as expected, for reducing the cost on IO resources. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16141 add eviction duration metrics Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
…#17297) ref tikv#14474 1. Refactor the `ProcessWrite` function in the transaction scheduler. 2. Add `start_ts` information to the `AlreadyExist` error types, so it would be easier to know the conflict write record and related transactions. 3. Abstract memory quota count into task allocation, ensure the re-scheduled commands and tasks are considered by the memory quota statistics. Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
…cription better (tikv#17274) close tikv#16362 Signed-off-by: qupeng <qupeng@pingcap.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17307 Signed-off-by: rishabh_mittal <mittalrishabh@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17300 Signed-off-by: Wenqi Mou <wenqimou@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Co-authored-by: wuhuizuo <wuhuizuo@126.com>
…#17326) close tikv#16229 Reduce the memory usage of peers' message channel Signed-off-by: Connor1996 <zbk602423539@gmail.com>
…ait duration in cop detail (tikv#17324) ref tikv#16141 show index/select-by-range-cache-engine in handle/wait duration in cop detail Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#17309 This pr make the default setting of `coprocessor.region-split-size` larger, from `96MB` to `256MB` by default. The new setting for `region-split-size` is compatible to the larger cluster. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com> Co-authored-by: Bisheng Huang <hbisheng@gmail.com>
ref tikv#16141 show eviction reasons in metrics Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
close tikv#17299 gc the range immediately after load Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17272 TiKV no longer names bloom filter blocks with suffix like "FullBloom" or "Ribbon". Signed-off-by: Yang Zhang <yang.zhang@pingcap.com>
close tikv#17356 Make the diskfull check mechanism compatible to the configuration `raft-engine.spill-dir`. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#14474 Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
…'s priority is different from that of apply poller. (tikv#17338) close tikv#17278 remove the behavior - immediately persist when the command's priority is different from that of apply poller. Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17360 remove scheduler.schedule(Task::MarkFailover(Instant::now()))?; as not needed anymore rename load_task to register_stream_task so can be more explicit of what the method is doing rename Service to BackupStreamGrpcService to be more explicit rename StreamTaskInfo to StreamTaskHandler as it contains methods acting as reactors on stream task events. It's more like a handler rather than read-only info. remove unused metrics tikv_backup_request_duration_seconds remove unused code handle_fatal_error Signed-off-by: Wenqi Mou <wenqimou@gmail.com>
ref tikv#15990 Fix io flow stat for snapshot sending. Its io flow is not taken into account previously because the io flow is recorded only when using `file_system::File` which is a wrapper of `std::fs::File`. Signed-off-by: Connor1996 <zbk602423539@gmail.com>
ref tikv#16141, close tikv#17147 Move range cache engine's range manager for range-base to region-base Signed-off-by: glorv <glorvs@163.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
…ethod (tikv#17357) close tikv#17368 * add one log to indicate the memory quota is freed when drop the `Drain` * free the truncated scanned event memory quota. * refactor `finish_scan_lock` method, to remove the else branch. * row size calculation should also consider old value * remove some outdate todo Signed-off-by: 3AceShowHand <jinl1037@hotmail.com> Signed-off-by: Ling Jin <7138436+3AceShowHand@users.noreply.github.com> Co-authored-by: Neil Shen <overvenus@gmail.com>
ref tikv#17290 Signed-off-by: EricZequan <110292382+EricZequan@users.noreply.github.com> Signed-off-by: “EricZequan” <zequany33@gmail.com> Co-authored-by: EricZequan <110292382+EricZequan@users.noreply.github.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
…ikv#17672) close tikv#17671 Supply necessary dependencies on `coprocessor_plugin` crates to make the CI normal. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17676 Fix grafana panels parse error and legend format Signed-off-by: Connor1996 <zbk602423539@gmail.com>
close tikv#17667 Fix the bug that when region is split before gc task start, the gc task only mark in_gc state for the region with the same id, but will unset it for all the regions within the given range. NOTE: for implementation simplicity, after this change, in the condition, the gc will only run on the region with the same id, other regions in the same range will skip this GC round. Signed-off-by: glorv <glorvs@163.com>
ref tikv#16141 consider tikv gc in cross check Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16141, close tikv#17626 evict on flashback Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#16141 * Unify in memory engine logs * Unify in memory engine failpoints * Correct range cache engine to in memory engine Signed-off-by: Neil Shen <overvenus@gmail.com>
…xists when schedule load (tikv#17685) close tikv#17645 fix data inconsistency due to unflushed data that exists when schedule load Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16141 display safe point and safe point gap Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
…ons metadata (tikv#17677) ref tikv#16141 Add a "/cached_regions" http inteface to fetch current cached regions metadata. Signed-off-by: glorv <glorvs@163.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16141 display region cache hit related metrics Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16141 Add a benchmark to test in-memory-engine load region performance. Signed-off-by: glorv <glorvs@163.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16141 display put/delete operations per second Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au> Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17700 Call `wb.clear` even if the write batch is empty to avoid leaving `RegionCacheWriteBatch` in a corrupted state. Signed-off-by: glorv <glorvs@163.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
…y engine (tikv#17702) ref tikv#16141 copy read throughput and latency panels to in-memory engine Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
ref tikv#16141, close tikv#17644 Evict region when destroy peer Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
close tikv#17707 fix compile fail for test Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
close tikv#17691 Update RocksDB to include fixes for ingestion * tikv/rocksdb@31eb76b * tikv/rocksdb@cf9cd5b Signed-off-by: Yang Zhang <yang.zhang@pingcap.com>
…act (tikv#17725) close tikv#17269 add test for triggering compaction in CheckAndCompact Signed-off-by: SpadeA-Tang <tangchenjie1210@gmail.com>
ref tikv#17181 This commit is a follow-up of tikv#17359. Since `HybridEngine` is now integrated into TiKV through coprocessor observers and RaftKv, it no longer needs to implement the Engine traits. This commit removes the unnecessary boilerplate code for `HybridEngine`. Signed-off-by: Neil Shen <overvenus@gmail.com> Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ref tikv#16890 Signed-off-by: purelind <purelind@gmail.com>
close tikv#17731 Add deprecated RocksDB option to avoid panic upgrading from TiKV 6.1.x to 8.4+ Signed-off-by: Yang Zhang <yang.zhang@pingcap.com>
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
|
|
@CalvinNeo: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is changed and how it works?
Issue Number: Close #xxx
What's Changed:
Related changes
pingcap/docs/pingcap/docs-cn:Check List
Tests
Side effects
Release note