[FLASH-575] Improve the performance of stream operations in DM #296
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -387,7 +391,7 @@ BlockInputStreams DeltaMergeStore::read(const Context & db_context, | |||
| task.ranges, | |||
| {}, | |||
| max_version, | |||
| std::min(expected_block_size, DEFAULT_BLOCK_SIZE)); | |||
| std::max(expected_block_size, STABLE_CHUNK_ROWS)); | |||
There was a problem hiding this comment.
Maybe we should use min rather than max? The signature is IStorage::read( ..., size_t max_block_size, ...)
There was a problem hiding this comment.
Normally expected_block_size = DEFAULT_BLOCK_SIZE (65536)。While STABLE_CHUNK_ROWS = 8192 。
flowbehappy
force-pushed
the
dm-performance
branch
from
a1f4966 to
4cdd5fa
Compare
|
/run-integration-tests |
5 similar comments
|
/run-integration-tests |
|
/run-integration-tests |
|
/run-integration-tests |
|
/run-integration-tests |
|
/run-integration-tests |
flowbehappy
force-pushed
the
dm-performance
branch
from
abb41fc to
94167b4
Compare
|
/run-integration-tests |
1 similar comment
|
/run-integration-tests |
|
/build |
|
/rebuild |
|
/run-integration-tests |
|
/rebuild |
|
/run-integration-tests |
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Oct 30, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 1, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 1, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
JaySon-Huang
pushed a commit
to JaySon-Huang/tiflash
that referenced
this pull request
Nov 5, 2019
…ap#296) * do handle range filter in * add status support for DM * compact continue insert * bug fix, and temporary split big insert block * enable split logical * add store restore log * fix lock in PageEntriesVersionSetWithDelta::listAllLiveFiles * disable posix_fadvise in PageStorage * Update default DM settings * optimize DMVersionFilterBlockInputStream * optimize DMVersionFilterBlockInputStream v2 * added some metrics * add dm_insert_max_rows setting * Fix compilation error of TiDB.h * address comment * fix broken unit tests * fix unit tests of DeltaTree * Bug fix: DiskValueSpace::createAppendTask remove data pages by mistake * remove debug code
12 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
mange table xxx statuscommand, to show the metrics about DM table.