Add rocksdb_block_cache_numshardbits for issue 1336 #1339
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This fixes facebook#1336 The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case.
|
@hermanlee has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@mdcallag has updated the pull request. You must reimport the pull request before landing. |
Done |
facebook-github-bot
pushed a commit
that referenced
this pull request
Jul 20, 2023
Summary: This fixes #1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: #1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
luqun
pushed a commit
to luqun/mysql-5.6
that referenced
this pull request
Aug 10, 2023
Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
hermanlee
pushed a commit
to hermanlee/mysql-5.6
that referenced
this pull request
Oct 3, 2023
Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
hermanlee
pushed a commit
to hermanlee/mysql-5.6
that referenced
this pull request
Oct 18, 2023
Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Dec 20, 2023
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Dec 20, 2023
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Dec 21, 2023
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Jan 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to percona/percona-server
that referenced
this pull request
Jan 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Jan 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Jan 17, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/percona-server
that referenced
this pull request
Feb 8, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Feb 9, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Feb 9, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Feb 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
Apr 15, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
May 24, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
oleksandr-kachan
pushed a commit
to oleksandr-kachan/percona-server
that referenced
this pull request
May 27, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
May 30, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
VarunNagaraju
pushed a commit
to VarunNagaraju/percona-server
that referenced
this pull request
May 31, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
VarunNagaraju
pushed a commit
to VarunNagaraju/percona-server
that referenced
this pull request
Jun 5, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 6, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 6, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 7, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 10, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 10, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
VarunNagaraju
pushed a commit
to VarunNagaraju/percona-server
that referenced
this pull request
Jun 10, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
VarunNagaraju
pushed a commit
to VarunNagaraju/percona-server
that referenced
this pull request
Jun 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
VarunNagaraju
pushed a commit
to VarunNagaraju/percona-server
that referenced
this pull request
Jun 12, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 13, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 13, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 14, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jun 21, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jul 2, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jul 19, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jul 19, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jul 19, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this pull request
Jul 25, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this pull request
Jul 30, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Jul 31, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 2, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 2, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this pull request
Aug 6, 2024
…ok#1339) Summary: This fixes facebook#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook#1339 Differential Revision: D47635762
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this pull request
Aug 21, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this pull request
Aug 28, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
dlenev
pushed a commit
to dlenev/percona-server
that referenced
this pull request
Aug 30, 2024
Upstream commit ID: facebook/mysql-5.6@730887a PS-8951: Merge percona-202305 (https://jira.percona.com/browse/PS-8951) Summary: This fixes facebook/mysql-5.6#1336 This adds the my.cnf options: rocksdb_block_cache_numshardbits This option can be set so that RocksDB to fix the number of block cache shards. The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper. The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case. Pull Request resolved: facebook/mysql-5.6#1339 Differential Revision: D47635762 fbshipit-source-id: 7ca759f9a001dbe1a20978ded5b614c209bd5b1f
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes #1336
This adds the my.cnf options: rocksdb_block_cache_numshardbits
This option can be set so that RocksDB to fix the number of block cache shards.
The default value is -1 to match existing behavior. When -1 RocksDB code will determine the number of block cache shards as min(6, rocksdb_block_cache_size / min_shard_size) and today min_shard_size is 512K for LRU and 32M for Hyper.
The math above frequently results in a block cache with too many small shards when rocksdb_block_cache_size is not too big (a few GB is not too big) and there will be perf problems that are hard to debug in such a case.