Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve memory utility with large txn #9722

Open
CalvinNeo opened this issue Dec 11, 2024 · 0 comments · Fixed by #9763
Open

Improve memory utility with large txn #9722

CalvinNeo opened this issue Dec 11, 2024 · 0 comments · Fixed by #9763
Assignees
@CalvinNeo
Labels
component/storage type/enhancement The issue or PR belongs to an enhancement.

Comments

@CalvinNeo
Copy link
Member

Enhancement
@CalvinNeo CalvinNeo added the type/enhancement The issue or PR belongs to an enhancement. label Dec 11, 2024
This was referenced Dec 11, 2024
Merged
Merged
ti-chi-bot bot added a commit that referenced this issue Dec 11, 2024
@CalvinNeo @ti-chi-bot
update proxy of master to raftstore-proxy for memory purge and memory…
96ae6d2 
… status (#9721)

ref #4982, ref #9722

- Provide an http interface "/debug/pprof/arena_purge" for purging the dirty/muzzy page in jemalloc. This could help reduce the memory consumption after large txn is committed.
    Reference: https://jemalloc.net/jemalloc.3.html#opt.muzzy_decay_ms
- Provide an http interface "/debug/pprof/memory_status" to get memory allocation information is using jemalloc. Note that the return result may take dozens MiB.

Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot added a commit that referenced this issue Dec 30, 2024
@CalvinNeo @ti-chi-bot @JinheLin
KVStore: Reduce the memory before and after a large txn is committed (#…
08abd71 
…9707)

ref #9722

Avoid store the built DecodedLockCFValue

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Co-authored-by: jinhelin <linjinhe33@gmail.com>
yibin87 pushed a commit to yibin87/tiflash that referenced this issue Jan 3, 2025
@CalvinNeo @ti-chi-bot
@JinheLin @yibin87
KVStore: Reduce the memory before and after a large txn is committed (p…
2c64fe8 
…ingcap#9707)

ref pingcap#9722

Avoid store the built DecodedLockCFValue

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Co-authored-by: jinhelin <linjinhe33@gmail.com>
@CalvinNeo CalvinNeo mentioned this issue Jan 3, 2025
Merged
12 tasks
@ti-chi-bot ti-chi-bot bot closed this as completed in #9763 Jan 7, 2025
@ti-chi-bot ti-chi-bot bot closed this as completed in 1b23623 Jan 7, 2025
@CalvinNeo CalvinNeo reopened this Jan 17, 2025
This was referenced Jan 17, 2025
Merged
Merged
ti-chi-bot bot pushed a commit that referenced this issue Jan 22, 2025
@CalvinNeo
Re-enable jemalloc's background thread by default (#9813)
9832885 
ref #9722, close #9812

Re-enable jemalloc's background thread by default

Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@ti-chi-bot ti-chi-bot mentioned this issue Jan 22, 2025
Merged
12 tasks
ti-chi-bot bot pushed a commit that referenced this issue Jan 23, 2025
@ti-chi-bot @CalvinNeo @JaySon-Huang
Re-enable jemalloc's background thread by default (#9813) (#9814)
65a91a3 
ref #9722, close #9812

Re-enable jemalloc's background thread by default

Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
Signed-off-by: JaySon-Huang <tshent@qq.com>

Co-authored-by: Calvin Neo <CalvinNeo@users.noreply.github.com>
Co-authored-by: JaySon-Huang <tshent@qq.com>
Co-authored-by: JaySon <tshent@qq.com>
This was referenced Jan 23, 2025
Closed
Closed
Open
Open
ti-chi-bot bot pushed a commit that referenced this issue Jan 25, 2025
@CalvinNeo @JaySon-Huang
KVStore: Record decoded memory of each Region (#9780)
d27d7c0 
ref #9722

Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>

Co-authored-by: JaySon-Huang <tshent@qq.com>
@CalvinNeo CalvinNeo mentioned this issue Feb 5, 2025
Open
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@CalvinNeo CalvinNeo

Labels
component/storage type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

KVStore: Fix unreleased Region instance CalvinNeo/tiflash
2 participants
@JaySon-Huang @CalvinNeo