Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tweak to Save Memory Consumption When Compiling Dictionaries #661

Merged
merged 5 commits into from
Jun 11, 2023

Conversation

WhiredPlanck
Copy link
Contributor

Pull request

Issue tracker

Fixes will automatically close the related issue

Fixes #583

Feature

Describe feature of pull request

Use several techniques to try to save the memory consumption when compiling dictionaries.

This may save about 30% memory consumption.

Unit test

  • Done

Manual test

  • Done

Code Review

  1. Unit and manual test pass
  2. GitHub Action CI pass
  3. At least one contributor reviews and votes
  4. Can be merged clean without conflicts
  5. PR will be merged by rebase upstream base

Additional Info

Tested schema: rime-ice

Before

b1

b2

After

a1

a2

- Use smart pointer
- Release memory in time

This may save about 12% memory usage when compiling dict.
This may reduce about 1% peak memory usage when compiling dict.
.cache is usually used by clangd
This may save about 18% peak memory consumption when compiling dict.
src/rime/dict/dict_compiler.cc Show resolved Hide resolved
src/rime/dict/dict_compiler.cc Outdated Show resolved Hide resolved
@@ -30,20 +30,36 @@ class Code : public vector<SyllableId> {
string ToString() const;
};

struct ShortDictEntry {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much more memory efficient is this?

If the diff by this single change is insignificant, I'd prefer not to repeat the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The screenshots I pasted in the Additional Info have showed the info you get interested. In my test case, it may save about 15 % ~ 18 % of the memory consumption.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

构建 .table.bin 文件所需的资源和用时是否能优化
2 participants