Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): introduce append batch #182

Merged
merged 1 commit into from
Oct 21, 2022
Merged

Conversation

ijsong
Copy link
Member

@ijsong ijsong commented Oct 13, 2022

What this PR does

There were already two kinds of batches in storage:

  • The write batch saves only data keyed by the LLSN.
  • The commit batch saves only the commit context and GLSN, and the commit context must exist.

This patch introduces a new batch type - append batch. The append batch saves both log entries and
a commit context. However, it is not necessary to keep the commit context. During synchronization,
the append batch must store log entries from a destination replica.

Which issue(s) this PR resolves

Updates #125

@ijsong ijsong requested a review from hungryjang as a code owner October 13, 2022 07:24
@ijsong ijsong self-assigned this Oct 13, 2022
@codecov-commenter
Copy link

codecov-commenter commented Oct 13, 2022

Codecov Report

Base: 62.91% // Head: 62.96% // Increases project coverage by +0.05% 🎉

Coverage data is based on head (0534ad0) compared to base (7b67ba2).
Patch coverage: 69.76% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #182      +/-   ##
==========================================
+ Coverage   62.91%   62.96%   +0.05%     
==========================================
  Files         126      127       +1     
  Lines       17178    17191      +13     
==========================================
+ Hits        10807    10824      +17     
- Misses       5815     5817       +2     
+ Partials      556      550       -6     
Impacted Files Coverage Δ
internal/storage/testing.go 36.36% <0.00%> (+10.55%) ⬆️
internal/storage/append_batch.go 82.35% <82.35%> (ø)
internal/storage/storage.go 88.15% <100.00%> (+0.11%) ⬆️
internal/storagenode/reportcommit_server.go 70.51% <0.00%> (-7.67%) ⬇️
internal/metarepos/report_collector.go 76.42% <0.00%> (-1.70%) ⬇️
internal/storagenode/logstream/executor.go 80.20% <0.00%> (-1.21%) ⬇️
...metarepos/dummy_storagenode_client_factory_impl.go 63.90% <0.00%> (-0.72%) ⬇️
internal/metarepos/raft_metadata_repository.go 72.24% <0.00%> (+0.26%) ⬆️
internal/metarepos/raft.go 80.41% <0.00%> (+0.41%) ⬆️
internal/metarepos/storage.go 81.13% <0.00%> (+1.16%) ⬆️
... and 1 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@ijsong ijsong marked this pull request as draft October 13, 2022 07:46
There were already two kinds of batches in storage:

- The write batch saves only data keyed by the LLSN.
- The commit batch saves only the commit context and GLSN, and the commit context must exist.

This patch introduces a new batch type - append batch. The append b catch saves both log entries and
a commit context. However, it is not necessary to keep the commit context. During synchronization,
the append batch must store log entries from a destination replica.

Updates kakao#125
@ijsong ijsong force-pushed the add-log-entry-batch branch from d7d5667 to 0534ad0 Compare October 13, 2022 08:12
@ijsong ijsong marked this pull request as ready for review October 13, 2022 08:12
@ijsong ijsong merged commit 50df027 into kakao:main Oct 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants