feat: add stone load and gc workflow #117
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
will-2012
changed the title
will-2012
force-pushed
the
sql_dev
branch
18 times, most recently
from
021bcfa to
500face
Compare
will-2012
changed the title
joeylichang
added a commit
that referenced
this pull request
Feb 13, 2023
* feat: add iterator and batch interface to db * chore: delete memory job db v1 code * feat: impl the iterator and batch of job memory db * feat: add stone load and gc db * chore: delete job sql db v1 impl * feat: job sql add iterator and batcher interface * refactor: rename job_db v2 interface to job_db interface --------- Co-authored-by: joeylichang <joeycli0919@gmail.com>
joeylichang
added a commit
that referenced
this pull request
Feb 26, 2023
* fix: docker push bug (#115) * feat: add stone load and gc workflow (#117) * feat: add iterator and batch interface to db * chore: delete memory job db v1 code * feat: impl the iterator and batch of job memory db * feat: add stone load and gc db * chore: delete job sql db v1 impl * feat: job sql add iterator and batcher interface * refactor: rename job_db v2 interface to job_db interface --------- Co-authored-by: joeylichang <joeycli0919@gmail.com> * feat: implement signer service (#119) * feat: implement signer service * feat: add api auth and ip whitelist for signer * refactor: redesign the interface of singer service * refactor: use greenfield-go-sdk * fix: msg to be signed should not be pre-hashed * feat: add VerifyBucketApproval, VerifyObjectApproval interface * feat: add signer client --------- Co-authored-by: joeylichang <joeycli0919@gmail.com> * feat: client opt (#129) * feat: implement gateway challenge workflow (#127) * feat: implement gateway challenge workflow * feat: support get range object * read:block_syncer * read:block_syncer * read:block-syncer * read:block-syncer * fix: use env var to get bucket url (#130) * fix: use env var to get bucket url * fix: improve loading config from env function * fix: rename variable names * fix: change piece store readme.md --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * read:toml * feat: stone node sends piece data to gateway (#128) * feat: stone node sends piece data to gateway * feat: alloc resp add bucket name and object name * fix: fix errors of stone node uint test * fix: fix gateway config * fix: add gateway syncer rpc ut * fix: add one log in gateway sync piece handler func * fix: add detailed log in gateway sync piece handler --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * read:cmd_complete * read:del * read:test-job * read:block-syncer-frame * read:fix * fix: upgrade greenfield version to fix the signing bug (#133) * ci: fix release job * dep: upgrade greenfield version * fix: unit test for gnfd sign client * fix: signer adapt to on-chain logic (#136) * feat: add metadata service to sp (#132) * feat: init commit * feat: register metadata service and run on 9733 port * feat: impl ListObjectsByBucketName api & model update * feat: cleanup files and add metadata service sdk * feat: add mock and unit tests * feat: uniform error return values * feat: metadata service case_driver update --------- Co-authored-by: BarryTong65 <barrytong.work@gmail.com> Co-authored-by: joeycli <joeycli0919@gmail.com> * read:del * read:block-syncer-update * read:block-syncer-db * fix: use greenfield-go-sdk and upgrade lib version (#135) * fix: use greenfield-go-sdk and upgrade lib version * ci: rename main to master * fix: update greenfield-common version --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: add chain client to sp (#131) * feat: impl the interaction between sp and chain interface * fix: return bucket id from auth object * feat: add check authorization * feat: add chain client to stonehub * feat: add chain client to uploader/downloader * feat: implement get approval workflow * feat: implement object seal and integrity hash sign workflow * chore: update dependencies * feat:implement secondary sp integrity hash sign workflow * test: add signer and chain to onebox * fix: fix itergrated test bug --------- Co-authored-by: joeylichang <joeycli0919@gmail.com> --------- Co-authored-by: joeycli <joeycli0919@gmail.com> Co-authored-by: dylanhuang <j75689@gmail.com> Co-authored-by: constwz <changbohao30@gmail.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: constwz <122766871+constwz@users.noreply.github.com> Co-authored-by: Annie <108039750+annielz@users.noreply.github.com> Co-authored-by: BarryTong65 <barrytong.work@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
this pr add batch/iterator interface and implement stone load and gc.
Rationale
load: stonehub load job context from db at setup, and replay to memory job context struct.
gc: stonehub scan job context from db periodically, if job has done or failed and gc it.
Example
N/A
Changes
Notable changes:
N/A