Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add service lifecycle module #9

Merged
merged 13 commits into from
Dec 26, 2022
Merged

add service lifecycle module #9

merged 13 commits into from
Dec 26, 2022

Conversation

sysvm
Copy link
Collaborator

@sysvm sysvm commented Dec 21, 2022

No description provided.

@joeylichang
Copy link
Contributor

Consider adding unit tests? Especially the core parts such as start, stop, signal

@sysvm
Copy link
Collaborator Author

sysvm commented Dec 26, 2022

Consider adding unit tests? Especially the core parts such as start, stop, signal

Yes, I'm adding UT now.

will-2012
will-2012 approved these changes Dec 26, 2022
View reviewed changes
Copy link
Collaborator

@will-2012 will-2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

joeylichang
joeylichang approved these changes Dec 26, 2022
View reviewed changes
pkg/lifecycle/lifecycle.go Outdated
default:
}
if err := c.Init(ctx); err != nil {
log.Panicf("Init %s error: %v", c.Name(), err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Panic? or Does return error and log Warn better?

pkg/lifecycle/lifecycle.go Outdated
func (s *ServiceLifecycle) start(ctx context.Context, service Service) {
defer s.innerCancel()
if err := service.Start(ctx); err != nil {
log.Panicf("Service %s starts error: %v", service.Name(), err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does return error and log Warn better?

pkg/lifecycle/lifecycle.go Outdated
go func(ctx context.Context, service Service) {
defer wg.Done()
if err := service.Stop(ctx); err != nil {
log.Panicf("Service %s stops failure: %v", service.Name(), err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let all the service run stop? the back service may have important things to do,e.g. store the stateful data to db.

pkg/lifecycle/component.go Outdated
@@ -0,0 +1,28 @@
package lifecycle

import "context"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what the different between service and component?

@joeylichang joeylichang merged commit 4bcf160 into develop Dec 26, 2022
@sysvm sysvm deleted the lifecycle branch December 26, 2022 07:08
joeylichang added a commit that referenced this pull request Jan 14, 2023
@joeylichang @sysvm
@yzhaoyu @flywukong @will-2012 @xibao-nr
feat: v0.1-alpha version, support uploader, downloader, challenge the…
f987c0b 
… user data (#55)

* fix mock file (#3) (#4)

Co-authored-by: DylanYong <dylan.y@nodereal.io>

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* add service proto file (#8)

* add rpc proto (#12)

* add service lifecycle module (#9)

* fix mock file (#3)

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* add service lifecycle module


Co-authored-by: DylanYong <dylan.y@nodereal.io>

* delete proto file (#13)

* Proto (#14)

* delete proto file

* adjust proto interface

* adjust the service name

* feat: implement of upload payload in stone hub side (#17)

* feat: implement of upload payload in stone hub

* support redundancy functions  (#15)

* support redundancy funcions

* fix import order

* support raw segment encoding api

Co-authored-by: joeycli <1297465889@qq.com>

* feat: add upload stone ut, add inscription chain mock, add memory jobdb (#19)

* [model] add job meta orm (#18)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: add test tool for rpc test (#20)

* feat: add test tool for rpc test

* feat: nrmalize to log format (#21)

* feat: add test tool for rpc test

* Add secondary syncer service (#16)

* add syncer service


Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat(gateway/uploader): add gateway and uploader skeleton (#7)

* feat(service): add gateway skeleton

* test(gateway): add gateway e2e case

* feat(service): add uploader skeleton and polish code

* chore(gateway/uploader): polish main entry func and conf

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* fix: secondary piece job count (#22)

* refactor(jobdb_orm): adjust jobdb_orm directory (#23)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* fix stone node service (#24)

* fix stone node service

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* chore: fix Makefile/uploader & polish code style (#25)

* chore: fix Makefile clean  & polish code style

* fix(uploader): fix gorountine catch error arg

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: regularized the stone node code and improved the coroutine model (#27)

* feat: canonical form entry function

* feat: regularized the stone node code and improved the coroutine model

* fix: sync to seconadry err, not send to stone hub

* fix: ston node goroutine model (#28)

* feat: canonical form entry function

* feat: regularized the stone node code and improved the coroutine model

* fix: sync to seconadry err, not send to stone hub

* fix: ston node goroutine model

* Stone node opt (#29)

* feat: adjust stonde file

* feat: adjust stonre node file (#30)

* Update README.md

* Update README.md

* feat: norm syncer code (#31)

* feat: adjust uploader file structure (#32)

* fix: move test_tool to test dir (#33)

* feat: adjust entry func code

* fix: adjust stone hub entry func

* fix: object info and job context safe copy

* perf: optimize piece store logic for test (#26)

* fix syncer and stone node service config


Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: stone support mysql db (#34)

* fix: piece key type error

* feat(service): polish service code which be found during test (#35)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat(jobdb): add setHeightAndObjectID interface and encode create_hash to hex str (#36)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: implement meta db (#37)

* faet: add challenge proto, fix buf lint error

* feat: implement the challenge service (#38)

* feat: add stone hub BeginUploadPayloadV2 and downloader proto

* feat: add upload payload asking info to metadb

* feat: immplement BeginUploadPayloadV2 in stone hub

* add uint test for stone node and syncer module (#39)

* add uint test for stone node and syncer module


Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: joeycli <1297465889@qq.com>

* fix: complie error by rpc client define

* fix: rpc client mock complie error

* feat(gateway): add getAuthentication interface (#40)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: implement downder service and client (#41)

* fix: fsm event cover

* feat: add integrity hash write logic

* fix: ignore repeat piece job

* fix: add repeat piece job log

* fix: add repeat piece job error

* fix(uploader): fix multi rountine update same data unsafely (#45)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat(uploader): add getAuth interface (#43)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat(gateway/uploader): add redundancy type and polish code (#46)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* Integration test (#44)

* stone node and syncer integration test

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat(gateway/downloader): connect gateway with downloader (#47)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* fix log and unit test (#48)

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* style(gateway): polish gateway code style (#49)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* style(gateway): polish code style (#50)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* [WIP]feat:stone hub code opt (#54)

* stone hub code opt

* feat: change project name from inscription-storage-provider to greenfield-storage-provider

* Update README.md

Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: flywukong <2229306838@qq.com>
Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: will-2012-nr <will.nodereal@gmail.com>
joeylichang added a commit that referenced this pull request Jan 15, 2023
@joeylichang @sysvm
@yzhaoyu @flywukong @will-2012 @xibao-nr
feat: adjust start log format and add default config for supporting o…
9cd2309 
…ne-shut deploy (#57)

* fix mock file (#3) (#4)

Co-authored-by: DylanYong <dylan.y@nodereal.io>

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* add service proto file (#8)

* add rpc proto (#12)

* add service lifecycle module (#9)

* fix mock file (#3)

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* add service lifecycle module


Co-authored-by: DylanYong <dylan.y@nodereal.io>

* delete proto file (#13)

* Proto (#14)

* delete proto file

* adjust proto interface

* adjust the service name

* feat: implement of upload payload in stone hub side (#17)

* feat: implement of upload payload in stone hub

* support redundancy functions  (#15)

* support redundancy funcions

* fix import order

* support raw segment encoding api

Co-authored-by: joeycli <1297465889@qq.com>

* feat: add upload stone ut, add inscription chain mock, add memory jobdb (#19)

* [model] add job meta orm (#18)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: add test tool for rpc test (#20)

* feat: add test tool for rpc test

* feat: nrmalize to log format (#21)

* feat: add test tool for rpc test

* Add secondary syncer service (#16)

* add syncer service


Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat(gateway/uploader): add gateway and uploader skeleton (#7)

* feat(service): add gateway skeleton

* test(gateway): add gateway e2e case

* feat(service): add uploader skeleton and polish code

* chore(gateway/uploader): polish main entry func and conf

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* fix: secondary piece job count (#22)

* refactor(jobdb_orm): adjust jobdb_orm directory (#23)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* fix stone node service (#24)

* fix stone node service

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* chore: fix Makefile/uploader & polish code style (#25)

* chore: fix Makefile clean  & polish code style

* fix(uploader): fix gorountine catch error arg

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: regularized the stone node code and improved the coroutine model (#27)

* feat: canonical form entry function

* feat: regularized the stone node code and improved the coroutine model

* fix: sync to seconadry err, not send to stone hub

* fix: ston node goroutine model (#28)

* feat: canonical form entry function

* feat: regularized the stone node code and improved the coroutine model

* fix: sync to seconadry err, not send to stone hub

* fix: ston node goroutine model

* Stone node opt (#29)

* feat: adjust stonde file

* feat: adjust stonre node file (#30)

* Update README.md

* Update README.md

* feat: norm syncer code (#31)

* feat: adjust uploader file structure (#32)

* fix: move test_tool to test dir (#33)

* feat: adjust entry func code

* fix: adjust stone hub entry func

* fix: object info and job context safe copy

* perf: optimize piece store logic for test (#26)

* fix syncer and stone node service config


Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: stone support mysql db (#34)

* fix: piece key type error

* feat(service): polish service code which be found during test (#35)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat(jobdb): add setHeightAndObjectID interface and encode create_hash to hex str (#36)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: implement meta db (#37)

* faet: add challenge proto, fix buf lint error

* feat: implement the challenge service (#38)

* feat: add stone hub BeginUploadPayloadV2 and downloader proto

* feat: add upload payload asking info to metadb

* feat: immplement BeginUploadPayloadV2 in stone hub

* add uint test for stone node and syncer module (#39)

* add uint test for stone node and syncer module


Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: joeycli <1297465889@qq.com>

* fix: complie error by rpc client define

* fix: rpc client mock complie error

* feat(gateway): add getAuthentication interface (#40)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat: implement downder service and client (#41)

* fix: fsm event cover

* feat: add integrity hash write logic

* fix: ignore repeat piece job

* fix: add repeat piece job log

* fix: add repeat piece job error

* fix(uploader): fix multi rountine update same data unsafely (#45)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat(uploader): add getAuth interface (#43)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* feat(gateway/uploader): add redundancy type and polish code (#46)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* Integration test (#44)

* stone node and syncer integration test

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat(gateway/downloader): connect gateway with downloader (#47)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* fix log and unit test (#48)

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* style(gateway): polish gateway code style (#49)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* style(gateway): polish code style (#50)

Co-authored-by: will-2012-nr <will.nodereal@gmail.com>

* [WIP]feat:stone hub code opt (#54)

* stone hub code opt

* feat: change project name from inscription-storage-provider to greenfield-storage-provider

* Update README.md

* Update README.md

* feat: adjust the import pkg order

* fix: metadb init repeat under test mode

* adjust: start log level and format

* fix: adjust stonehub close channel order to resolve stop process panic

* fix: adjust service name format

* adjust: log format for line type uploader and downloader

* fix: snyc and stone node log opt

* fix: log opt

* fix: stone hub log opt

* fix: ec type checkout check

* fix: replicate piece count compute

* feat:implemet replicate type sync in stone node service

* fix: pick up secondary check

* fix: pick up secondary check

* fix: replicate type check

* fix: reset stone node

* feat: add default config support one-shut deploy

Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: flywukong <2229306838@qq.com>
Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: will-2012-nr <will.nodereal@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@will-2012 will-2012 will-2012 approved these changes

@joeylichang joeylichang joeylichang approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sysvm @joeylichang @will-2012 @yzhaoyu