feat: improve sync piece efficiency #165
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
will-2012
force-pushed
the
improve_sync
branch
9 times, most recently
from
b0738b1 to
2f07592
Compare
will-2012
requested review from
joeylichang and
sysvm
and removed request for
joeylichang
joeylichang
added a commit
that referenced
this pull request
Mar 4, 2023
Co-authored-by: joeycli <joeycli0919@gmail.com>
joeylichang
added a commit
that referenced
this pull request
Mar 4, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
will-2012
pushed a commit
that referenced
this pull request
Mar 5, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
will-2012
pushed a commit
that referenced
this pull request
Mar 5, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
will-2012
pushed a commit
that referenced
this pull request
Mar 5, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
will-2012
pushed a commit
that referenced
this pull request
Mar 5, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
will-2012
pushed a commit
that referenced
this pull request
Mar 6, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
will-2012
pushed a commit
that referenced
this pull request
Mar 6, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
will-2012
pushed a commit
that referenced
this pull request
Mar 6, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer opt with chain data struct (#154) * feat: syncer opt with chain data struct * feat: change stone node proto by chain * feat: change sync replicate to async * feat:change uploader protoby chain * feat: change proto dir * chore: model dir clean * chorr: util dir clean * chore: test dir clean * chore: fix cr commemts feat: code tidy up (#155) * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code chore: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * fix: fix code to build * fix: fix sql db config error * fix: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix set object info func * fix: fix sp db name * fix: fix sp db error * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: joeylichang <joeycli0919@gmail.com> chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkgs * fix: define some constants in model --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * fix: add table name * fix: fix get db error --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeycli <joeycli0919@gmail.com>
joeylichang
added a commit
that referenced
this pull request
Mar 6, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer refactor with chain data struct (#154) feat: change stone node proto by chain * feat: change sync replicate to async * feat: change proto dir feat: change uploader proto by chain chore: tidy up code by dir * chore: model dir clean * chore: util dir clean * chore: test dir clean * chore: fix cr commemts * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code feat: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * feat: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkg dir * fix: define some constants in model dir --------- feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * feat: add table name --------- Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeylichang <joeycli0919@gmail.com>
joeylichang
added a commit
that referenced
this pull request
Mar 13, 2023
chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer refactor with chain data struct (#154) feat: change stone node proto by chain * feat: change sync replicate to async * feat: change proto dir feat: change uploader proto by chain chore: tidy up code by dir * chore: model dir clean * chore: util dir clean * chore: test dir clean * chore: fix cr commemts * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code feat: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * feat: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkg dir * fix: define some constants in model dir --------- feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * feat: add table name --------- Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeylichang <joeycli0919@gmail.com>
annielz
pushed a commit
that referenced
this pull request
Mar 14, 2023
* fix: bypass v2 auth for test (#152) (#153) * feat: update grpc metadata getway (#151) * feat: update grpc metadata gateway * fix: buf-lint * fix: remove store * feat: load metadata service db config from ENV * feat: storage provider refactor (#168) chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer refactor with chain data struct (#154) feat: change stone node proto by chain * feat: change sync replicate to async * feat: change proto dir feat: change uploader proto by chain chore: tidy up code by dir * chore: model dir clean * chore: util dir clean * chore: test dir clean * chore: fix cr commemts * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code feat: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * feat: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkg dir * fix: define some constants in model dir --------- feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * feat: add table name --------- Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeylichang <joeycli0919@gmail.com> * fix: refactor metadata (#170) * fix: refactor metadata according to the new commit * fix: recover default config and change http address to grpc * fix: remove MetadataCfg and refactor log * fix: check if metadata is nil and add error response * fix: remove status code * feat: piece store adds minio storage type (#169) * feat: piece store adds minio storage type * test: add minio and sharding e2e test --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: implement manager module (#172) * feat: implement manager module * fix: polish by comment tips * feat: add localup script (#171) * feat: add localup script * fix: polish by comment tips * doc: add link to doc (#180) * fix: add links to deployment and setup local docs * ci: add branch naming rules (#179) * ci: add branch naming rules --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * docs: add change log for release v0.0.3 (#182) * doc: add changelog for release v0.0.3 * fix: PR links * refactor: refactor metadata service * fix: remove atomic.Bool * fix: rebase develop branch * fix: fix all member variables * fix: fix named return values * fix: update comments * fix: update metadata proto * fix: update comments --------- Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com> Co-authored-by: joeylichang <joeycli0919@gmail.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io>
joeylichang
added a commit
that referenced
this pull request
Mar 18, 2023
* chore: add notes for sp (#193) * chore: add notes for sql db * fix: cmd dir notes * chore: mode dir note tidy * chore: pkg dir note tidy * chore: service dir note tidy * fix: cr comments * fix: the store notes * refactor: refactor metadata service (#188) * fix: bypass v2 auth for test (#152) (#153) * feat: update grpc metadata getway (#151) * feat: update grpc metadata gateway * fix: buf-lint * fix: remove store * feat: load metadata service db config from ENV * feat: storage provider refactor (#168) chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer refactor with chain data struct (#154) feat: change stone node proto by chain * feat: change sync replicate to async * feat: change proto dir feat: change uploader proto by chain chore: tidy up code by dir * chore: model dir clean * chore: util dir clean * chore: test dir clean * chore: fix cr commemts * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code feat: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * feat: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkg dir * fix: define some constants in model dir --------- feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * feat: add table name --------- Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeylichang <joeycli0919@gmail.com> * fix: refactor metadata (#170) * fix: refactor metadata according to the new commit * fix: recover default config and change http address to grpc * fix: remove MetadataCfg and refactor log * fix: check if metadata is nil and add error response * fix: remove status code * feat: piece store adds minio storage type (#169) * feat: piece store adds minio storage type * test: add minio and sharding e2e test --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: implement manager module (#172) * feat: implement manager module * fix: polish by comment tips * feat: add localup script (#171) * feat: add localup script * fix: polish by comment tips * doc: add link to doc (#180) * fix: add links to deployment and setup local docs * ci: add branch naming rules (#179) * ci: add branch naming rules --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * docs: add change log for release v0.0.3 (#182) * doc: add changelog for release v0.0.3 * fix: PR links * refactor: refactor metadata service * fix: remove atomic.Bool * fix: rebase develop branch * fix: fix all member variables * fix: fix named return values * fix: update comments * fix: update metadata proto * fix: update comments --------- Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com> Co-authored-by: joeylichang <joeycli0919@gmail.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io> * docs: add sp docs (#196) docs: add workflow docs (#192) * refactor: rename stonenode, syncer to tasknode, recevier (#197) * refactor: rename stonenode,syncer to tasknode,recevier * fix: update comment tips * fix: update by comment tips * fix: polish by comment tips * docs: refining readme (#200) * refining readme * build: replace go1.19 with go1.18 (#198) * build: replace go1.19 with go1.18 --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: add endpoint for service * feat: chang config template dir for renewed on time * docs: add block syncer config (#208) * docs: add block syncer config * fix: add blocksyncer service to list * fix: add blocksyncer to config template * feat: update get bucket apis (#202) * feat: update metadata apis feat: update metadata apis feat: update check_sums [][]bytes feat: update metadata fix: update ci workflows fix: update ci workflows fix: grpc zero value feat: imple getUserBucketsCount feat: update ci workflows fix: listObjectsByBucketRouterName router path feat: use s3 styles for apis unblock env * fix: update deleted object api * fix: delete metadata workflows * refactor: refactor all the comments and styles * fix: fix bucket api * fix: rename service and fix comments * refactor: impl bsdb for metadata * refactor: impl bsdb for metadata * fix: fix comments * fix: update image tag * refactor: refactor db schema * fix: remove metadata ci workflow * fix: db response style (#209) * fix: change to make build in readme * fix: potential attack risks in on-chain storage module (#215) * fix: add annotation in singer.proto * fix: refine signer config * fix: IntegrityHash signature format * chore: upgrade github.com/bnb-chain/greenfield * fix: comments from sysvm * feat: support query quota and list read record (#206) * feat: add query traffic interface * feat: use chain verify permission api * chore: polish by comment tips * docs: add changelog for releasing v0.0.4 * fix: import pkg order * fix: import pkg error --------- Co-authored-by: BarryTong65 <122767193+BarryTong65@users.noreply.github.com> Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: krish-nr <krish.z@nodereal.io> Co-authored-by: krish-z <122767080+krish-nr@users.noreply.github.com> Co-authored-by: constwz <122766871+constwz@users.noreply.github.com> Co-authored-by: dylanhuang <j75689@gmail.com>
joeylichang
added a commit
that referenced
this pull request
Mar 20, 2023
* chore: add notes for sp (#193) * chore: add notes for sql db * fix: cmd dir notes * chore: mode dir note tidy * chore: pkg dir note tidy * chore: service dir note tidy * fix: cr comments * fix: the store notes * refactor: refactor metadata service (#188) * fix: bypass v2 auth for test (#152) (#153) * feat: update grpc metadata getway (#151) * feat: update grpc metadata gateway * fix: buf-lint * fix: remove store * feat: load metadata service db config from ENV * feat: storage provider refactor (#168) chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer refactor with chain data struct (#154) feat: change stone node proto by chain * feat: change sync replicate to async * feat: change proto dir feat: change uploader proto by chain chore: tidy up code by dir * chore: model dir clean * chore: util dir clean * chore: test dir clean * chore: fix cr commemts * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code feat: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * feat: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkg dir * fix: define some constants in model dir --------- feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * feat: add table name --------- Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeylichang <joeycli0919@gmail.com> * fix: refactor metadata (#170) * fix: refactor metadata according to the new commit * fix: recover default config and change http address to grpc * fix: remove MetadataCfg and refactor log * fix: check if metadata is nil and add error response * fix: remove status code * feat: piece store adds minio storage type (#169) * feat: piece store adds minio storage type * test: add minio and sharding e2e test --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: implement manager module (#172) * feat: implement manager module * fix: polish by comment tips * feat: add localup script (#171) * feat: add localup script * fix: polish by comment tips * doc: add link to doc (#180) * fix: add links to deployment and setup local docs * ci: add branch naming rules (#179) * ci: add branch naming rules --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * docs: add change log for release v0.0.3 (#182) * doc: add changelog for release v0.0.3 * fix: PR links * refactor: refactor metadata service * fix: remove atomic.Bool * fix: rebase develop branch * fix: fix all member variables * fix: fix named return values * fix: update comments * fix: update metadata proto * fix: update comments --------- Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com> Co-authored-by: joeylichang <joeycli0919@gmail.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io> * docs: add sp docs (#196) docs: add workflow docs (#192) * refactor: rename stonenode, syncer to tasknode, recevier (#197) * refactor: rename stonenode,syncer to tasknode,recevier * fix: update comment tips * fix: update by comment tips * fix: polish by comment tips * docs: refining readme (#200) * refining readme * build: replace go1.19 with go1.18 (#198) * build: replace go1.19 with go1.18 --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: add endpoint for service * feat: chang config template dir for renewed on time * docs: add block syncer config (#208) * docs: add block syncer config * fix: add blocksyncer service to list * fix: add blocksyncer to config template * feat: update get bucket apis (#202) * feat: update metadata apis feat: update metadata apis feat: update check_sums [][]bytes feat: update metadata fix: update ci workflows fix: update ci workflows fix: grpc zero value feat: imple getUserBucketsCount feat: update ci workflows fix: listObjectsByBucketRouterName router path feat: use s3 styles for apis unblock env * fix: update deleted object api * fix: delete metadata workflows * refactor: refactor all the comments and styles * fix: fix bucket api * fix: rename service and fix comments * refactor: impl bsdb for metadata * refactor: impl bsdb for metadata * fix: fix comments * fix: update image tag * refactor: refactor db schema * fix: remove metadata ci workflow * fix: db response style (#209) * fix: change to make build in readme * fix: potential attack risks in on-chain storage module (#215) * fix: add annotation in singer.proto * fix: refine signer config * fix: IntegrityHash signature format * chore: upgrade github.com/bnb-chain/greenfield * fix: comments from sysvm * feat: support query quota and list read record (#206) * feat: add query traffic interface * feat: use chain verify permission api * chore: polish by comment tips * docs: add changelog for releasing v0.0.4 * fix: import pkg order * fix: import pkg error * fix: increase the data size sent and received by gRPC (#222) * build: change greenfield version to v0.0.9 (#223) * build: chang greenfield version to v0.0.9 * fix: build dependency error --------- Co-authored-by: BarryTong65 <122767193+BarryTong65@users.noreply.github.com> Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: krish-nr <krish.z@nodereal.io> Co-authored-by: krish-z <122767080+krish-nr@users.noreply.github.com> Co-authored-by: constwz <122766871+constwz@users.noreply.github.com> Co-authored-by: dylanhuang <j75689@gmail.com>
joeylichang
added a commit
that referenced
this pull request
Mar 20, 2023
* chore: add notes for sp (#193) * chore: add notes for sql db * fix: cmd dir notes * chore: mode dir note tidy * chore: pkg dir note tidy * chore: service dir note tidy * fix: cr comments * fix: the store notes * refactor: refactor metadata service (#188) * fix: bypass v2 auth for test (#152) (#153) * feat: update grpc metadata getway (#151) * feat: update grpc metadata gateway * fix: buf-lint * fix: remove store * feat: load metadata service db config from ENV * feat: storage provider refactor (#168) chore: change makefile for install gogofaster chore: remove the deprecate code feat:add spdb interface chore: add sp db config feat: add syncer proto chore: change the proto of syncer feat: syncer refactor with chain data struct (#154) feat: change stone node proto by chain * feat: change sync replicate to async * feat: change proto dir feat: change uploader proto by chain chore: tidy up code by dir * chore: model dir clean * chore: util dir clean * chore: test dir clean * chore: fix cr commemts * chore:model and config comments and variable name tidying up * chore: tidy up pky code * chore: proto tidy up * chore:change UnSupported to unsupport * chore: tidy up service code feat: changeto cli command to manage the app chore: add dump config and list env var cmd refactor: polish gateway module (#157) * refactor: polish gateway module * fix: fix comment tips refactor: implement sp db, remove meta db and job db (#156) * refactor: implement sp db, remove meta db and job dbwq * refactor: implement sp db, remove meta db and job db * feat: add sp tables * fix: fix sp errors * chore: add init sp db in service * fix: fix sp db use func name * fix: add detailed comments for each func, struct and interface --------- chore: cmd tidy up (#159) fix: uploader and stone node bug chore: update db after success to replicate feat: add command for devops and config log (#162) * feat: add command for DevOps and config log * fix: use until spit func to slit flag * feat: add deployment doc fix: doc format fix: rename variable names for standard (#161) * fix: rename variable names for standard * fix: handle repetitive imported pkg dir * fix: define some constants in model dir --------- feat: sp db adds service config interface (#164) * feat: sp db adds service config interface * feat: add table name --------- Main refactor (#166) * chore: add spdb address in env var * fix: stone node replicate idx * refactor: the entry and config * fix: ci lint error fix: deployment doc feat:add config template doc chore: add config template link fix: config template error feat: improve sync piece efficiency (#165) Co-authored-by: joeylichang <joeycli0919@gmail.com> * fix: refactor metadata (#170) * fix: refactor metadata according to the new commit * fix: recover default config and change http address to grpc * fix: remove MetadataCfg and refactor log * fix: check if metadata is nil and add error response * fix: remove status code * feat: piece store adds minio storage type (#169) * feat: piece store adds minio storage type * test: add minio and sharding e2e test --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: implement manager module (#172) * feat: implement manager module * fix: polish by comment tips * feat: add localup script (#171) * feat: add localup script * fix: polish by comment tips * doc: add link to doc (#180) * fix: add links to deployment and setup local docs * ci: add branch naming rules (#179) * ci: add branch naming rules --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * docs: add change log for release v0.0.3 (#182) * doc: add changelog for release v0.0.3 * fix: PR links * refactor: refactor metadata service * fix: remove atomic.Bool * fix: rebase develop branch * fix: fix all member variables * fix: fix named return values * fix: update comments * fix: update metadata proto * fix: update comments --------- Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com> Co-authored-by: joeylichang <joeycli0919@gmail.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io> * docs: add sp docs (#196) docs: add workflow docs (#192) * refactor: rename stonenode, syncer to tasknode, recevier (#197) * refactor: rename stonenode,syncer to tasknode,recevier * fix: update comment tips * fix: update by comment tips * fix: polish by comment tips * docs: refining readme (#200) * refining readme * build: replace go1.19 with go1.18 (#198) * build: replace go1.19 with go1.18 --------- Co-authored-by: DylanYong <dylan.y@nodereal.io> * feat: add endpoint for service * feat: chang config template dir for renewed on time * docs: add block syncer config (#208) * docs: add block syncer config * fix: add blocksyncer service to list * fix: add blocksyncer to config template * feat: update get bucket apis (#202) * feat: update metadata apis feat: update metadata apis feat: update check_sums [][]bytes feat: update metadata fix: update ci workflows fix: update ci workflows fix: grpc zero value feat: imple getUserBucketsCount feat: update ci workflows fix: listObjectsByBucketRouterName router path feat: use s3 styles for apis unblock env * fix: update deleted object api * fix: delete metadata workflows * refactor: refactor all the comments and styles * fix: fix bucket api * fix: rename service and fix comments * refactor: impl bsdb for metadata * refactor: impl bsdb for metadata * fix: fix comments * fix: update image tag * refactor: refactor db schema * fix: remove metadata ci workflow * fix: db response style (#209) * fix: change to make build in readme * fix: potential attack risks in on-chain storage module (#215) * fix: add annotation in singer.proto * fix: refine signer config * fix: IntegrityHash signature format * chore: upgrade github.com/bnb-chain/greenfield * fix: comments from sysvm * feat: support query quota and list read record (#206) * feat: add query traffic interface * feat: use chain verify permission api * chore: polish by comment tips * docs: add changelog for releasing v0.0.4 * fix: import pkg order * fix: import pkg error * fix: increase the data size sent and received by gRPC (#222) * build: change greenfield version to v0.0.9 (#223) * build: chang greenfield version to v0.0.9 * fix: build dependency error * fix: fix verify permission bug (#225) --------- Co-authored-by: joeycli <joeycli0919@gmail.com> Co-authored-by: BarryTong65 <122767193+BarryTong65@users.noreply.github.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: DylanYong <dylan.y@nodereal.io> Co-authored-by: krish-nr <krish.z@nodereal.io> Co-authored-by: krish-z <122767080+krish-nr@users.noreply.github.com> Co-authored-by: constwz <122766871+constwz@users.noreply.github.com> Co-authored-by: dylanhuang <j75689@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
improve sync piece efficiency
Rationale
N/A
Example
N/A
Changes
Notable changes:
N/A