Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: refactor metadata config and change http address to grpc address #170

Merged
merged 6 commits into from
Mar 7, 2023
Merged

fix: refactor metadata config and change http address to grpc address #170

merged 6 commits into from
Mar 7, 2023

Conversation

BarryTong65
Copy link
Contributor

@BarryTong65 BarryTong65 commented Mar 6, 2023 •
edited
Loading

Description

refactor metadata according to the new commit and change metadata http address to gRPC address

Rationale

refactor metadata according to the new commit and change metadata http address to gRPC address

Example

add an example CLI or API response...

Changes

Notable changes:

  • fix the below issues and change metadata http address to gRPC address

image

@BarryTong65 BarryTong65 requested a review from will-2012 March 6, 2023 08:47
@BarryTong65 BarryTong65 force-pushed the dev_env/rebase_develop_20230306 branch from 4a3de65 to 9e43ed9 Compare March 6, 2023 08:51
@BarryTong65 BarryTong65 added the r4r Ready for review label Mar 6, 2023
joeylichang
joeylichang reviewed Mar 6, 2023
View reviewed changes
config/config.go Outdated
@@ -31,6 +32,7 @@ type StorageProviderConfig struct {
ChainConfig *gnfd.GreenfieldChainConfig
SignerCfg *signer.SignerConfig
BlockSyncerCfg *tomlconfig.TomlConfig
MetadataCfg *metadata.MetadataConfig
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MetadataCfg contains the MetadataGrpcAddress and SpDBConfig?
Can come form StorageProviderConfig.GRPCAddress[Metadataservice] and StorageProviderConfig.SpDBConfig
no need repeat stetement MetadataCfg

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, deleted all related code and import config from StorageProviderConfig.GRPCAddress[Metadataservice] and StorageProviderConfig.SpDBConfig
image

config/config.go Outdated
@@ -74,6 +77,7 @@ var DefaultStorageProviderConfig = &StorageProviderConfig{
ChainConfig: DefaultGreenfieldChainConfig,
SignerCfg: signer.DefaultSignerChainConfig,
BlockSyncerCfg: blocksyncer.DefaultBlockSyncerConfig,
MetadataCfg: DefaultMetadataConfig,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in the previous comment
the default value should put GRPCAddress[Metadataservice]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, deleted all related code and import config from StorageProviderConfig.GRPCAddress[Metadataservice] and StorageProviderConfig.SpDBConfig

config/config.go Outdated
@@ -107,6 +111,11 @@ var DefaultGreenfieldChainConfig = &gnfd.GreenfieldChainConfig{
}},
}

var DefaultMetadataConfig = &metadata.MetadataConfig{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in the previous comment, should delete

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

config/subconfig.go Outdated
if _, ok := cfg.GRPCAddress[model.MetadataService]; ok {
gCfg.MetadataServiceAddress = cfg.GRPCAddress[model.MetadataService]
} else {
return nil, fmt.Errorf("missing syncer gPRC address configuration for gateway service")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fmt.Errorf("missing syncer gPRC address configuration for gateway service")
change to
fmt.Errorf("missing meta dada gPRC address configuration for gateway service")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

config/subconfig.go Outdated
if _, ok := cfg.HTTPAddress[model.MetadataService]; ok {
mCfg.Address = cfg.HTTPAddress[model.MetadataService]
if _, ok := cfg.GRPCAddress[model.MetadataService]; ok {
mCfg.Address = cfg.GRPCAddress[model.MetadataService]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unified naming convention ： Address -> GRPCAddress

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

model/const.go Outdated
@@ -44,7 +44,7 @@ const (
// GatewayHTTPAddress default HTTP address of gateway
GatewayHTTPAddress = "localhost:9033"
// MetaDataServiceHTTPAddress default HTTP address of meta data service
MetaDataServiceHTTPAddress = "localhost:9733"
MetaDataServiceGRPCAddress = "localhost:9733"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MetaDataServiceGRPCAddress -> MetaDataGRPCAddress

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@will-2012
Copy link
Collaborator

will-2012 commented Mar 6, 2023 •
edited
Loading

PR title can be more simple and meaningful; Just explain what has been done; The details of how and why can be put in the description.

will-2012
will-2012 reviewed Mar 6, 2023
View reviewed changes
service/gateway/gateway.go
return nil, err

if cfg.MetadataServiceAddress != "" {
if g.metadata, err = client.NewMetadataClient(cfg.MetadataServiceAddress); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

metadata handler needs to check g.metadata != nil

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i just follow the way of the other services.
image
and it will only return nil when the err occurs(already check err in the above code)
image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If metadata is nil, will metadata related requests(the metadata handler) cause the gateway to crash?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, add if statement for error response and create errorJSONResponse for response(Clyde said we will provide JSON type in phase 1 api)

@BarryTong65 BarryTong65 changed the title fix: refactor metadata according to the new commit fix: refactor metadata config and change http address to grpc address Mar 6, 2023
joeylichang
joeylichang reviewed Mar 6, 2023
View reviewed changes
config/subconfig.go Outdated
@@ -153,7 +153,7 @@ func (cfg *StorageProviderConfig) MakeStoneNodeConfig() (*stonenode.StoneNodeCon
if _, ok := cfg.GRPCAddress[model.SignerService]; ok {
snCfg.SignerGrpcAddress = cfg.GRPCAddress[model.SignerService]
} else {
return nil, fmt.Errorf("missing signer gPRC address configuration for stone node service")
return nil, fmt.Errorf("missing metadata gPRC address configuration for stone node service")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

signer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

will-2012
will-2012 reviewed Mar 6, 2023
View reviewed changes
service/gateway/gateway.go Outdated

if cfg.MetadataServiceAddress != "" {
if g.metadata, err = client.NewMetadataClient(cfg.MetadataServiceAddress); err != nil {
log.Errorw("failed to create metadata client", "err", err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"err" --> "error"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@joeylichang
Copy link
Contributor

LGTM

will-2012
will-2012 reviewed Mar 6, 2023
View reviewed changes
config/subconfig.go Outdated
} else {
return nil, fmt.Errorf("missing meta data HTTP address configuration for mate data service")
return nil, fmt.Errorf("missing meta data gRPC address configuration for mete data service")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mete->meta

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

service/gateway/metadata_handler.go Outdated
statusCode = errDescription.statusCode
_ = errDescription.errorJSONResponse(w, reqContext)
}
if statusCode == http.StatusOK {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StatusCode can be omitted, and errDescription.statusCode can be used directly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

service/gateway/metadata_handler.go Outdated
statusCode = errDescription.statusCode
_ = errDescription.errorJSONResponse(w, reqContext)
}
if statusCode == http.StatusOK {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

will-2012
will-2012 approved these changes Mar 7, 2023
View reviewed changes
Copy link
Collaborator

@will-2012 will-2012 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@BarryTong65 BarryTong65 merged commit 65cde20 into develop Mar 7, 2023
@BarryTong65 BarryTong65 deleted the dev_env/rebase_develop_20230306 branch March 7, 2023 05:50
joeylichang pushed a commit that referenced this pull request Mar 13, 2023
@BarryTong65 @joeylichang
fix: refactor metadata (#170)
831bad0 
* fix: refactor metadata according to the new commit

* fix: recover default config and change http address to grpc

* fix: remove MetadataCfg and refactor log

* fix: check if metadata is nil and add error response

* fix: remove status code
annielz pushed a commit that referenced this pull request Mar 14, 2023
@BarryTong65 @will-2012
@joeylichang @sysvm @yzhaoyu
refactor: refactor metadata service (#188)
2cb9a9a 
* fix: bypass v2 auth for test (#152) (#153)

* feat: update grpc metadata getway (#151)

* feat: update grpc metadata gateway

* fix: buf-lint

* fix: remove store

* feat: load metadata service db config from ENV

* feat: storage provider refactor (#168)

chore: change makefile for install gogofaster

chore: remove the deprecate code

feat:add spdb interface

chore: add sp db config

feat: add syncer proto

chore: change the proto of syncer

feat: syncer refactor with chain data struct (#154)

feat: change stone node proto by chain

* feat: change sync replicate to async

* feat: change proto dir

feat: change uploader proto by chain

chore: tidy up code by dir

* chore: model dir clean

* chore: util dir clean

* chore: test dir clean

* chore: fix cr commemts

* chore:model and config comments and variable name tidying up

* chore: tidy up pky code

* chore: proto tidy up

* chore:change UnSupported to unsupport

* chore: tidy up service code

feat: changeto cli command to manage the app

chore: add dump config and list env var cmd

refactor: polish gateway module (#157)

* refactor: polish gateway module

* fix: fix comment tips

refactor: implement sp db, remove meta db and job db (#156)

* refactor: implement sp db, remove meta db and job dbwq

* refactor: implement sp db, remove meta db and job db

* feat: add sp tables

* fix: fix sp errors

* chore: add init sp db in service

* fix: fix sp db use func name

* fix: add detailed comments for each func, struct and interface

---------




chore: cmd tidy up (#159)

fix: uploader and stone node bug

chore: update db after success to replicate

feat: add command for devops and config log (#162)

* feat: add command for DevOps and config log

* fix: use until spit func to slit flag

* feat: add deployment doc

fix: doc format

fix: rename variable names for standard (#161)

* fix: rename variable names for standard

* fix: handle repetitive imported pkg dir

* fix: define some constants in model dir

---------



feat: sp db adds service config interface (#164)

* feat: sp db adds service config interface

* feat: add table name


---------



Main refactor (#166)

* chore: add spdb address in env var

* fix: stone node replicate idx

* refactor: the entry and config

* fix: ci lint error

fix: deployment doc

feat:add config template doc

chore: add config template link

fix: config template error

feat: improve sync piece efficiency (#165)

Co-authored-by: joeylichang <joeycli0919@gmail.com>

* fix: refactor metadata (#170)

* fix: refactor metadata according to the new commit

* fix: recover default config and change http address to grpc

* fix: remove MetadataCfg and refactor log

* fix: check if metadata is nil and add error response

* fix: remove status code

* feat: piece store adds minio storage type (#169)

* feat: piece store adds minio storage type

* test: add minio and sharding e2e test

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: implement manager module (#172)

* feat: implement manager module

* fix: polish by comment tips

* feat: add localup script (#171)

* feat: add localup script

* fix: polish by comment tips

* doc: add link to doc (#180)

* fix: add links to deployment and setup local docs

* ci: add branch naming rules (#179)

* ci: add branch naming rules

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* docs: add change log for release v0.0.3 (#182)

* doc: add changelog for release v0.0.3

* fix: PR links

* refactor: refactor metadata service

* fix: remove atomic.Bool

* fix: rebase develop branch

* fix: fix all member variables

* fix: fix named return values

* fix: update comments

* fix: update metadata proto

* fix: update comments

---------

Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: joeylichang <joeycli0919@gmail.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>
joeylichang added a commit that referenced this pull request Mar 18, 2023
@joeylichang @BarryTong65
@will-2012 @sysvm @yzhaoyu @krish-nr @constwz @j75689
feat: merge develop to master for release v0.0.4 (#220)
4732c89 
* chore: add notes for sp  (#193)

* chore: add notes for sql db

* fix: cmd dir notes

* chore: mode dir note tidy

* chore: pkg dir note tidy

* chore: service dir note tidy

* fix: cr comments

* fix: the store notes

* refactor: refactor metadata service (#188)

* fix: bypass v2 auth for test (#152) (#153)

* feat: update grpc metadata getway (#151)

* feat: update grpc metadata gateway

* fix: buf-lint

* fix: remove store

* feat: load metadata service db config from ENV

* feat: storage provider refactor (#168)

chore: change makefile for install gogofaster

chore: remove the deprecate code

feat:add spdb interface

chore: add sp db config

feat: add syncer proto

chore: change the proto of syncer

feat: syncer refactor with chain data struct (#154)

feat: change stone node proto by chain

* feat: change sync replicate to async

* feat: change proto dir

feat: change uploader proto by chain

chore: tidy up code by dir

* chore: model dir clean

* chore: util dir clean

* chore: test dir clean

* chore: fix cr commemts

* chore:model and config comments and variable name tidying up

* chore: tidy up pky code

* chore: proto tidy up

* chore:change UnSupported to unsupport

* chore: tidy up service code

feat: changeto cli command to manage the app

chore: add dump config and list env var cmd

refactor: polish gateway module (#157)

* refactor: polish gateway module

* fix: fix comment tips

refactor: implement sp db, remove meta db and job db (#156)

* refactor: implement sp db, remove meta db and job dbwq

* refactor: implement sp db, remove meta db and job db

* feat: add sp tables

* fix: fix sp errors

* chore: add init sp db in service

* fix: fix sp db use func name

* fix: add detailed comments for each func, struct and interface

---------




chore: cmd tidy up (#159)

fix: uploader and stone node bug

chore: update db after success to replicate

feat: add command for devops and config log (#162)

* feat: add command for DevOps and config log

* fix: use until spit func to slit flag

* feat: add deployment doc

fix: doc format

fix: rename variable names for standard (#161)

* fix: rename variable names for standard

* fix: handle repetitive imported pkg dir

* fix: define some constants in model dir

---------



feat: sp db adds service config interface (#164)

* feat: sp db adds service config interface

* feat: add table name


---------



Main refactor (#166)

* chore: add spdb address in env var

* fix: stone node replicate idx

* refactor: the entry and config

* fix: ci lint error

fix: deployment doc

feat:add config template doc

chore: add config template link

fix: config template error

feat: improve sync piece efficiency (#165)

Co-authored-by: joeylichang <joeycli0919@gmail.com>

* fix: refactor metadata (#170)

* fix: refactor metadata according to the new commit

* fix: recover default config and change http address to grpc

* fix: remove MetadataCfg and refactor log

* fix: check if metadata is nil and add error response

* fix: remove status code

* feat: piece store adds minio storage type (#169)

* feat: piece store adds minio storage type

* test: add minio and sharding e2e test

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: implement manager module (#172)

* feat: implement manager module

* fix: polish by comment tips

* feat: add localup script (#171)

* feat: add localup script

* fix: polish by comment tips

* doc: add link to doc (#180)

* fix: add links to deployment and setup local docs

* ci: add branch naming rules (#179)

* ci: add branch naming rules

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* docs: add change log for release v0.0.3 (#182)

* doc: add changelog for release v0.0.3

* fix: PR links

* refactor: refactor metadata service

* fix: remove atomic.Bool

* fix: rebase develop branch

* fix: fix all member variables

* fix: fix named return values

* fix: update comments

* fix: update metadata proto

* fix: update comments

---------

Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: joeylichang <joeycli0919@gmail.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>

* docs: add sp docs (#196)

docs: add workflow docs (#192)

* refactor: rename stonenode, syncer to tasknode, recevier (#197)

* refactor: rename stonenode,syncer to tasknode,recevier

* fix: update comment tips

* fix: update by comment tips

* fix: polish by comment tips

* docs: refining readme (#200)

* refining readme

* build: replace go1.19 with go1.18 (#198)

* build: replace go1.19 with go1.18

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: add endpoint for service

* feat: chang config template dir for renewed on time

* docs: add block syncer config (#208)

* docs: add block syncer config

* fix: add blocksyncer service to list

* fix: add blocksyncer to config template

* feat: update get bucket apis (#202)

* feat: update metadata apis

feat: update metadata apis

feat: update check_sums [][]bytes

feat: update metadata

fix: update ci workflows

fix: update ci workflows

fix: grpc zero value

feat: imple getUserBucketsCount

feat: update ci workflows

fix: listObjectsByBucketRouterName router path

feat: use s3 styles for apis

unblock env

* fix: update deleted object api

* fix: delete metadata workflows

* refactor: refactor all the comments and styles

* fix: fix bucket api

* fix: rename service and fix comments

* refactor: impl bsdb for metadata

* refactor: impl bsdb for metadata

* fix: fix comments

* fix: update image tag

* refactor: refactor db schema

* fix: remove metadata ci workflow

* fix: db response style (#209)

* fix: change to make build in readme

* fix: potential attack risks in on-chain storage module (#215)

* fix: add annotation in singer.proto

* fix: refine signer config

* fix: IntegrityHash signature format

* chore: upgrade github.com/bnb-chain/greenfield

* fix: comments from sysvm

* feat: support query quota and list read record (#206)

* feat: add query traffic interface

* feat: use chain verify permission api

* chore: polish by comment tips

* docs: add changelog for releasing v0.0.4

* fix: import pkg order

* fix: import pkg error

---------

Co-authored-by: BarryTong65 <122767193+BarryTong65@users.noreply.github.com>
Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: krish-nr <krish.z@nodereal.io>
Co-authored-by: krish-z <122767080+krish-nr@users.noreply.github.com>
Co-authored-by: constwz <122766871+constwz@users.noreply.github.com>
Co-authored-by: dylanhuang <j75689@gmail.com>
joeylichang added a commit that referenced this pull request Mar 20, 2023
@joeylichang @BarryTong65
@will-2012 @sysvm @yzhaoyu @krish-nr @constwz @j75689
build: change greenfield version (#224)
c38e5da 
* chore: add notes for sp  (#193)

* chore: add notes for sql db

* fix: cmd dir notes

* chore: mode dir note tidy

* chore: pkg dir note tidy

* chore: service dir note tidy

* fix: cr comments

* fix: the store notes

* refactor: refactor metadata service (#188)

* fix: bypass v2 auth for test (#152) (#153)

* feat: update grpc metadata getway (#151)

* feat: update grpc metadata gateway

* fix: buf-lint

* fix: remove store

* feat: load metadata service db config from ENV

* feat: storage provider refactor (#168)

chore: change makefile for install gogofaster

chore: remove the deprecate code

feat:add spdb interface

chore: add sp db config

feat: add syncer proto

chore: change the proto of syncer

feat: syncer refactor with chain data struct (#154)

feat: change stone node proto by chain

* feat: change sync replicate to async

* feat: change proto dir

feat: change uploader proto by chain

chore: tidy up code by dir

* chore: model dir clean

* chore: util dir clean

* chore: test dir clean

* chore: fix cr commemts

* chore:model and config comments and variable name tidying up

* chore: tidy up pky code

* chore: proto tidy up

* chore:change UnSupported to unsupport

* chore: tidy up service code

feat: changeto cli command to manage the app

chore: add dump config and list env var cmd

refactor: polish gateway module (#157)

* refactor: polish gateway module

* fix: fix comment tips

refactor: implement sp db, remove meta db and job db (#156)

* refactor: implement sp db, remove meta db and job dbwq

* refactor: implement sp db, remove meta db and job db

* feat: add sp tables

* fix: fix sp errors

* chore: add init sp db in service

* fix: fix sp db use func name

* fix: add detailed comments for each func, struct and interface

---------




chore: cmd tidy up (#159)

fix: uploader and stone node bug

chore: update db after success to replicate

feat: add command for devops and config log (#162)

* feat: add command for DevOps and config log

* fix: use until spit func to slit flag

* feat: add deployment doc

fix: doc format

fix: rename variable names for standard (#161)

* fix: rename variable names for standard

* fix: handle repetitive imported pkg dir

* fix: define some constants in model dir

---------



feat: sp db adds service config interface (#164)

* feat: sp db adds service config interface

* feat: add table name


---------



Main refactor (#166)

* chore: add spdb address in env var

* fix: stone node replicate idx

* refactor: the entry and config

* fix: ci lint error

fix: deployment doc

feat:add config template doc

chore: add config template link

fix: config template error

feat: improve sync piece efficiency (#165)

Co-authored-by: joeylichang <joeycli0919@gmail.com>

* fix: refactor metadata (#170)

* fix: refactor metadata according to the new commit

* fix: recover default config and change http address to grpc

* fix: remove MetadataCfg and refactor log

* fix: check if metadata is nil and add error response

* fix: remove status code

* feat: piece store adds minio storage type (#169)

* feat: piece store adds minio storage type

* test: add minio and sharding e2e test

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: implement manager module (#172)

* feat: implement manager module

* fix: polish by comment tips

* feat: add localup script (#171)

* feat: add localup script

* fix: polish by comment tips

* doc: add link to doc (#180)

* fix: add links to deployment and setup local docs

* ci: add branch naming rules (#179)

* ci: add branch naming rules

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* docs: add change log for release v0.0.3 (#182)

* doc: add changelog for release v0.0.3

* fix: PR links

* refactor: refactor metadata service

* fix: remove atomic.Bool

* fix: rebase develop branch

* fix: fix all member variables

* fix: fix named return values

* fix: update comments

* fix: update metadata proto

* fix: update comments

---------

Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: joeylichang <joeycli0919@gmail.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>

* docs: add sp docs (#196)

docs: add workflow docs (#192)

* refactor: rename stonenode, syncer to tasknode, recevier (#197)

* refactor: rename stonenode,syncer to tasknode,recevier

* fix: update comment tips

* fix: update by comment tips

* fix: polish by comment tips

* docs: refining readme (#200)

* refining readme

* build: replace go1.19 with go1.18 (#198)

* build: replace go1.19 with go1.18

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: add endpoint for service

* feat: chang config template dir for renewed on time

* docs: add block syncer config (#208)

* docs: add block syncer config

* fix: add blocksyncer service to list

* fix: add blocksyncer to config template

* feat: update get bucket apis (#202)

* feat: update metadata apis

feat: update metadata apis

feat: update check_sums [][]bytes

feat: update metadata

fix: update ci workflows

fix: update ci workflows

fix: grpc zero value

feat: imple getUserBucketsCount

feat: update ci workflows

fix: listObjectsByBucketRouterName router path

feat: use s3 styles for apis

unblock env

* fix: update deleted object api

* fix: delete metadata workflows

* refactor: refactor all the comments and styles

* fix: fix bucket api

* fix: rename service and fix comments

* refactor: impl bsdb for metadata

* refactor: impl bsdb for metadata

* fix: fix comments

* fix: update image tag

* refactor: refactor db schema

* fix: remove metadata ci workflow

* fix: db response style (#209)

* fix: change to make build in readme

* fix: potential attack risks in on-chain storage module (#215)

* fix: add annotation in singer.proto

* fix: refine signer config

* fix: IntegrityHash signature format

* chore: upgrade github.com/bnb-chain/greenfield

* fix: comments from sysvm

* feat: support query quota and list read record (#206)

* feat: add query traffic interface

* feat: use chain verify permission api

* chore: polish by comment tips

* docs: add changelog for releasing v0.0.4

* fix: import pkg order

* fix: import pkg error

* fix: increase the data size sent and received by gRPC (#222)

* build: change greenfield version to v0.0.9 (#223)

* build: chang greenfield version to v0.0.9

* fix: build dependency error

---------

Co-authored-by: BarryTong65 <122767193+BarryTong65@users.noreply.github.com>
Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: krish-nr <krish.z@nodereal.io>
Co-authored-by: krish-z <122767080+krish-nr@users.noreply.github.com>
Co-authored-by: constwz <122766871+constwz@users.noreply.github.com>
Co-authored-by: dylanhuang <j75689@gmail.com>
joeylichang added a commit that referenced this pull request Mar 20, 2023
@will-2012 @joeylichang
@BarryTong65 @sysvm @yzhaoyu @krish-nr @constwz @j75689
feat: merge develop to master (#226)
9709bc8 
* chore: add notes for sp  (#193)

* chore: add notes for sql db

* fix: cmd dir notes

* chore: mode dir note tidy

* chore: pkg dir note tidy

* chore: service dir note tidy

* fix: cr comments

* fix: the store notes

* refactor: refactor metadata service (#188)

* fix: bypass v2 auth for test (#152) (#153)

* feat: update grpc metadata getway (#151)

* feat: update grpc metadata gateway

* fix: buf-lint

* fix: remove store

* feat: load metadata service db config from ENV

* feat: storage provider refactor (#168)

chore: change makefile for install gogofaster

chore: remove the deprecate code

feat:add spdb interface

chore: add sp db config

feat: add syncer proto

chore: change the proto of syncer

feat: syncer refactor with chain data struct (#154)

feat: change stone node proto by chain

* feat: change sync replicate to async

* feat: change proto dir

feat: change uploader proto by chain

chore: tidy up code by dir

* chore: model dir clean

* chore: util dir clean

* chore: test dir clean

* chore: fix cr commemts

* chore:model and config comments and variable name tidying up

* chore: tidy up pky code

* chore: proto tidy up

* chore:change UnSupported to unsupport

* chore: tidy up service code

feat: changeto cli command to manage the app

chore: add dump config and list env var cmd

refactor: polish gateway module (#157)

* refactor: polish gateway module

* fix: fix comment tips

refactor: implement sp db, remove meta db and job db (#156)

* refactor: implement sp db, remove meta db and job dbwq

* refactor: implement sp db, remove meta db and job db

* feat: add sp tables

* fix: fix sp errors

* chore: add init sp db in service

* fix: fix sp db use func name

* fix: add detailed comments for each func, struct and interface

---------




chore: cmd tidy up (#159)

fix: uploader and stone node bug

chore: update db after success to replicate

feat: add command for devops and config log (#162)

* feat: add command for DevOps and config log

* fix: use until spit func to slit flag

* feat: add deployment doc

fix: doc format

fix: rename variable names for standard (#161)

* fix: rename variable names for standard

* fix: handle repetitive imported pkg dir

* fix: define some constants in model dir

---------



feat: sp db adds service config interface (#164)

* feat: sp db adds service config interface

* feat: add table name


---------



Main refactor (#166)

* chore: add spdb address in env var

* fix: stone node replicate idx

* refactor: the entry and config

* fix: ci lint error

fix: deployment doc

feat:add config template doc

chore: add config template link

fix: config template error

feat: improve sync piece efficiency (#165)

Co-authored-by: joeylichang <joeycli0919@gmail.com>

* fix: refactor metadata (#170)

* fix: refactor metadata according to the new commit

* fix: recover default config and change http address to grpc

* fix: remove MetadataCfg and refactor log

* fix: check if metadata is nil and add error response

* fix: remove status code

* feat: piece store adds minio storage type (#169)

* feat: piece store adds minio storage type

* test: add minio and sharding e2e test

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: implement manager module (#172)

* feat: implement manager module

* fix: polish by comment tips

* feat: add localup script (#171)

* feat: add localup script

* fix: polish by comment tips

* doc: add link to doc (#180)

* fix: add links to deployment and setup local docs

* ci: add branch naming rules (#179)

* ci: add branch naming rules

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* docs: add change log for release v0.0.3 (#182)

* doc: add changelog for release v0.0.3

* fix: PR links

* refactor: refactor metadata service

* fix: remove atomic.Bool

* fix: rebase develop branch

* fix: fix all member variables

* fix: fix named return values

* fix: update comments

* fix: update metadata proto

* fix: update comments

---------

Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com>
Co-authored-by: joeylichang <joeycli0919@gmail.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>

* docs: add sp docs (#196)

docs: add workflow docs (#192)

* refactor: rename stonenode, syncer to tasknode, recevier (#197)

* refactor: rename stonenode,syncer to tasknode,recevier

* fix: update comment tips

* fix: update by comment tips

* fix: polish by comment tips

* docs: refining readme (#200)

* refining readme

* build: replace go1.19 with go1.18 (#198)

* build: replace go1.19 with go1.18

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: add endpoint for service

* feat: chang config template dir for renewed on time

* docs: add block syncer config (#208)

* docs: add block syncer config

* fix: add blocksyncer service to list

* fix: add blocksyncer to config template

* feat: update get bucket apis (#202)

* feat: update metadata apis

feat: update metadata apis

feat: update check_sums [][]bytes

feat: update metadata

fix: update ci workflows

fix: update ci workflows

fix: grpc zero value

feat: imple getUserBucketsCount

feat: update ci workflows

fix: listObjectsByBucketRouterName router path

feat: use s3 styles for apis

unblock env

* fix: update deleted object api

* fix: delete metadata workflows

* refactor: refactor all the comments and styles

* fix: fix bucket api

* fix: rename service and fix comments

* refactor: impl bsdb for metadata

* refactor: impl bsdb for metadata

* fix: fix comments

* fix: update image tag

* refactor: refactor db schema

* fix: remove metadata ci workflow

* fix: db response style (#209)

* fix: change to make build in readme

* fix: potential attack risks in on-chain storage module (#215)

* fix: add annotation in singer.proto

* fix: refine signer config

* fix: IntegrityHash signature format

* chore: upgrade github.com/bnb-chain/greenfield

* fix: comments from sysvm

* feat: support query quota and list read record (#206)

* feat: add query traffic interface

* feat: use chain verify permission api

* chore: polish by comment tips

* docs: add changelog for releasing v0.0.4

* fix: import pkg order

* fix: import pkg error

* fix: increase the data size sent and received by gRPC (#222)

* build: change greenfield version to v0.0.9 (#223)

* build: chang greenfield version to v0.0.9

* fix: build dependency error

* fix: fix verify permission bug (#225)

---------

Co-authored-by: joeycli <joeycli0919@gmail.com>
Co-authored-by: BarryTong65 <122767193+BarryTong65@users.noreply.github.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: krish-nr <krish.z@nodereal.io>
Co-authored-by: krish-z <122767080+krish-nr@users.noreply.github.com>
Co-authored-by: constwz <122766871+constwz@users.noreply.github.com>
Co-authored-by: dylanhuang <j75689@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@will-2012 will-2012 will-2012 approved these changes

@joeylichang joeylichang joeylichang approved these changes

Assignees
No one assigned
Labels
r4r Ready for review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@BarryTong65 @will-2012 @joeylichang