Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement signer service #119

Merged
merged 20 commits into from
Feb 19, 2023
Merged

feat: implement signer service #119

merged 20 commits into from
Feb 19, 2023

Conversation

j75689
Copy link
Contributor

@j75689 j75689 commented Feb 14, 2023 •
edited
Loading

Description

proto
syntax = "proto3";
package service.types.v1;
import "pkg/types/v1/object.proto";
import "service/types/v1/uploader.proto";
import "greenfield/storage/tx.proto";

option go_package = "github.com/bnb-chain/greenfield-storage-provider/service/types/v1";

message SignBucketApprovalRequest {
  bnbchain.greenfield.storage.MsgCreateBucket create_bucket_msg = 1;
}

message SignBucketApprovalResponse {
  bytes signature = 1;
  ErrMessage err_message = 2;
}

message VerifyBucketApprovalRequest {
  bnbchain.greenfield.storage.MsgCreateBucket create_bucket_msg = 1;
}

message VerifyBucketApprovalResponse {
  bool result = 1;
}

message SignObjectApprovalRequest {
  bnbchain.greenfield.storage.MsgCreateObject create_object_msg = 1;
}

message SignObjectApprovalResponse {
  bytes signature = 1;
  ErrMessage err_message = 2;
}

message VerifyObjectApprovalRequest {
  bnbchain.greenfield.storage.MsgCreateObject create_object_msg = 1;
}

message VerifyObjectApprovalResponse {
  bool result = 1;
}

message SignIntegrityHashRequest {
  repeated bytes data = 1;
}

message SignIntegrityHashResponse {
  bytes integrity_hash = 1;
  bytes signature = 2;
  ErrMessage err_message = 3;
}

message SealObjectOnChainRequest {
  pkg.types.v1.ObjectInfo object_info = 1;
}

message SealObjectOnChainResponse {
  bytes tx_hash = 1;
  ErrMessage err_message = 2;
}

service SignerService {
  rpc SignBucketApproval(SignBucketApprovalRequest) returns (SignBucketApprovalResponse) {};
  rpc VerifyBucketApproval(VerifyBucketApprovalRequest) returns (VerifyBucketApprovalResponse) {};
  rpc SignObjectApproval(SignObjectApprovalRequest) returns (SignObjectApprovalResponse) {};
  rpc VerifyObjectApproval(VerifyObjectApprovalRequest) returns (VerifyObjectApprovalResponse) {};
  rpc SignIntegrityHash(SignIntegrityHashRequest) returns (SignIntegrityHashResponse) {};
  rpc SealObjectOnChain(SealObjectOnChainRequest) returns (SealObjectOnChainResponse) {};
}

Example

add an example CLI or API response...

Changes

Notable changes:

  • new singer service

sysvm
sysvm reviewed Feb 14, 2023
View reviewed changes
service/signer/gnfd_chain.go Outdated
if err != nil {
log.Panic(err)
}
cli := &GreenfieldChain{
Copy link
Collaborator

@sysvm sysvm Feb 14, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return straightly &GreenfieldChain, don't use claim a new var? What does cli mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

sysvm
sysvm reviewed Feb 14, 2023
View reviewed changes
service/signer/gnfd_chain.go Outdated
xauthsigning "github.com/cosmos/cosmos-sdk/x/auth/signing"
rpcclient "github.com/tendermint/tendermint/rpc/client"

ptypes "github.com/bnb-chain/greenfield-storage-provider/pkg/types/v1"
Copy link
Collaborator

@sysvm sysvm Feb 14, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use goimports to regularize imported packages

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

sysvm
sysvm reviewed Feb 14, 2023
View reviewed changes
service/signer/gnfd_chain.go Outdated

var (
secondarySPAccs = make([]types.AccAddress, 0, len(object.SecondarySps))
secondarySpSignatures = make([][]byte, 0, len(object.SecondarySps))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

secondarySpSignatures --> secondarySPSignatures

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

sysvm
sysvm reviewed Feb 14, 2023
View reviewed changes
service/signer/gnfd_client.go Outdated
}

func newRpcClient(addr string) *http.HTTP {
httpClient, err := libclient.DefaultHTTPClient(addr)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newRpcClient --> newRPCClient

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the client is replaced by greenfield-go-sdk

sysvm
sysvm reviewed Feb 14, 2023
View reviewed changes
service/signer/gnfd_config.go Outdated
type GreenfieldChainConfig struct {
RPCAddrs []string
GRPCAddrs []string
ChainId uint16
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ChainId --> ChainID

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

unclezoro
unclezoro reviewed Feb 14, 2023
View reviewed changes
config/config.toml Outdated Show resolved Hide resolved
unclezoro
unclezoro requested changes Feb 14, 2023
View reviewed changes
Copy link
Contributor

@unclezoro unclezoro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need refactor

@j75689 j75689 marked this pull request as ready for review February 16, 2023 09:52
@j75689 j75689 requested a review from unclezoro February 16, 2023 10:01
sysvm
sysvm reviewed Feb 16, 2023
View reviewed changes
service/signer/gnfd_client.go Outdated
@@ -0,0 +1,129 @@
package signer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe gnfd_client.go could be moved to service/client directory

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

joeylichang
joeylichang reviewed Feb 17, 2023
View reviewed changes
service/signer/gnfd_client.go Outdated
)

// GreenfieldChainClient the greenfield chain client
type GreenfieldChainClient struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'GreenfieldChainClient' change to 'GreenfieldChainSignClient' maybe better?
other modules will interact with greenfield-chain , but send queries requests, only the signer send tx.
so it's better to differentiate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@j75689 j75689 force-pushed the signer branch 2 times, most recently from d0ad086 to 4457039 Compare February 17, 2023 05:36
joeylichang
joeylichang reviewed Feb 17, 2023
View reviewed changes
proto/service/types/v1/signer.proto
bytes tx_hash = 1;
ErrMessage err_message = 2;
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add VerifyCreateBucketApproval and VerifyCreateObjectApproval, and verify the msg is signed by approval_address.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

joeylichang
joeylichang reviewed Feb 17, 2023
View reviewed changes
service/client/gnfd_client.go Outdated
@@ -0,0 +1,136 @@
package client
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gnfd_client.go change to gnfd_sign_client.go?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@unclezoro
Copy link
Contributor

LGTM

will-2012
will-2012 reviewed Feb 17, 2023
View reviewed changes
service/client/gnfd_sign_client.go Outdated
return false
}
sigHash := crypto.Keccak256(msg)
return VerifySignature(km.GetAddr(), sigHash, sig) == nil
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

print log when an error occurs?

@sysvm
Copy link
Collaborator

sysvm commented Feb 19, 2023

LGTM

@joeylichang joeylichang merged commit 193bee4 into develop Feb 19, 2023
@sysvm sysvm deleted the signer branch February 20, 2023 12:49
joeylichang added a commit that referenced this pull request Feb 26, 2023
@will-2012 @joeylichang
@j75689 @constwz @sysvm @yzhaoyu @annielz @BarryTong65
feat: merge develop into master (#148)
6fa0507 
* fix: docker push bug (#115)

* feat: add stone load and gc workflow (#117)

* feat: add iterator and batch interface to db

* chore: delete memory job db v1 code

* feat: impl the iterator and batch of job memory db

* feat: add stone load and gc db

* chore: delete job sql db v1 impl

* feat: job sql add iterator and batcher interface

* refactor: rename job_db v2 interface to job_db interface

---------

Co-authored-by: joeylichang <joeycli0919@gmail.com>

* feat: implement signer service (#119)

* feat: implement signer service

* feat: add api auth and ip whitelist for signer

* refactor: redesign the interface of singer service

* refactor: use greenfield-go-sdk

* fix: msg to be signed should not be pre-hashed

* feat: add VerifyBucketApproval, VerifyObjectApproval interface

* feat: add signer client

---------

Co-authored-by: joeylichang <joeycli0919@gmail.com>

* feat: client opt (#129)

* feat: implement gateway challenge workflow (#127)

* feat: implement gateway challenge workflow

* feat: support get range object

* read:block_syncer

* read:block_syncer

* read:block-syncer

* read:block-syncer

* fix: use env var to get bucket url (#130)

* fix: use env var to get bucket url

* fix: improve loading config from env function

* fix: rename variable names

* fix: change piece store readme.md

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* read:toml

* feat: stone node sends piece data to gateway (#128)

* feat: stone node sends piece data to gateway

* feat: alloc resp add bucket name and object name

* fix: fix errors of stone node uint test

* fix: fix gateway config

* fix: add gateway syncer rpc ut

* fix: add one log in gateway sync piece handler func

* fix: add detailed log in gateway sync piece handler

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* read:cmd_complete

* read:del

* read:test-job

* read:block-syncer-frame

* read:fix

* fix: upgrade greenfield version to fix the signing bug (#133)

* ci: fix release job

* dep: upgrade greenfield version

* fix: unit test for gnfd sign client

* fix: signer adapt to on-chain logic (#136)

* feat: add metadata service to sp (#132)

* feat: init commit

* feat: register metadata service and run on 9733 port

* feat: impl ListObjectsByBucketName api & model update

* feat: cleanup files and add metadata service sdk

* feat: add mock and unit tests

* feat: uniform error return values

* feat: metadata service case_driver update

---------

Co-authored-by: BarryTong65 <barrytong.work@gmail.com>
Co-authored-by: joeycli <joeycli0919@gmail.com>

* read:del

* read:block-syncer-update

* read:block-syncer-db

* fix: use greenfield-go-sdk and upgrade lib version (#135)

* fix: use greenfield-go-sdk and upgrade lib version

* ci: rename main to master

* fix: update greenfield-common version

---------

Co-authored-by: DylanYong <dylan.y@nodereal.io>

* feat: add chain client to sp (#131)

* feat: impl the interaction between sp and chain interface

* fix: return bucket id from auth object

* feat: add check authorization

* feat: add chain client to stonehub

* feat: add chain client to uploader/downloader

* feat: implement get approval workflow

* feat: implement object seal and integrity hash sign workflow

* chore: update dependencies

* feat:implement secondary sp integrity hash sign workflow

* test: add signer and chain to onebox

* fix: fix itergrated test bug

---------

Co-authored-by: joeylichang <joeycli0919@gmail.com>

---------

Co-authored-by: joeycli <joeycli0919@gmail.com>
Co-authored-by: dylanhuang <j75689@gmail.com>
Co-authored-by: constwz <changbohao30@gmail.com>
Co-authored-by: VM <112189277+sysvm@users.noreply.github.com>
Co-authored-by: DylanYong <dylan.y@nodereal.io>
Co-authored-by: constwz <122766871+constwz@users.noreply.github.com>
Co-authored-by: Annie <108039750+annielz@users.noreply.github.com>
Co-authored-by: BarryTong65 <barrytong.work@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@will-2012 will-2012 will-2012 left review comments

@sysvm sysvm sysvm left review comments

@joeylichang joeylichang joeylichang left review comments

@unclezoro unclezoro unclezoro approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@j75689 @unclezoro @sysvm @joeylichang @will-2012