feat: add get object meta and get bucket meta apis #412
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xibao-nr
reviewed
May 10, 2023
service/gateway/metadata_handler.go
Outdated
| @@ -124,3 +124,124 @@ func (gateway *Gateway) listObjectsByBucketNameHandler(w http.ResponseWriter, r | |||
| w.Header().Set(model.ContentTypeHeader, model.ContentTypeJSONHeaderValue) | |||
| w.Write(b.Bytes()) | |||
| } | |||
|
|
|||
| // getObjectMetaByNameHandler handle get object info by name request | |||
| func (gateway *Gateway) getObjectByNameHandler(w http.ResponseWriter, r *http.Request) { | |||
There was a problem hiding this comment.
Rename getObjectByNameHandler to getObjectMetaByNameHandler?
service/gateway/router.go
Outdated
| @@ -26,6 +26,8 @@ const ( | |||
| queryUploadProgressRouterName = "queryUploadProgress" | |||
| downloadObjectByUniversalEndpointName = "DownloadObjectByUniversalEndpoint" | |||
| viewObjectByUniversalEndpointName = "ViewObjectByUniversalEndpoint" | |||
| getObjectByNameRouterName = "getObjectByName" | |||
service/gateway/router.go
Outdated
| Name(getBucketAndPaymentRouterName). | ||
| Methods(http.MethodGet). | ||
| Path("/bucket/payment"). | ||
| HandlerFunc(g.getBucketWithPaymentHandler) |
There was a problem hiding this comment.
What happens if objectName=/bucket/payment?
It is best to modify the router and supplement the corresponding unit tests. It's easy to go wrong here.
There was a problem hiding this comment.
Fixed, using query param instead and added unit tests.
service/gateway/router.go
Outdated
| Name(getObjectByNameRouterName). | ||
| Methods(http.MethodGet). | ||
| Path("/{object:.+}/metadata"). | ||
| HandlerFunc(g.getObjectByNameHandler) |
There was a problem hiding this comment.
Ditto, what happens if objectName=/abc/metadata?
There was a problem hiding this comment.
Fixed, using query param instead.
service/gateway/router.go
Outdated
| @@ -85,7 +97,7 @@ func (g *Gateway) registerHandler(r *mux.Router) { | |||
| HandlerFunc(g.listObjectsByBucketNameHandler) | |||
| hostBucketRouter.NotFoundHandler = http.HandlerFunc(g.notFoundHandler) | |||
|
|
|||
| // bucket list router, virtual-hosted style | |||
| // bucket list router, path style | |||
| bucketListRouter := r.Host(g.config.Domain).Subrouter() | |||
There was a problem hiding this comment.
If it is a path style, it does not need to match host.
service/gateway/router.go
Outdated
| Name(getBucketAndPaymentRouterName). | ||
| Methods(http.MethodGet). | ||
| Path("/bucket/payment"). | ||
| HandlerFunc(g.getBucketWithPaymentHandler) |
There was a problem hiding this comment.
what happens if bucketName=bucket and objectName=payment?
There was a problem hiding this comment.
Fixed, using query param instead.
service/gateway/router.go
Outdated
| Name(getObjectByNameRouterName). | ||
| Methods(http.MethodGet). | ||
| Path("/{object:.+}/metadata"). | ||
| HandlerFunc(g.getObjectByNameHandler) |
There was a problem hiding this comment.
Fixed, using query param instead.
feat: modify long name feat: sort import correctly fix: typos fix: typo
annielz
force-pushed
the
feat-getobjectandbucketbyname
branch
from
2743b7e to
d167387
Compare
annielz
force-pushed
the
feat-getobjectandbucketbyname
branch
from
d167387 to
f447c3c
Compare
annielz
changed the title
will-2012
approved these changes
May 11, 2023
model/const.go
Outdated
| // GetBucketMetaByNameQuery defines get bucket metadata query, which is used to route request | ||
| GetBucketMetaByNameQuery = "bucket-meta" | ||
| // GetObjectMetaByNameQuery defines get object metadata query, which is used to route request | ||
| GetObjectMetaByNameQuery = "object-meta" |
There was a problem hiding this comment.
I think it is also suitable to remove ByName, because in our scene, the default is indexed by Name. (Maybe only ByObjectID needs to be pointed out).
There was a problem hiding this comment.
Make sense, I renamed them and related methods, thanks for the suggestion!
service/gateway/metadata_handler.go
Outdated
| }() | ||
|
|
||
| if gateway.metadata == nil { | ||
| log.Error("failed to get object by name due to not config metadata") |
service/gateway/metadata_handler.go
Outdated
| ctx := log.Context(context.Background(), req) | ||
| resp, err := gateway.metadata.GetObjectByObjectNameAndBucketName(ctx, req) | ||
| if err != nil { | ||
| log.Errorf("failed to get object by object name and bucket name", "error", err) |
service/gateway/metadata_handler.go
Outdated
|
|
||
| m := jsonpb.Marshaler{EmitDefaults: true, OrigName: true, EnumsAsInts: true} | ||
| if err = m.Marshal(&b, resp); err != nil { | ||
| log.Errorf("failed to get object by object name and bucket name", "error", err) |
service/gateway/router_test.go
Outdated
| wantedRouterName: getObjectMetaByNameRouterName, | ||
| }, | ||
| { | ||
| name: "Get object upload progress router, path style", |
service/gateway/router_test.go
Outdated
| wantedRouterName: getBucketMetaByNameRouterName, | ||
| }, | ||
| { | ||
| name: "Get bucket upload progress router, path style", |
There was a problem hiding this comment.
Pls add list user buckets router ut.
There was a problem hiding this comment.
The list user buckets router ut is already there, named getUserBucketsRouterName.
fix: import position
annielz
force-pushed
the
feat-getobjectandbucketbyname
branch
from
6f75b48 to
9b42fa6
Compare
clydemeng
approved these changes
May 11, 2023
ruojunm
approved these changes
May 11, 2023
joeylichang
added a commit
that referenced
this pull request
May 15, 2023
* feat: sp services add pprof * feat:block syncer add read concurrency support * fix: rename pprof var name * fix:debug log * fix: fix the content of sp readme * feat: add universal endpoint view option (#383) * feat: add universal endpoint view option fix: imports * fix: add space before comments * fix: put header in right place (#385) * feat: signer async send sealObject tx * fix: simplify gnfd client signature test case * fix: return function when get nonce on chain failed * fix: annotation * fix: cross region netroek opt * feat: implement stop serving on testnet (#336) * feat: localup shell adds generate sp.info and db.info function (#398) * feat: localup shell adds generate sp.info and db.info function * fix: fix reset db function name * feat: support full-memory replicate task (#410) * feat: add gc worker (#408) * feat: bsdb switch (#402) * feat: update metadata db config and startDBSwitchListener * feat: update e2e config * fix: rename bsdbflag and update config/config_template.toml * feat:add upload download add bandwidth limit * feat: list objects pagination & folder path (#404) * feat: add pagination for list objects * fix: define constant queries & update func name * fix: update filter for listobjects when delimiter is empty * fix: refine query name * fix: nil config.SQLDBConfig (#413) * feat: add get object meta and get bucket meta apis (#412) * feat: add get object and get bucket with payment apis feat: modify long name feat: sort import correctly fix: typos fix: typo * fix: address comments, change path params to query params fix: typo * feat: add unit tests for new apis * fix: rename services and comments according to comments fix: import position * fix:BandwidthLimit config * fix:bandwidth config * feat: adapt greenfield v0.47 (#406) * fix: upgrade greenfield version to v0.2.0 (#415) Co-authored-by: VM <arimas@foxmail.com> * feat: add dual db warm up support for blocksyncer (#401) * feat: add dual db warm up support for blocksyncer * feat: add stop main service logic * feat: update chain version to v0.47 and some adapts * fix: upgrade e2e ci greenfield version to v0.2.0 (#418) Co-authored-by: VM <arimas@foxmail.com> * docs: add v0.2.0 changelog (#417) * docs: add v0.2.0 changelog * fix: add blocksycner dual db feature in changelog * docs: add localup shell feature in changelog --------- Co-authored-by: actioncli <joeycli0919@qq.com> Co-authored-by: krish-nr <krish.z@nodereal.io> Co-authored-by: VM <arimas@foxmail.com> * feat: replace github.com/gogo/protobuf with github.com/cosmos/gogoproto (#422) --------- Co-authored-by: VM <arimas@foxmail.com> Co-authored-by: constwz <changbohao30@gmail.com> Co-authored-by: Annie <108039750+annielz@users.noreply.github.com> Co-authored-by: j75689 <j75689@gmail.com> Co-authored-by: constwz <122766871+constwz@users.noreply.github.com> Co-authored-by: forcodedancing <just.haha.it@gmail.com> Co-authored-by: VM <112189277+sysvm@users.noreply.github.com> Co-authored-by: will-2012 <117156346+will-2012@users.noreply.github.com> Co-authored-by: Barry <122767193+BarryTong65@users.noreply.github.com> Co-authored-by: Chris Li <jingjunLi@users.noreply.github.com> Co-authored-by: krish-z <122767080+krish-nr@users.noreply.github.com> Co-authored-by: actioncli <joeycli0919@qq.com> Co-authored-by: krish-nr <krish.z@nodereal.io> Co-authored-by: yutianwu <wzxingbupt@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add getObjectMeta and getBucketMeta apis in metadata service.
Rationale
Adding 2 apis for DCellar frontend optimization usage.
Example
getObjectMeta:
{bucket_name}.localhost:9033/{object_name}?object-meta
localhost:9033/{bucket_name}/{object_name}?object-meta
getBucketMeta:
{bucket_name}.localhost:9033?bucket-meta
localhost:9033/{bucket_name}?bucket-meta
Changes
Notable changes: