add filter gateway #948

kpango · 2021-01-24T19:44:01Z

Signed-off-by: kpango kpango@vdaas.org

Description:

this PR includes 4 feature.

Added filter Gateway
Add Container SecurityContext
change grpc GetObject interface for filter gateway implementation
update k8s manifests for resolving kube-linter.

Related Issue:

How Has This Been Tested?:

Environment:

Go Version: 1.15.7
Docker Version: 19.03.8
Kubernetes Version: 1.18.2
NGT Version: 1.13.1

Types of changes:

Bug fix [type/bug]
New feature [type/feature]
Add tests [type/test]
Security related changes [type/security]
Add documents [type/documentation]
Refactoring [type/refactoring]
Update dependencies [type/dependency]
Update benchmarks and performances [type/bench]
Update CI [type/ci]

Changes to Core Features:

Have you added an explanation of what your changes do and why you'd like us to include them?
Have you written new tests for your core changes, as applicable?
Have you successfully ran tests with your changes locally?

Checklist:

I have read the CONTRIBUTING document.
I have checked open Pull Requests for the similar feature or fixes?
I have added tests and benchmarks to cover my changes.
I have ensured all new and existing tests passed.
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly.

Signed-off-by: kpango <kpango@vdaas.org>

vdaas-ci · 2021-01-24T19:44:21Z

[WARNING:INTCFG] Changes in interal/config may require you to change Helm charts. Please check.

vdaas-ci · 2021-01-24T19:44:23Z

[CHATOPS:HELP] ChatOps commands.

🙆‍♀️ /approve - approve
💌 /changelog - add changelog comment
🍱 /format - format codes and add licenses
✅ /gen-test - generate test codes
🏷️ /label - add labels
⏪ /rebase - rebase master
🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

pkg/gateway/filter/router/router.go

internal/client/v1/client/filter/egress/client.go

pkg/gateway/filter/handler/grpc/handler.go

internal/client/v1/client/filter/ingress/client.go

codecov · 2021-01-24T19:59:37Z

Codecov Report

Merging #948 (e3e900e) into master (d4bf810) will decrease coverage by 1.03%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #948      +/-   ##
==========================================
- Coverage   16.23%   15.20%   -1.04%     
==========================================
  Files         478      493      +15     
  Lines       23955    25683    +1728     
==========================================
+ Hits         3889     3904      +15     
- Misses      19827    21530    +1703     
- Partials      239      249      +10

Impacted Files	Coverage Δ
cmd/gateway/filter/main.go	`0.00% <0.00%> (ø)`
hack/benchmark/internal/client/ngtd/grpc/client.go	`0.00% <0.00%> (ø)`
hack/benchmark/internal/client/ngtd/grpc/stream.go	`0.00% <0.00%> (ø)`
hack/benchmark/internal/client/ngtd/rest/client.go	`0.00% <0.00%> (ø)`
internal/client/v1/client/discoverer/discover.go	`0.00% <ø> (ø)`
internal/client/v1/client/filter/egress/client.go	`0.00% <0.00%> (ø)`
internal/client/v1/client/filter/egress/option.go	`0.00% <0.00%> (ø)`
internal/client/v1/client/filter/ingress/client.go	`0.00% <0.00%> (ø)`
internal/client/v1/client/filter/ingress/option.go	`0.00% <0.00%> (ø)`
internal/client/v1/client/vald/vald.go	`0.00% <0.00%> (ø)`
... and 48 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d4bf810...98d38c8. Read the comment docs.

Makefile.d/k8s.mk

apis/proto/v1/payload/payload.proto

charts/vald/templates/discoverer/daemonset.yaml

charts/vald/templates/jobs/db/initialize/scylla/configmap.yaml

Signed-off-by: kpango <kpango@vdaas.org>

cmd/gateway/filter/main_test.go

internal/client/v1/client/vald/vald.go

pkg/gateway/filter/config/config_test.go

internal/client/v1/client/filter/egress/client_test.go

pkg/gateway/filter/usecase/vald.go

pkg/gateway/filter/handler/grpc/handler.go

rinx · 2021-01-26T02:23:56Z

The Cassandra cluster deployed by using make k8s/external/cassandra/deploy doesn't start correctly on my K8s cluster.
I also tried to deploy by using the task on the master branch, it failed too. So it has been already broken.
We should fix it later. 😅

kpango · 2021-01-26T02:29:16Z

seems k8s/external/cassandra directory is not changed since v0.0.26 and formatted on v0.0.25

rinx · 2021-01-26T02:30:49Z

The problem is not caused by our changes. Because the problem is happened in preStop hook. Maybe there's some changes in the cassandra:latest container.

kpango · 2021-01-26T02:31:54Z

I see, how about start using,
https://k8ssandra.io/docs/topics/ingress/traefik/k3d-deployment/
or
https://github.com/datastax/cass-operator
managed cassandra can save maintainance cost.

kpango · 2021-01-26T02:32:45Z

Oh, sorry let's discuss it on issue, (could you please make new issue?)

kpango · 2021-01-26T02:33:02Z

How about this PR?

rinx · 2021-01-26T02:33:28Z

Right. But it is not the scope of the current PR. So let's deal with it later.

rinx · 2021-01-26T02:35:43Z

There's no more comments about this PR from me.

@kevindiu @kmrmt @hlts2 @vankichi @datelier Could you review it?

kpango · 2021-01-26T11:18:31Z

@kevindiu @kmrmt @hlts2 @vankichi @datelier can you please review this PR by thursday?

Signed-off-by: kpango <kpango@vdaas.org>

internal/client/v1/client/filter/egress/client.go

pkg/gateway/filter/handler/doc.go

pkg/gateway/filter/handler/grpc/handler.go

Signed-off-by: kpango <kpango@vdaas.org>

rinx · 2021-01-27T07:04:03Z

The new changes about chart is okay.

However, the newly added build options and upx may be a bad combination. Please fix the builds.

Signed-off-by: kpango <kpango@vdaas.org>

internal/errors/filter.go

internal/client/v1/client/filter/ingress/client.go

vankichi

LGTM

internal/client/v1/client/filter/egress/client.go

internal/client/v1/client/filter/egress/option.go

internal/client/v1/client/filter/egress/client.go

kpango · 2021-01-27T07:26:09Z

@rinx I fixed about building failure, may be upx cannot use with pie build...

kpango · 2021-01-27T07:31:26Z

@rinx @hlts2 @vankichi @datelier thank you for review.
I'm going to merge this pr.

add filter gateway

3299317

Signed-off-by: kpango <kpango@vdaas.org>

github-actions bot added area/agent/core area/gateway area/gateway/backup area/gateway/filter area/gateway/lb area/gateway/meta area/helm area/internal area/makefile area/manager/index team/set SET team team/sre SRE team type/ci size/XXXL type/feature New feature priority/medium labels Jan 24, 2021

github-actions bot reviewed Jan 24, 2021

View reviewed changes

kpango requested review from rinx, vankichi, datelier, hlts2 and kevindiu January 25, 2021 02:54

kpango mentioned this pull request Jan 25, 2021

add tensorflow filter #951

Merged

18 tasks

rinx reviewed Jan 25, 2021

View reviewed changes

Makefile.d/k8s.mk Show resolved Hide resolved

apis/proto/v1/payload/payload.proto Show resolved Hide resolved

charts/vald/templates/discoverer/daemonset.yaml Show resolved Hide resolved

charts/vald/templates/jobs/db/initialize/scylla/configmap.yaml Outdated Show resolved Hide resolved

fix helmcharts & rename Object.Request to Object.VectorRequest

5658987

Signed-off-by: kpango <kpango@vdaas.org>

github-actions bot reviewed Jan 25, 2021

View reviewed changes

rinx mentioned this pull request Jan 26, 2021

Makefile task k8s/external/cassandra/deploy broken. #955

Closed

datelier previously approved these changes Jan 27, 2021

View reviewed changes

kpango dismissed datelier’s stale review via d82b240 January 27, 2021 06:43

fix miss configured filter configmap chart templates

8079793

Signed-off-by: kpango <kpango@vdaas.org>

kpango force-pushed the feature/gateway-filter/add-filter-gateway branch from d82b240 to 8079793 Compare January 27, 2021 06:47

hlts2 reviewed Jan 27, 2021

View reviewed changes

internal/client/v1/client/filter/egress/client.go Outdated Show resolved Hide resolved

github-actions bot reviewed Jan 27, 2021

View reviewed changes

change empty slice check

7fc5b78

Signed-off-by: kpango <kpango@vdaas.org>

datelier previously approved these changes Jan 27, 2021

View reviewed changes

remove buildmode=pie

98d38c8

Signed-off-by: kpango <kpango@vdaas.org>

kpango dismissed datelier’s stale review via 98d38c8 January 27, 2021 07:10

github-actions bot reviewed Jan 27, 2021

View reviewed changes

internal/errors/filter.go Show resolved Hide resolved

internal/client/v1/client/filter/ingress/client.go Show resolved Hide resolved

internal/client/v1/client/filter/ingress/client.go Show resolved Hide resolved

internal/client/v1/client/filter/ingress/client.go Show resolved Hide resolved

vankichi approved these changes Jan 27, 2021

View reviewed changes

github-actions bot reviewed Jan 27, 2021

View reviewed changes

kpango merged commit f8abd67 into master Jan 27, 2021

kpango deleted the feature/gateway-filter/add-filter-gateway branch January 27, 2021 07:34

vdaas-ci mentioned this pull request Jan 29, 2021

Bug fix for gateway #963

Merged

18 tasks

kpango mentioned this pull request Feb 2, 2021

[major] v1.0.0 Release #974

Merged

11 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add filter gateway #948

add filter gateway #948

kpango commented Jan 24, 2021 •

edited

Loading

vdaas-ci commented Jan 24, 2021

vdaas-ci commented Jan 24, 2021

codecov bot commented Jan 24, 2021 •

edited

Loading

rinx commented Jan 26, 2021

kpango commented Jan 26, 2021 •

edited

Loading

rinx commented Jan 26, 2021 •

edited

Loading

kpango commented Jan 26, 2021

kpango commented Jan 26, 2021

kpango commented Jan 26, 2021

rinx commented Jan 26, 2021

rinx commented Jan 26, 2021

kpango commented Jan 26, 2021

rinx commented Jan 27, 2021 •

edited

Loading

vankichi left a comment

kpango commented Jan 27, 2021

kpango commented Jan 27, 2021

add filter gateway #948

add filter gateway #948

Conversation

kpango commented Jan 24, 2021 • edited Loading

Description:

Related Issue:

How Has This Been Tested?:

Environment:

Types of changes:

Changes to Core Features:

Checklist:

vdaas-ci commented Jan 24, 2021

vdaas-ci commented Jan 24, 2021

codecov bot commented Jan 24, 2021 • edited Loading

Codecov Report

rinx commented Jan 26, 2021

kpango commented Jan 26, 2021 • edited Loading

rinx commented Jan 26, 2021 • edited Loading

kpango commented Jan 26, 2021

kpango commented Jan 26, 2021

kpango commented Jan 26, 2021

rinx commented Jan 26, 2021

rinx commented Jan 26, 2021

kpango commented Jan 26, 2021

rinx commented Jan 27, 2021 • edited Loading

vankichi left a comment

Choose a reason for hiding this comment

kpango commented Jan 27, 2021

kpango commented Jan 27, 2021

kpango commented Jan 24, 2021 •

edited

Loading

codecov bot commented Jan 24, 2021 •

edited

Loading

kpango commented Jan 26, 2021 •

edited

Loading

rinx commented Jan 26, 2021 •

edited

Loading

rinx commented Jan 27, 2021 •

edited

Loading