[Merge pending] Add mirror gateway definition #1948

hlts2 · 2023-02-09T13:53:19Z

Description:

WHAT

Added proto definition for mirror gateway.
The following RPCs are added.

Register RPC: register mirror servers.
Target RPC: get other mirror servers.

WHY

we need to support request mirroring for BCP

⚠️ FYI

Although there are many differences, most of them are due to automatic generation.
The following is the main addition.

Related PR:

#1792

Versions:

Go Version: 1.19.5
Docker Version: 20.10.8
Kubernetes Version: 1.22.0
NGT Version: 2.0.9

Checklist:

I have read the CONTRIBUTING document and completed our CLA agreement.
I have checked open Pull Requests for the similar feature or fixes?

Special notes for your reviewer:

How to deploy

Update helm repo for monitoring

helm repo update

Create k3d cluster

make k3d/restart

Deploy monitoring stack

 make k8s/external/cert-manager/deploy k8s/monitoring/deploy

Deploy multiple vald clusters

 make k8s/multi/vald/deploy

Deploy MirrorTarget resources

---
apiVersion: vald.vdaas.org/v1
kind: ValdMirrorTarget
metadata:
  name: mirror-target-01
  labels:
    group: mirror-group-01
spec:
  colocation: dc1
  target:
    host: vald-mirror-gateway.vald-01.svc.cluster.local
    port: 8081
---
apiVersion: vald.vdaas.org/v1
kind: ValdMirrorTarget
metadata:
  name: mirror-target-02
  labels:
    group: mirror-group-01
spec:
  colocation: dc1
  target:
    host: vald-mirror-gateway.vald-02.svc.cluster.local
    port: 8081

(E2E test)

Please execute the following command if you want to perform the e2e test.

E2E_TARGET_NAME=vald-mirror-gateway E2E_TARGET_NAMESPACE=vald-03 make e2e

vdaas-ci · 2023-02-09T13:53:43Z

[CHATOPS:HELP] ChatOps commands.

🙆‍♀️ /approve - approve
💌 /changelog - replace the PR body by changelog details
🍱 /format - format codes and add licenses
✅ /gen-test - generate test codes
🏷️ /label - add labels
⏪ /rebase - rebase main
🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

github-actions · 2023-02-09T13:54:27Z

apis/docs/v1/docs.md

+
+Copyright (C) 2019-2022 vdaas.org vald team &lt;vald@vdaas.org&gt;
+
+Licensed under the Apache License, Version 2.0 (the &#34;License&#34;);


[LanguageTool] _{reported by reviewdog 🐶}
Unpaired symbol: ‘)’ seems to be missing (EN_UNPAIRED_BRACKETS)
URL: https://languagetool.org/insights/post/punctuation-guide/#what-are-parentheses
Rule: https://community.languagetool.org/rule/show/EN_UNPAIRED_BRACKETS?lang=en-US
Category: PUNCTUATION

cloudflare-pages · 2023-03-03T02:47:39Z

Deploying with Cloudflare Pages

Latest commit:	`08c44fb`
Status:	✅ Deploy successful!
Preview URL:	https://d80d7551.vald.pages.dev
Branch Preview URL:	https://feature-mirror-gateway-defin.vald.pages.dev

View logs

vdaas-ci · 2023-04-25T07:51:51Z

[WARNING:INTCFG] Changes in interal/config may require you to change Helm charts. Please check.

github-actions · 2023-04-25T07:51:55Z

dockers/gateway/mirror/Dockerfile

+ENV APP_NAME mirror
+
+# skipcq: DOK-DL3008
+RUN apt-get update && apt-get install -y --no-install-recommends \


⚠️ [hadolint] <DL3008> _{reported by reviewdog 🐶}
Pin versions in apt get install. Instead of apt-get install <package> use apt-get install <package>=<version>

This reverts commit 7ece16f.

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

This reverts commit 454eaf3.

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

* fix: returns all connection information in the registration rpc Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: lint warnning Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * refactor: deleted unnecessary parameter Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> --------- Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

* feat: add new crud logic Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * refactor code Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * refactor: error handling logic Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: bugfix error handling for rpc Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * feat: add test for RemoveByTimestamp Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: fails error test Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * refactor: test logic Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: add new condition and add test case Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * refactor code Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: error handling and comment Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: test case name and test case order Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: execute format Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: add new comment Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> --------- Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

* fix: update mirror document Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: format Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * feat: add removeByTimestamp section Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: grammar warning Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: grammar warning and refactor document Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: grammar warning Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * feat: add mirror gateway troubleshooting docs Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * style: format code with Gofumpt and Prettier This commit fixes the style issues introduced in 2c492f7 according to the output from Gofumpt and Prettier. Details: #2207 * feat: add contents of troubleshooting document Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: grammar warning Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: grammar Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: link path Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: invalid link path Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: refactor docs Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: deleted unnecessary contents Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: status handling logic document Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: document refactor Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: bugfix status handling document Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> * fix: refactor sentence Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> --------- Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

github-actions · 2023-11-16T06:04:56Z

docs/user-guides/mirroring-configuration.md

+      # gRPC client configuration (overrides defaults.grpc.client)
+      client: {}
+      # The duration to register other Mirror Gateways.
+      register_duration: "1s"


[LanguageTool] _{reported by reviewdog 🐶}
This sentence does not start with an uppercase letter. (UPPERCASE_SENTENCE_START)
Suggestions: Register
URL: https://languagetool.org/insights/post/spelling-capital-letters/
Rule: https://community.languagetool.org/rule/show/UPPERCASE_SENTENCE_START?lang=en-US
Category: CASING

github-actions · 2023-11-16T06:07:51Z

pkg/gateway/mirror/handler/grpc/handler_test.go

+			ctx, cancel := context.WithCancel(context.Background())
+			eg, egctx := errgroup.New(ctx)
+
+			uuid := "test"


🚫 [golangci] _{reported by reviewdog 🐶}
string test has 26 occurrences, make it a constant (goconst)

github-actions · 2023-11-16T06:07:51Z

internal/config/mirror.go

+	// PodName represents the mirror gateway pod name.
+	PodName string `json:"pod_name"           yaml:"pod_name"`
+	// RegisterDuration represents the duration to register Mirror Gateway.
+	RegisterDuration string `json:"register_duration"  yaml:"register_duration"`


🚫 [golangci] _{reported by reviewdog 🐶}
tag is not aligned , should be: json:"register_duration" yaml:"register_duration" (tagalign)

github-actions · 2023-11-16T06:07:51Z

pkg/gateway/mirror/handler/grpc/handler_test.go

+		UnimplementedValdServerWithMirror vald.UnimplementedValdServerWithMirror
+	}
+	type want struct {
+		wantId *payload.Object_ID


🚫 [golangci] _{reported by reviewdog 🐶}
ST1003: struct field wantId should be wantID (stylecheck)

github-actions · 2023-11-16T06:07:51Z

pkg/gateway/mirror/handler/grpc/handler_test.go

+				UnimplementedValdServerWithMirror: test.fields.UnimplementedValdServerWithMirror,
+			}
+
+			gotId, err := s.Exists(test.args.ctx, test.args.meta)


🚫 [golangci] _{reported by reviewdog 🐶}
ST1003: var gotId should be gotID (stylecheck)

github-actions bot added area/gateway/filter area/gateway/lb area/manager/index size/S type/feature New feature priority/medium labels Feb 9, 2023

github-actions bot reviewed Feb 9, 2023

View reviewed changes

hlts2 changed the title ~~[WIP] Add mirror gateway definition~~ Add mirror gateway definition Feb 13, 2023

hlts2 force-pushed the feature/mirror-gateway-definition branch from 59a8a5f to 568db0d Compare March 3, 2023 02:47

hlts2 force-pushed the feature/mirror-gateway-definition branch 3 times, most recently from 8ece068 to 2909af0 Compare March 8, 2023 08:17

hlts2 force-pushed the feature/mirror-gateway-definition branch 2 times, most recently from 9fdc06b to 9c1ef3f Compare March 17, 2023 02:43

hlts2 force-pushed the feature/mirror-gateway-definition branch 3 times, most recently from bb05061 to 618cc97 Compare April 10, 2023 07:18

hlts2 force-pushed the feature/mirror-gateway-definition branch 4 times, most recently from 11eec16 to 3496ac1 Compare April 17, 2023 06:36

hlts2 force-pushed the feature/mirror-gateway-definition branch from 3496ac1 to a54e74e Compare April 25, 2023 07:43

github-actions bot reviewed Apr 25, 2023

View reviewed changes

github-actions bot added area/helm area/internal area/makefile team/set SET team labels Apr 25, 2023

Revert "add cluster-info dump command"

2ed49ca

This reverts commit 7ece16f.

hlts2 added actions/e2e-deploy and removed actions/e2e-deploy labels Sep 26, 2023

debug: add logs command

454eaf3

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

hlts2 added actions/e2e-deploy and removed actions/e2e-deploy labels Sep 27, 2023

hlts2 added 2 commits September 27, 2023 11:06

Revert "debug: add logs command"

388f8ef

This reverts commit 454eaf3.

add RemoveByTimestamp for mirror

91deb87

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

hlts2 added actions/e2e-deploy and removed actions/e2e-deploy labels Sep 27, 2023

fix: update sleep time for operator deploy

0b96823

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

hlts2 added actions/e2e-deploy and removed actions/e2e-deploy labels Sep 28, 2023

hlts2 added 3 commits September 28, 2023 11:50

update wait time for vmt

86f4211

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

hlts2 changed the title ~~Add mirror gateway definition~~ [Merge pending] Add mirror gateway definition Oct 12, 2023

hlts2 and others added 3 commits November 8, 2023 16:08

Merge branch 'main' into feature/mirror-gateway-definition

746e01c

feat: schema and protobuf update

08c44fb

Signed-off-by: hlts2 <hiroto.funakoshi.hiroto@gmail.com>

github-actions bot removed area/discoverer area/agent/core area/agent/sidecar labels Nov 16, 2023

github-actions bot reviewed Nov 16, 2023

View reviewed changes

hlts2 closed this Nov 16, 2023

hlts2 mentioned this pull request Nov 16, 2023

[Merge pending] Add mirror gateway definition #2237

Closed

2 tasks

hlts2 mentioned this pull request Dec 7, 2023

Add mirror gateway definitions and Implementations #2262

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Merge pending] Add mirror gateway definition #1948

[Merge pending] Add mirror gateway definition #1948

hlts2 commented Feb 9, 2023 •

edited

Loading

vdaas-ci commented Feb 9, 2023

github-actions bot Feb 9, 2023

cloudflare-pages bot commented Mar 3, 2023 •

edited

Loading

vdaas-ci commented Apr 25, 2023

github-actions bot Apr 25, 2023

github-actions bot Nov 16, 2023

github-actions bot Nov 16, 2023

github-actions bot Nov 16, 2023

github-actions bot Nov 16, 2023

github-actions bot Nov 16, 2023


		Copyright (C) 2019-2022 vdaas.org vald team <vald@vdaas.org>

		Licensed under the Apache License, Version 2.0 (the "License");

[Merge pending] Add mirror gateway definition #1948

[Merge pending] Add mirror gateway definition #1948

Conversation

hlts2 commented Feb 9, 2023 • edited Loading

Description:

WHAT

WHY

⚠️ FYI

Related PR:

Versions:

Checklist:

Special notes for your reviewer:

How to deploy

vdaas-ci commented Feb 9, 2023

github-actions bot Feb 9, 2023

Choose a reason for hiding this comment

cloudflare-pages bot commented Mar 3, 2023 • edited Loading

Deploying with Cloudflare Pages

vdaas-ci commented Apr 25, 2023

github-actions bot Apr 25, 2023

Choose a reason for hiding this comment

github-actions bot Nov 16, 2023

Choose a reason for hiding this comment

github-actions bot Nov 16, 2023

Choose a reason for hiding this comment

github-actions bot Nov 16, 2023

Choose a reason for hiding this comment

github-actions bot Nov 16, 2023

Choose a reason for hiding this comment

github-actions bot Nov 16, 2023

Choose a reason for hiding this comment

hlts2 commented Feb 9, 2023 •

edited

Loading

cloudflare-pages bot commented Mar 3, 2023 •

edited

Loading