-
Notifications
You must be signed in to change notification settings - Fork 78
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implement base of continuous benchmark tool (#1776)
* Create Continuous Bench Search Job tool (#1733) * ✨ create bench job search tools Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add load hdf5 functions Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix docker and use hdf5 data Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix proto Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 add benchmark job image build ci Signed-off-by: vankichi <kyukawa315@gmail.com> * 💚 invest Signed-off-by: vankichi <kyukawa315@gmail.com> * Revert "💚 invest" This reverts commit f0f585c. * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply code review Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * ✨ apply from feedback Signed-off-by: vankichi <kyukawa315@gmail.com> * Update internal/config/benchmark.go Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ change directory path Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> * Add crds for continuous benchmark tools (#1789) * ✨ add crds for continuous benchmark operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ add benchmark operator/job scheme Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ rename package names and add doc.go Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create runtime object Signed-off-by: vankichi <kyukawa315@gmail.com> * Apply suggestions from code review Co-authored-by: Yusuke Kato <kpango@vdaas.org> * ♻️ apply feedback Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> * Add Job reconciler & Change directory constitution of internal/k8s for benchmark (#1825) * ✨ ♻️ add Job reconciler & use scenario instead of operator Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ fix format & rename file Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Add benchmark operator framework (#1916) * ✨ impl benchmark reconciler Signed-off-by: vankichi <kyukawa315@gmail.com> * ✨ create benchmark operator framework Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ remove unness changes Signed-off-by: vankichi <kyukawa315@gmail.com> Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * impl reconcile logic for create benchmark job (#1923) * ✨ impl reconcile logic for create benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ fix Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark's crds Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ resolve error due to update conn bench crds for pkg/tools/benchmark/job Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor continuous benchmark job logic Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ update charts Signed-off-by: vankichi <kyukawa315@gmail.com> * Format code with prettier and gofumpt * ♻️ rafactor con bench config and bug fix reconcile logic Signed-off-by: vankichi <kyukawa315@gmail.com> * 🐛 Bugfix: fix typo and recall function logic Signed-off-by: vankichi <kyukawa315@gmail.com> * ♻️ refactor pkg benchmark job Signed-off-by: vankichi <kyukawa315@gmail.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com> --------- Signed-off-by: vankichi <kyukawa315@gmail.com> Co-authored-by: Yusuke Kato <kpango@vdaas.org> Co-authored-by: Hiroto Funakoshi <hiroto.funakoshi.hiroto@gmail.com> Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
- Loading branch information
4 people
committed
Dec 26, 2023
1 parent
8dc67ee
commit fb2fb66
Showing
80 changed files
with
7,103 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| # | ||
| # Copyright (C) 2019-2022 vdaas.org vald team <vald@vdaas.org> | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # https://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| # | ||
| name: "Build docker image: benchmark-job" | ||
| on: | ||
| push: | ||
| branches: | ||
| - master | ||
| tags: | ||
| - "*.*.*" | ||
| - "v*.*.*" | ||
| - "*.*.*-*" | ||
| - "v*.*.*-*" | ||
| paths: | ||
| - ".github/actions/docker-build/actions.yaml" | ||
| - ".github/workflows/dockers-benchmak-job-image.yml" | ||
| - "go.mod" | ||
| - "go.sum" | ||
| - "internal/**" | ||
| - "!internal/**/*_test.go" | ||
| - "!internal/db/**" | ||
| - "!internal/k8s/**" | ||
| - "apis/grpc/**" | ||
| - "pkg/benchmark/job/**" | ||
| - "cmd/benchmark/job/**" | ||
| - "dockers/benchmark/job/Dockerfile" | ||
| - "versions/GO_VERSION" | ||
| - "versions/NGT_VERSION" | ||
| pull_request: | ||
| paths: | ||
| - ".github/actions/docker-build/actions.yaml" | ||
| - ".github/workflows/dockers-benchmak-job-image.yml" | ||
| - "go.mod" | ||
| - "go.sum" | ||
| - "internal/**" | ||
| - "!internal/**/*_test.go" | ||
| - "!internal/db/**" | ||
| - "!internal/k8s/**" | ||
| - "apis/grpc/**" | ||
| - "pkg/benchmark/job/**" | ||
| - "cmd/benchmark/job/**" | ||
| - "dockers/benchmark/job/Dockerfile" | ||
| - "versions/GO_VERSION" | ||
| - "versions/NGT_VERSION" | ||
|
|
||
| jobs: | ||
| build: | ||
| strategy: | ||
| max-parallel: 4 | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| - name: Setup QEMU | ||
| uses: docker/setup-qemu-action@v1 | ||
| with: | ||
| platforms: all | ||
| - name: Setup Docker Buildx | ||
| id: buildx | ||
| uses: docker/setup-buildx-action@v1 | ||
| with: | ||
| buildkitd-flags: "--debug" | ||
| - name: Login to DockerHub | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| username: ${{ secrets.DOCKERHUB_USER }} | ||
| password: ${{ secrets.DOCKERHUB_PASS }} | ||
| - name: Login to GitHub Container Registry | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ secrets.PACKAGE_USER }} | ||
| password: ${{ secrets.PACKAGE_TOKEN }} | ||
| - name: Build and Publish | ||
| id: build_and_publish | ||
| uses: ./.github/actions/docker-build | ||
| with: | ||
| target: benchmark-job | ||
| builder: ${{ steps.buildx.outputs.name }} | ||
| - name: Initialize CodeQL | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: github/codeql-action/init@v2 | ||
| - name: Run vulnerability scanner (table) | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: aquasecurity/trivy-action@master | ||
| with: | ||
| image-ref: "${{ steps.build_and_publish.outputs.IMAGE_NAME }}:${{ steps.build_and_publish.outputs.PRIMARY_TAG }}" | ||
| format: "table" | ||
| - name: Run vulnerability scanner (sarif) | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: aquasecurity/trivy-action@master | ||
| with: | ||
| image-ref: "${{ steps.build_and_publish.outputs.IMAGE_NAME }}:${{ steps.build_and_publish.outputs.PRIMARY_TAG }}" | ||
| format: "template" | ||
| template: "@/contrib/sarif.tpl" | ||
| output: "trivy-results.sarif" | ||
| - name: Upload Trivy scan results to Security tab | ||
| if: startsWith( github.ref, 'refs/tags/') | ||
| uses: github/codeql-action/upload-sarif@v2 | ||
| with: | ||
| sarif_file: "trivy-results.sarif" | ||
| slack: | ||
| name: Slack notification | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| if: github.ref == 'refs/heads/master' || startsWith( github.ref, 'refs/tags/') | ||
| steps: | ||
| - uses: technote-space/workflow-conclusion-action@v2 | ||
| with: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| - uses: 8398a7/action-slack@v3 | ||
| with: | ||
| author_name: benchmark-job image build | ||
| status: ${{ env.WORKFLOW_CONCLUSION }} | ||
| only_mention_fail: channel | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| SLACK_WEBHOOK_URL: ${{ secrets.SLACK_NOTIFY_WEBHOOK_URL }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.