Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[suggestion] [ci] improve iroha2 build #2461

Closed
BAStos525 opened this issue Jul 8, 2022 · 9 comments
Closed

[suggestion] [ci] improve iroha2 build #2461

BAStos525 opened this issue Jul 8, 2022 · 9 comments
Assignees
Labels
CI Dev defect The defect was found at the development stage. Did not have an impact on users. Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST

Comments

@BAStos525
Copy link
Contributor

BAStos525 commented Jul 8, 2022

Feature request

After merging this PR, Soramitsu DevOps team has to improve a couple of things in the iroha2 CI:

  • separate building of iroha2:base docker image into a dedicated workflow which runs on Dockerfile.ci change, and replace 7272721/i2-ci:latest in .container.image with hyperledger/iroha2:env
  • the CI task Build and push iroha2:dev docker image for iroha2:dev, iroha2:stable, iroha2:release images works only as bash command to build and push the corresponding docker image. For some reason, GH Action docker/build-push-action@v3 fails with following error:
    buildx failed with: error: failed to solve: failed to compute cache key: failed to calculate checksum of ref x8jvn8e81it8h6cbjf205gxgt::u6fgfckzgk3t5lxzms465fgd1: failed to walk /tmp/buildkit-mount878509508/release: lstat /tmp/buildkit-mount878509508/release: no such file or directory
    DevOps team could try to resolve this issue if it is possible.

Motivation

To make iroha2 CI faster, more effective and reliable.

Who can help?

@appetrosyan @s8sato

@BAStos525 BAStos525 added Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST labels Jul 8, 2022
@BAStos525 BAStos525 changed the title [suggestion] improve CI for iroha2 [suggestion] [ci] improve iroha2 build Jul 8, 2022
@appetrosyan
Copy link
Contributor

A few corrections, we should never push a non-iroha image into a separate tag, instead we fought hard and won an image called iroha2-ci which we can use to pin the version of Iroha to the container that was used to test and build it (so LTS has its own, while the normal image has a separate image).

@BAStos525
Copy link
Contributor Author

Also we have to decide what to do with rust-toolchain.toml file according to this comment

@s8sato
Copy link
Contributor

s8sato commented Jul 22, 2022

Check if the coverage behaves as expected after #2433 @s8sato

@s8sato
Copy link
Contributor

s8sato commented Jul 22, 2022

@s8sato
Copy link
Contributor

s8sato commented Jul 22, 2022

@BAStos525
Copy link
Contributor Author

BAStos525 commented Jul 24, 2022

image

https://github.com/hyperledger/iroha/runs/7463816572

dawidd6/action-download-artifact@v2.21.1 action extracts an artifact as it is named in mold --run cargo llvm-cov... test:

==> Extracting: lcov-0c3b217c27f218ae50f96606e4c1d9395fa48d03.info.zip
    inflating: lcov.info

ls la:
-rw-rw-rw- 1 root root 11238732 Jul 24 14:51 lcov.info
This a reason why we can not rename it:

Run mv lcov-0c3b217c27f218ae50f96606e4c1d9395fa48d03.info lcov.info
mv: cannot stat 'lcov-0c3b217c27f218ae50f9[6](https://github.com/BAStos525/soramitsu-iroha/runs/7488733522?check_suite_focus=true#step:6:7)606e4c1d9395fa48d03.info': No such file or directory

There are a two ways to avoid it:

  1. Just to remove the renaming action from I2::Dev::Deploy workflow.
  2. Generate a lcov report wtih PR head SHA variable:
iroha2-dev-pr.yaml
  tests_with_coverage:
    runs-on: ubuntu-latest
    container:
      image: 7272721/i2-ci:latest
    steps:
      - uses: actions/checkout@v2

      # TODO Remove these steps #2165
      - uses: actions-rs/toolchain@v1
        with:
          toolchain: stable
          override: true
          components: llvm-tools-preview
      - uses: taiki-e/install-action@cargo-llvm-cov
      - uses: Swatinem/rust-cache@v1
      - name: Run tests and generate lcov coverage report
        run: mold --run cargo llvm-cov --all-features --workspace --lcov --output-path lcov-${{ github.event.pull_request.head.sha }}.info
      - name: Upload to codecov
        uses: codecov/codecov-action@v3
        with:
          file: lcov-${{ github.event.pull_request.head.sha }}.info
          fail_ci_if_error: true
      - name: Upload codecov report artifact
        uses: actions/upload-artifact@v3
        with:
          name: lcov-${{ github.event.pull_request.head.sha }}.info
          path: lcov-${{ github.event.pull_request.head.sha }}.info
iroha2-dev.yaml
  upload_coverage:
    if: github.event.pull_request.merged
    runs-on: ubuntu-latest
    container:
      image: 7272721/i2-ci:latest
    steps:
      - uses: actions/checkout@v2
      - name: Download codecov report artifact
        uses: dawidd6/action-download-artifact@v2.21.1
        with:
          workflow: iroha2-dev-pr.yml
          name: lcov-${{ github.event.pull_request.head.sha }}.info
          search_artifacts: true
      - name: Upload to codecov
        uses: codecov/codecov-action@v3
        with:
          files: lcov-${{ github.event.pull_request.head.sha }}.info

@BAStos525
Copy link
Contributor Author

image

https://github.com/hyperledger/iroha/runs/7463817017

This is weird. I suppose that there is not a problem in secrets due iroha1 uses the same credentials for dockerhub account, and there are no an errors from iroha1 pushing images. May be try to come back to docker/login-action@v1?

@BAStos525
Copy link
Contributor Author

It seems like print-telemetry job in I2::Dev::Deploy workflow has been broken with new ci-image. We see only |null|null|null| content as output of this job.

s8sato pushed a commit that referenced this issue Aug 10, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
@s8sato
Copy link
Contributor

s8sato commented Aug 24, 2022

Closed as main topics were resolved by #2576.
Remaining topics within this issue are #2646, #2606, and the toolchain file.
If I'm missing anything please newly create issues @BAStos525 @appetrosyan

@s8sato s8sato closed this as completed Aug 24, 2022
mversic pushed a commit to mversic/iroha that referenced this issue Sep 6, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
mversic pushed a commit to mversic/iroha that referenced this issue Sep 6, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
mversic pushed a commit to mversic/iroha that referenced this issue Sep 6, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
mversic pushed a commit to mversic/iroha that referenced this issue Sep 7, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
BAStos525 added a commit to mversic/iroha that referenced this issue Sep 7, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
Signed-off-by: BAStos525 <jungle.vas@yandex.ru>
mversic pushed a commit to mversic/iroha that referenced this issue Sep 8, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
mversic pushed a commit to mversic/iroha that referenced this issue Sep 9, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
mversic pushed a commit to mversic/iroha that referenced this issue Sep 9, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
BAStos525 added a commit to mversic/iroha that referenced this issue Sep 9, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
Signed-off-by: BAStos525 <jungle.vas@yandex.ru>
mversic pushed a commit to mversic/iroha that referenced this issue Sep 9, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
BAStos525 added a commit to mversic/iroha that referenced this issue Sep 9, 2022
Signed-off-by: BAStos525 jungle.vas@yandex.ru
Co-authored-by: Ekaterina Mekhnetsova <mekkatya@gmail.com>
Co-authored-by: 6r1d <vic.6r1d@gmail.com>
Signed-off-by: BAStos525 <jungle.vas@yandex.ru>
@AlexStroke AlexStroke added the Dev defect The defect was found at the development stage. Did not have an impact on users. label Nov 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Dev defect The defect was found at the development stage. Did not have an impact on users. Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST
Projects
None yet
Development

No branches or pull requests

4 participants