CI: Remove unnecessary pipes #9681

5chdn · 2018-10-01T11:36:25Z

remove snap and arm targets (except android)
~~remove docker (except hub)~~ moved them to scripts/docker
remove gitlab release (never worked)
remove ever failing pipes (linting, kcov)
move optional pipelines to a "misc" staget that only runs if linux stuff is green ✔️
add cargo audit job, closes paritytech/devops/issues/216
simplify caching and add global cargo target dir environment variable
remove unused docs from readme

pull request pipeline: https://gitlab.parity.io/parity/parity-ethereum/pipelines/24333
full publish pipeline: https://gitlab.parity.io/parity/parity-ethereum/pipelines/24334

ordian · 2018-10-01T13:42:10Z

scripts/gitlab/audit.sh

+
+rustup default stable
+
+git submodule update --init --recursive


we don't really need submodules here

cargo +stable install cargo-audit cargo +stable audit

would be enough

Btw, is this job required or allowed to fail? Looks like you've removed all allow_failures.

Also, for cargo installs it'd nice to specify CARGO_TARGET_DIR to enable caching (see https://github.com/jtgeibel/cargo/blob/a81b1437de95920618119272c7a92bab9974ee30/src/etc/man/cargo-install.1#L45-L47).

I was under the impression we cache all jobs? see line 16 in .gitlab.yml

it is at the end of the pipeline and can happily fail :)

I was under the impression we cache all jobs? see line 16 in .gitlab.yml

Yes, but it looks like we specify CARGO_TARGET_DIR only in wasm-tests?

we default to ./target everywhere and that should be cached per job name, just double-checked.

ordian · 2018-10-01T13:51:13Z

docker/centos/Dockerfile

@@ -1,28 +0,0 @@
-FROM centos:latest


This is sad, since it was written by an external contributor (#9280).

yeah, we should put them somewhere else https://github.com/paritytech/scripts

General-Beck · 2018-10-01T15:13:46Z

snap packages removed???

gabreal

I haven't tested it in general looks good.

gabreal · 2018-10-01T14:55:45Z

.gitlab-ci.yml

+  stage:                           publish
+  only:
+    - nightly
+    - /^v2.*$/


releasable_branches (stable, beta, tags) are build but only these tags are published (except from linux docker).

yeah, that's desired, just the name "releasable branches" is misleading, 👍

gabreal · 2018-10-01T15:06:04Z

scripts/gitlab/coverage.sh

-echo "__________Cover RUST___________"
-for FILE in `find target/debug/deps ! -name "*.*" -type f`
-do
-  timeout --signal=SIGKILL 5m kcov --include-path=$(pwd) --exclude-path=$(pwd)/target $KCOV_FLAGS $KCOV_TARGET $FILE


no code test coverage anymore?

it's broken forever due to a rust bug

should we remove coverage badge from readme then?

gabreal · 2018-10-01T15:08:08Z

scripts/gitlab/publish-awss3.sh

+for DIR in "${filetest[@]}";
+do
+  cd $DIR
+  if [[ $DIR == "*windows*" ]];


I'm not so familiar with bash but musn't it be =~ for matching regex?

I did not author this code, happy to change it to =~

gabreal · 2018-10-01T15:09:16Z

scripts/gitlab/publish-awss3.sh

+    else
+      WIN="";
+  fi
+  for binary in $(ls parity.sha3)


My guess is it will only match the one file parity.sha3, so why loop?

I did not author this code, but it also puzzled me, also why binary?

gabreal · 2018-10-01T15:09:52Z

scripts/gitlab/publish-awss3.sh

+  fi
+  for binary in $(ls parity.sha3)
+  do
+    sha3=$(cat ${binary/sha3} | awk '{ print $1}' )


I don't understand: binary=parity.sha3, then I assume ${binary/sha3} gives "parity." which is a binary (actually without the dot) and does not contain the hash value.

I did not author this code; I did not understand it either.

5chdn · 2018-10-01T18:11:35Z

Ok, now this is ready to go

https://docs.gitlab.com/ee/ci/variables/#9-0-renaming

…um into a5-gitlab-light

* ci: reduce gitlab pipelines significantly * ci: build pipeline for PR * ci: remove dead weight * ci: remove github release script * ci: remove forever broken aura tests * ci: add random stuff to the end of the pipes * ci: add wind and mac to the end of the pipe * ci: remove snap artifacts * ci: (re)move dockerfiles * ci: clarify job names * ci: add cargo audit job * ci: make audit script executable * ci: ignore snap and docker files for rust check * ci: simplify audit script * ci: rename misc to optional * ci: add publish script to releaseable branches * ci: more verbose cp command for windows build * ci: fix weird binary checksum logic in push script * ci: fix regex in push script for windows * ci: simplify gitlab caching * docs: align README with ci changes * ci: specify default cargo target dir * ci: print verbose environment * ci: proper naming of scripts * ci: restore docker files * ci: use docker hub file * ci: use cargo home instead of cargo target dir * ci: touch random rust file to trigger real builds * ci: set cargo target dir for audit script * ci: remove temp file * ci: don't export the cargo target dir in the audit script * ci: fix windows unbound variable * docs: fix gitlab badge path * rename deprecated gitlab ci variables https://docs.gitlab.com/ee/ci/variables/#9-0-renaming * ci: fix git compare for nightly builds * test: skip c++ example for all platforms but linux * ci: add random rust file to trigger tests * ci: remove random rust file * disable cpp lib test for mac, win and beta (#9686)

…mon-deps * origin/master: CI: Remove unnecessary pipes (#9681) test.sh: use cargo --target for platforms other than linux, win or mac (#9650) ci: fix push script (#9679) Hardfork the testnets (#9562) Calculate sha3 instead of sha256 for push-release. (#9673) ethcore-io retries failed work steal (#9651) fix(light_fetch): avoid race with BlockNumber::Latest (#9665) Test fix for windows cache name... (#9658) refactor(fetch) : light use only one `DNS` thread (#9647)

* parity-version: bump beta to 2.1.2 * docs(rpc): push the branch along with tags (#9578) * docs(rpc): push the branch along with tags * ci: remove old rpc docs script * Remove snapcraft clean (#9585) * Revert " add snapcraft package image (master) (#9584)" This reverts commit ceaedbb. * Update package-snap.sh * Update .gitlab-ci.yml * ci: fix regex 🙄 (#9597) * docs(rpc): annotate tag with the provided message (#9601) * Update ropsten.json (#9602) * HF in POA Sokol (2018-09-19) (#9607) poanetwork/poa-chain-spec#86 * fix(network): don't disconnect reserved peers (#9608) The priority of && and || was borked. * fix failing node-table tests on mac os, closes #9632 (#9633) * ethcore-io retries failed work steal (#9651) * ethcore-io uses newer version of crossbeam && retries failed work steal * ethcore-io non-mio service uses newer crossbeam * remove master from releasable branches (#9655) * remove master from releasable branches need backporting in beta fix https://gitlab.parity.io/parity/parity-ethereum/-/jobs/101065 etc * add except for snap packages for master * Test fix for windows cache name... (#9658) * Test fix for windows cache name... * Fix variable name. * fix(light_fetch): avoid race with BlockNumber::Latest (#9665) * Calculate sha3 instead of sha256 for push-release. (#9673) * Calculate sha3 instead of sha256 for push-release. * Add pushes to the script. * Hardfork the testnets (#9562) * ethcore: propose hardfork block number 4230000 for ropsten * ethcore: propose hardfork block number 9000000 for kovan * ethcore: enable kip-4 and kip-6 on kovan * etcore: bump kovan hardfork to block 9.2M * ethcore: fix ropsten constantinople block number to 4.2M * ethcore: disable difficulty_test_ropsten until ethereum/tests are updated upstream * ci: fix push script (#9679) * ci: fix push script * Fix copying & running on windows. * CI: Remove unnecessary pipes (#9681) * ci: reduce gitlab pipelines significantly * ci: build pipeline for PR * ci: remove dead weight * ci: remove github release script * ci: remove forever broken aura tests * ci: add random stuff to the end of the pipes * ci: add wind and mac to the end of the pipe * ci: remove snap artifacts * ci: (re)move dockerfiles * ci: clarify job names * ci: add cargo audit job * ci: make audit script executable * ci: ignore snap and docker files for rust check * ci: simplify audit script * ci: rename misc to optional * ci: add publish script to releaseable branches * ci: more verbose cp command for windows build * ci: fix weird binary checksum logic in push script * ci: fix regex in push script for windows * ci: simplify gitlab caching * docs: align README with ci changes * ci: specify default cargo target dir * ci: print verbose environment * ci: proper naming of scripts * ci: restore docker files * ci: use docker hub file * ci: use cargo home instead of cargo target dir * ci: touch random rust file to trigger real builds * ci: set cargo target dir for audit script * ci: remove temp file * ci: don't export the cargo target dir in the audit script * ci: fix windows unbound variable * docs: fix gitlab badge path * rename deprecated gitlab ci variables https://docs.gitlab.com/ee/ci/variables/#9-0-renaming * ci: fix git compare for nightly builds * test: skip c++ example for all platforms but linux * ci: add random rust file to trigger tests * ci: remove random rust file * disable cpp lib test for mac, win and beta (#9686) * cleanup ci merge * ci: fix tests * fix bad-block reporting no reason (#9638) * ethcore: fix detection of major import (#9552) * sync: set state to idle after sync is completed * sync: refactor sync reset * Don't hash the init_code of CREATE. (#9688) * Docker: run as parity user (#9689) * Implement CREATE2 gas changes and fix some potential overflowing (#9694) * Implement CREATE2 gas changes and fix some potential overflowing * Ignore create2 state tests * Split CREATE and CREATE2 in gasometer * Generalize rounding (x + 31) / 32 to to_word_size * make instantSeal engine backwards compatible, closes #9696 (#9700) * ethcore: delay ropsten hardfork (#9704) * fix (light/provider) : Make `read_only executions` read-only (#9591) * `ExecutionsRequest` from light-clients as read-only This changes so all `ExecutionRequests` from light-clients are executed as read-only which the `virtual``flag == true ensures. This boost up the current transaction to always succeed Note, this only affects `eth_estimateGas` and `eth_call` AFAIK. * grumbles(revert renaming) : TransactionProof * grumbles(trace) : remove incorrect trace * grumbles(state/prove_tx) : explicit `virt` Remove the boolean flag to determine that a `state::prove_transaction` whether it should be executed in a virtual context or not. Because of that also rename the function to `state::prove_transction_virtual` to make more clear * CI: Skip docs job for nightly (#9693) * ci: force-tag wiki changes * ci: force-tag wiki changes * ci: skip docs job for master and nightly * ci: revert docs job checking for nightly tag * ci: exclude docs job from nightly builds in gitlab script

* ci: reduce gitlab pipelines significantly * ci: build pipeline for PR * ci: remove dead weight * ci: remove github release script * ci: remove forever broken aura tests * ci: add random stuff to the end of the pipes * ci: add wind and mac to the end of the pipe * ci: remove snap artifacts * ci: (re)move dockerfiles * ci: clarify job names * ci: add cargo audit job * ci: make audit script executable * ci: ignore snap and docker files for rust check * ci: simplify audit script * ci: rename misc to optional * ci: add publish script to releaseable branches * ci: more verbose cp command for windows build * ci: fix weird binary checksum logic in push script * ci: fix regex in push script for windows * ci: simplify gitlab caching * docs: align README with ci changes * ci: specify default cargo target dir * ci: print verbose environment * ci: proper naming of scripts * ci: restore docker files * ci: use docker hub file * ci: use cargo home instead of cargo target dir * ci: touch random rust file to trigger real builds * ci: set cargo target dir for audit script * ci: remove temp file * ci: don't export the cargo target dir in the audit script * ci: fix windows unbound variable * docs: fix gitlab badge path * rename deprecated gitlab ci variables https://docs.gitlab.com/ee/ci/variables/#9-0-renaming * ci: fix git compare for nightly builds * test: skip c++ example for all platforms but linux * ci: add random rust file to trigger tests * ci: remove random rust file * disable cpp lib test for mac, win and beta (#9686)

* parity-version: bump stable to 2.0.7 * HF in POA Sokol (2018-09-19) (#9607) poanetwork/poa-chain-spec#86 * fix failing node-table tests on mac os, closes #9632 (#9633) * fix(light_fetch): avoid race with BlockNumber::Latest (#9665) * CI: Remove unnecessary pipes (#9681) * ci: reduce gitlab pipelines significantly * ci: build pipeline for PR * ci: remove dead weight * ci: remove github release script * ci: remove forever broken aura tests * ci: add random stuff to the end of the pipes * ci: add wind and mac to the end of the pipe * ci: remove snap artifacts * ci: (re)move dockerfiles * ci: clarify job names * ci: add cargo audit job * ci: make audit script executable * ci: ignore snap and docker files for rust check * ci: simplify audit script * ci: rename misc to optional * ci: add publish script to releaseable branches * ci: more verbose cp command for windows build * ci: fix weird binary checksum logic in push script * ci: fix regex in push script for windows * ci: simplify gitlab caching * docs: align README with ci changes * ci: specify default cargo target dir * ci: print verbose environment * ci: proper naming of scripts * ci: restore docker files * ci: use docker hub file * ci: use cargo home instead of cargo target dir * ci: touch random rust file to trigger real builds * ci: set cargo target dir for audit script * ci: remove temp file * ci: don't export the cargo target dir in the audit script * ci: fix windows unbound variable * docs: fix gitlab badge path * rename deprecated gitlab ci variables https://docs.gitlab.com/ee/ci/variables/#9-0-renaming * ci: fix git compare for nightly builds * test: skip c++ example for all platforms but linux * ci: add random rust file to trigger tests * ci: remove random rust file * disable cpp lib test for mac, win and beta (#9686) * cleanup ci merge * parity: bump clib * ci: fix tests * ci: disable c++ example * Docker: run as parity user (#9689) * CI: Skip docs job for nightly (#9693) * ci: force-tag wiki changes * ci: force-tag wiki changes * ci: skip docs job for master and nightly * ci: revert docs job checking for nightly tag * ci: exclude docs job from nightly builds in gitlab script * fix (light/provider) : Make `read_only executions` read-only (#9591) * `ExecutionsRequest` from light-clients as read-only This changes so all `ExecutionRequests` from light-clients are executed as read-only which the `virtual``flag == true ensures. This boost up the current transaction to always succeed Note, this only affects `eth_estimateGas` and `eth_call` AFAIK. * grumbles(revert renaming) : TransactionProof * grumbles(trace) : remove incorrect trace * grumbles(state/prove_tx) : explicit `virt` Remove the boolean flag to determine that a `state::prove_transaction` whether it should be executed in a virtual context or not. Because of that also rename the function to `state::prove_transction_virtual` to make more clear * ethcore: fix detection of major import (#9552) * sync: set state to idle after sync is completed * sync: refactor sync reset * parity: revert clib bump and fix tests * Fix path to parity.h (#9274) * Fix path to parity.h * Fix other paths as well * ethcore-io retries failed work steal (#9651) * ethcore-io uses newer version of crossbeam && retries failed work steal * ethcore-io non-mio service uses newer crossbeam

5chdn added 2 commits October 1, 2018 12:33

ci: reduce gitlab pipelines significantly

aa34dfa

ci: build pipeline for PR

94cf19d

5chdn added A3-inprogress ⏳ Pull request is in progress. No review needed at this stage. B0-patchthis 🕷 M1-ci 🙉 Continuous integration. labels Oct 1, 2018

5chdn added this to the 2.2 milestone Oct 1, 2018

5chdn added 8 commits October 1, 2018 12:42

ci: remove dead weight

ff311d4

ci: remove github release script

ffadc7f

ci: remove forever broken aura tests

32c812a

ci: add random stuff to the end of the pipes

42f3100

ci: add wind and mac to the end of the pipe

99e8652

ci: remove snap artifacts

bc3c111

ci: (re)move dockerfiles

e18270f

ci: clarify job names

88b6169

5chdn added A0-pleasereview 🤓 Pull request needs code review. and removed A3-inprogress ⏳ Pull request is in progress. No review needed at this stage. labels Oct 1, 2018

5chdn requested review from gabreal, kirushik and General-Beck October 1, 2018 12:38

ci: add cargo audit job

244528e

5chdn requested a review from ordian October 1, 2018 13:03

5chdn added 2 commits October 1, 2018 14:41

ci: make audit script executable

ab9c371

ci: ignore snap and docker files for rust check

269d7c1

ordian reviewed Oct 1, 2018

View reviewed changes

5chdn added 2 commits October 1, 2018 14:58

ci: simplify audit script

f0f123c

ci: rename misc to optional

597bbfc

gabreal reviewed Oct 1, 2018

View reviewed changes

5chdn added 2 commits October 1, 2018 16:38

ci: add publish script to releaseable branches

d41e3bc

ci: more verbose cp command for windows build

8eb65a9

5chdn added A8-looksgood 🦄 Pull request is reviewed well. and removed A7-looksgoodtestsfail 🤖 Pull request is reviewed well, but cannot be merged due to tests failing. labels Oct 1, 2018

ci: don't export the cargo target dir in the audit script

d890ab3

5chdn mentioned this pull request Oct 1, 2018

Remove caching for osx tests. #9664

Closed

ci: fix windows unbound variable

a99c6a3

This was referenced Oct 1, 2018

Backports for beta 2.1.2 #9649

Merged

Backports for stable 2.0.7 #9648

Merged

docs: fix gitlab badge path

506842b

ordian added A9-buythatmanabeer 🍻 Pull request is reviewed well and worth buying the author a beer. and removed A8-looksgood 🦄 Pull request is reviewed well. labels Oct 1, 2018

gabreal and others added 7 commits October 1, 2018 20:16

rename deprecated gitlab ci variables

da0f3cc

https://docs.gitlab.com/ee/ci/variables/#9-0-renaming

ci: fix git compare for nightly builds

f749b0e

test: skip c++ example for all platforms but linux

0ababae

Merge branch 'a5-gitlab-light' of github.com:paritytech/parity-ethere…

b1f501a

…um into a5-gitlab-light

ci: add random rust file to trigger tests

a02990e

ci: remove random rust file

4783af1

disable cpp lib test for mac, win and beta (#9686)

c146243

5chdn merged commit 7781cbb into master Oct 1, 2018

5chdn deleted the a5-gitlab-light branch October 1, 2018 23:04

5chdn added the B0-patch-stable 🕷 Pull request should also be back-ported to the stable branch. label Oct 3, 2018

5chdn mentioned this pull request Oct 9, 2018

[stable] Backport constantinople changes #9723

Merged

16 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Remove unnecessary pipes #9681

CI: Remove unnecessary pipes #9681

5chdn commented Oct 1, 2018 •

edited by ordian

Loading

ordian Oct 1, 2018

ordian Oct 1, 2018

5chdn Oct 1, 2018

5chdn Oct 1, 2018

ordian Oct 1, 2018

5chdn Oct 1, 2018

ordian Oct 1, 2018

5chdn Oct 1, 2018

General-Beck commented Oct 1, 2018

gabreal left a comment

gabreal Oct 1, 2018

5chdn Oct 1, 2018

gabreal Oct 1, 2018

5chdn Oct 1, 2018

ordian Oct 1, 2018

5chdn Oct 1, 2018

gabreal Oct 1, 2018

5chdn Oct 1, 2018

gabreal Oct 1, 2018

5chdn Oct 1, 2018

gabreal Oct 1, 2018

5chdn Oct 1, 2018

5chdn commented Oct 1, 2018


		rustup default stable

		git submodule update --init --recursive

CI: Remove unnecessary pipes #9681

CI: Remove unnecessary pipes #9681

Conversation

5chdn commented Oct 1, 2018 • edited by ordian Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

General-Beck commented Oct 1, 2018

gabreal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

5chdn commented Oct 1, 2018

5chdn commented Oct 1, 2018 •

edited by ordian

Loading