Set up a smoke test pass that check if Substrate/Polkadot build using dist sccache #77
Conversation
Xanewok
force-pushed
the
igor-smoke-test-experiment
branch
from
850f6ed to
82c8836
Compare
Xanewok
changed the title
Xanewok
force-pushed
the
igor-smoke-test-experiment
branch
from
5aac069 to
27eac25
Compare
This hardcodes IPs for the time being as well as uses virtually no security for scheduler<->server and client<->scheduler but seems like a good starting point.
Xanewok
force-pushed
the
igor-smoke-test-experiment
branch
from
27eac25 to
2c5675c
Compare
| # A version of `Dockerfile.build.sccache-dist` that skips the build stage and | ||
| # instead copies over the locally pre-built `sccache-dist` binary. | ||
| # TODO: Switch to BuildKit with Docker/Compose support when available, which | ||
| # allows us to unify these into a single Dockerfile |
There was a problem hiding this comment.
My Docker-fu is not that great and maybe there's a way to share a single Dockerfile that allows to either vendor a binary from outside or build the project if needed, without resorting to Buildkit (IIUC Compose needs a very recent version that even supports that). @drahnr @gww-parity do you have an idea if we can deduplicate Dockerfile.{build,vendor}.sccache-dist somehow?
There was a problem hiding this comment.
The vendor one is just to save time in the CI while the build one is to provide a more self-contained example Dockerfile that can be used by others (technically vendor can be as well but the user needs to be on a compatible distro or know to compile targeting musl)
There was a problem hiding this comment.
The way to uniform those Dockerfiles (if that's your goal) could be to extract common parts into scripts, copy them into containers and execute. Something alnog the lines
COPY ./common_ubuntu18_setup.sh /common_ubuntu18_setup.sh
RUN ./common_ubuntu18_setup.sh
COPY ./following_steps_for_X.sh
RUN ./following_steps_for_X.sh
|
I talked with @gww-parity and we agreed to move this forward and not separate the Docker-related scripts for now |
Builds on #51 (merged) and #76 (included, closed in favor of this).
Fixes #52.
For now this builds sccache locally, spins up a local sccache dist cluster using the Compose file and runs a simple
cargo checkon the target repository.The sole reason for this is to uncover any discrepancies between the local sccache-enabled builds and the distributed ones, since the sandboxing logic can unearth some bugs that wholly prevent us from building in the dist scenario (like not including files which are required by the compilation but not included in the
rustc-emitted dep-info file, see #51).This is implemented as a concurrentThat's done now.smokejob for simplicity. We should probably set it up only after thebuildjob, download the createdmusl-enabled binaries there (TODO: Set up a cache) to save time and only then do a fan-out to test that X different repositories build successfully using the sccache in dist mode.