Skip to content

Commit

Permalink
Add script to generate reproducibility index file
Browse files Browse the repository at this point in the history
Unfortunately, the artifacts are not actually reproducibly buildable
right now, so we cannot enforce this yet, but I'm checking in the
generated file that I get on my machine, which seems to differ between
my machine and GCP.

When we figure out the source of the discrepancy, then we can enforce
that the hashes are actually checked in when changes are made to the
source.

Ref project-oak#861
  • Loading branch information
tiziano88 committed Apr 17, 2020
1 parent 55a93d7 commit aff745a
Show file tree
Hide file tree
Showing 3 changed files with 52 additions and 0 deletions.
17 changes: 17 additions & 0 deletions cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,12 @@ steps:
timeout: 60m
entrypoint: 'bash'
args: ['./scripts/build_server', '-s', 'base']
- name: 'gcr.io/oak-ci/oak:latest'
id: build_server_rust
waitFor: ['bazel_init']
timeout: 60m
entrypoint: 'bash'
args: ['./scripts/build_server', '-s', 'rust']

# Package the Hello World application in a Docker image.
- name: 'gcr.io/oak-ci/oak:latest'
Expand Down Expand Up @@ -88,6 +94,17 @@ steps:
entrypoint: 'bash'
args: ['./scripts/run_clang_tidy']

# Rebuild the index file with hashes of reproducible artifacts. If this changed compared to the
# checked-in version, it will be detected by the `git_check_diff` step below.
# TODO(#861): Re-enable this step (before git_check_diff) when artifacts are actually reproducibly
# built.
# - name: 'gcr.io/oak-ci/oak:latest'
# id: build_reproducibility_index
# waitFor: ['run_examples', 'build_server_rust']
# timeout: 5m
# entrypoint: 'bash'
# args: ['./scripts/build_reproducibility_index']

# Check whether any of the previous steps resulted in file diffs that were not checked in or
# ignored by git.
- name: 'gcr.io/oak-ci/oak:latest'
Expand Down
10 changes: 10 additions & 0 deletions reproducibility_index
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
61fdf25b4e2b4171a579dd9e4716b24348fb3f9f90ad936f9c91046618c93ed5 ./target/wasm32-unknown-unknown/release/abitest_0_frontend.wasm
32a5f6aa9cbac2f71f40be10c52da32653a5b579810e5927af42df171a0f9dfc ./target/wasm32-unknown-unknown/release/abitest_1_backend.wasm
c697982360ea5d5a95b795889b4633ed27d3bc892f9bc9ac1df480c3759cb9b5 ./target/wasm32-unknown-unknown/release/aggregator.wasm
2f5b176dbe8c9269d6bc2253c82d627d556db8a7214de8133ebfa185075a08dc ./target/wasm32-unknown-unknown/release/chat.wasm
c9a7fe26f4ad22af68ec375946a212c0356f5d90827f22884df540008dfc30dc ./target/wasm32-unknown-unknown/release/hello_world.wasm
874c832f9e16cc78e393b73e98ec08ee0c6b475ed67298e4094d31eaaf07f98c ./target/wasm32-unknown-unknown/release/machine_learning.wasm
97906ca9165f63281b03647ec55203e38a7acd2207f9518de52098a4c7c81bde ./target/wasm32-unknown-unknown/release/running_average.wasm
c6f59836ce3f0b2206134ee93215da39f37658cdabd5e90fb40c216ea0f1d694 ./target/wasm32-unknown-unknown/release/translator.wasm
b2e0fe6e68f082564019b986128de56aad7eab08b791e3baacc66c784064dd04 ./bazel-clang-bin/oak/server/loader/oak_runner
6fdb4a4a2c4ec7dab0581dbb5a179c9f66b28834de32c170a4bba74cd46b72e2 ./target/debug/oak_loader
25 changes: 25 additions & 0 deletions scripts/build_reproducibility_index
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/bin/bash
#
# Check that artifacts can be reproducibly built, both when re-building them on the same machine
# over time, but also when built on different machines (e.g. locally vs on GCP).
#
# This script should be re-run every time any of these artifacts changes, and should be checked in
# so that we can enforce in CI that the hashes are indeed consistent.

readonly SCRIPTS_DIR="$(dirname "$(readlink -f "$0")")"
# shellcheck source=scripts/common
source "$SCRIPTS_DIR/common"

# List of artifacts that are expected to be reproducibly built.
readonly REPRODUCIBLE_ARTIFACTS=(
./target/wasm32-unknown-unknown/release/*.wasm
./bazel-clang-bin/oak/server/loader/oak_runner
./target/debug/oak_loader
)

# Index file containing hashes of the reproducible artifacts, alongside their file names.
readonly REPRODUCIBILITY_INDEX='./reproducibility_index'

# Generate the index file by computing the hashes of the artifacts.
# The index file must be checked in, and
sha256sum "${REPRODUCIBLE_ARTIFACTS[@]}" > "${REPRODUCIBILITY_INDEX}"

0 comments on commit aff745a

Please sign in to comment.