Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Streaming contribution and fast contribution i/o #1188

Merged
merged 3 commits into from
Jun 29, 2020
Merged

Streaming contribution and fast contribution i/o #1188

merged 3 commits into from
Jun 29, 2020

Conversation

porcuquine
Copy link
Collaborator

@porcuquine porcuquine commented Jun 25, 2020 •
edited
Loading

This PR adds streaming contribution to phase2 and makes it the default. This caps memory usage. The somewhat arbitrarily chosen batch size seems to keep memory usage around 20MB without a performance penalty.

This PR also add support for 'raw i/o'. This means that the read/write operations in the streaming hot path avoid expensive transformations in and out of montgomery space. This makes contribution ~10x faster.

The expected net result of this PR is that contribution to even our large circuits should be possible in reasonable time and without extremely beefy hardware. (Actual times and RAM usage will need to be measured on real hardware, of course.)

Summary of benchmarks for contribution:

Before (non-streaming non-raw) contribution time: 51.62s
Before mem: 53460992 bytes

After (streaming raw) contribution time: 5.26s
After mem: 19374080

Note: these benchmarks don't reflect realistic RAM reduction for large circuits, because the small circuit being used only requires ~50MB RAM. The benefits of streaming will rise for large circuits, which otherwise currently require 55GB RAM. NOTE: streaming is not yet tested on large params, so the claimed cap is a reasoned expectation which still needs verification.

(base) ➜  rust-proofs git:(feat/streaming-contribution) ✗ /usr/bin/time -l cargo run --release --bin phase2 contribute-non-streaming winning_poseidon_2kib_b3cfbeb_1_small
a    Finished release [optimized] target(s) in 0.22s
     Running `target/release/phase2 contribute-non-streaming winning_poseidon_2kib_b3cfbeb_1_small`
[filecoin-proofs/src/bin/phase2.rs:687] "in contribute_to_params" = "in contribute_to_params"
03:26:46 [ INFO] contributing to params for circuit: Winning-Poseidon-2KiB-b3cfbeb-Small 1->2
✔ Please randomly press your keyboard (press Return/Enter when finished) · ********
03:26:47 [ INFO] reading small 'before' params as small: winning_poseidon_2kib_b3cfbeb_1_small
03:27:33 [ INFO] phase2::MPCSmall::read() read vector lengths: h=131071, l=91674, contributions=1
03:27:33 [ INFO] successfully read 'before' params, dt_read=46s
03:27:33 [ INFO] making contribution
03:27:33 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
03:27:36 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
03:27:36 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
03:27:38 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
03:27:38 [ INFO] successfully made contribution: 85b1e53599103a7aeb3aec92dcbfa73bf95e626700cb90e97e9ec738f9c1388628afe26e84023aac74055959b5b980ee440cda0ba737b8219544612c9ec05edd, dt_contribute=4s
03:27:38 [ INFO] writing small 'after' params to file: winning_poseidon_2kib_b3cfbeb_2_small
03:27:38 [ INFO] writing contribution hash to file: winning_poseidon_2kib_b3cfbeb_2_small.contrib
03:27:38 [ INFO] successfully made contribution, dt_total=50s
       51.62 real       117.36 user         0.32 sys
  53460992  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
     22487  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         0  messages sent
         0  messages received
         1  signals received
        36  voluntary context switches
     50437  involuntary context switches

After:

(base) ➜  rust-proofs git:(feat/streaming-contribution) ✗ /usr/bin/time -l cargo run --release --bin phase2 contribute winning_poseidon_2kib_b3cfbeb_1_small_raw
    Finished release [optimized] target(s) in 0.22s
     Running `target/release/phase2 contribute winning_poseidon_2kib_b3cfbeb_1_small_raw`
04:13:06 [ INFO] contributing to params for circuit: Winning-Poseidon-2KiB-b3cfbeb-Small 1->2
✔ Please randomly press your keyboard (press Return/Enter when finished) · ********
04:13:06 [ INFO] making contribution
04:13:06 [ INFO] making streamer from small 'before' params: winning_poseidon_2kib_b3cfbeb_1_small_raw
04:13:06 [ INFO] writing small 'after' params to file: winning_poseidon_2kib_b3cfbeb_2_small_raw
04:13:06 [ INFO] phase2::MPCParameters::contribute() beginning streaming h
04:13:06 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:07 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:08 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of h
04:13:09 [ INFO] phase2::MPCParameters::contribute() finished streaming h
04:13:09 [ INFO] phase2::MPCParameters::contribute() beginning streaming l
04:13:09 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:09 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:09 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:10 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() finished batch_exp of l
04:13:11 [ INFO] phase2::MPCParameters::contribute() finished streaming l
04:13:11 [ INFO] successfully made contribution: 77c8f42b050bfabe26c18efb361ae52d0f99f4bd52aac01539e1825e723daf6618f7ffd8e899c94e1b9fe81c604862a4357d6c4211da273a1a5beab0a1e01839, dt_contribute=4s
04:13:11 [ INFO] writing contribution hash to file: winning_poseidon_2kib_b3cfbeb_2_small_raw.contrib
04:13:11 [ INFO] successfully made contribution, dt_total=4s
        5.26 real        70.33 user         0.21 sys
  19374080  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
     16551  page reclaims
         0  page faults
         0  swaps
         0  block input operations
         0  block output operations
         0  messages sent
         0  messages received
         1  signals received
         7  voluntary context switches
     26203  involuntary context switches```

@porcuquine porcuquine changed the title Feat/streaming contribution Streaming contribution and fast contribution i/o Jun 25, 2020
@porcuquine porcuquine mentioned this pull request Jun 25, 2020
Closed
@porcuquine porcuquine force-pushed the feat/streaming-contribution branch 3 times, most recently from b432487 to 2c1d520 Compare June 26, 2020 01:10
@porcuquine porcuquine added the cryptocomputelab CryptoComputeLab work label Jun 26, 2020
@porcuquine porcuquine marked this pull request as ready for review June 26, 2020 05:05
@DrPeterVanNostrand
Copy link
Contributor

Whoo, those are nice benchmarks.

@porcuquine porcuquine mentioned this pull request Jun 26, 2020
Merged
@porcuquine porcuquine force-pushed the feat/streaming-contribution branch 2 times, most recently from 9be7d91 to b1c789b Compare June 26, 2020 23:39
@DrPeterVanNostrand
Copy link
Contributor

Just a few comments, mostly about logs.

@porcuquine porcuquine force-pushed the feat/streaming-contribution branch 2 times, most recently from 135dcbd to faba65f Compare June 28, 2020 21:21
@porcuquine
Copy link
Collaborator Author

@DrPeterVanNostrand I think I have addressed all of your comments in my latest commit. In some cases, the resolution was different than your suggestion, for reasons I have clarified in my responses above.

DrPeterVanNostrand
DrPeterVanNostrand reviewed Jun 29, 2020
View reviewed changes
filecoin-proofs/Cargo.toml Outdated
phase2 = { package = "phase21", version = "0.6.0" }
simplelog = "0.8.0"
phase2 = { version = "0.7.0", package = "phase21" }
simplelog = "0.7.4"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should accept the simplelog version bump that's currently in master.

@DrPeterVanNostrand
Copy link
Contributor

Everything looks good to me. Thank you for these changes!

@porcuquine porcuquine merged commit 4e4f766 into master Jun 29, 2020
@cryptonemo cryptonemo deleted the feat/streaming-contribution branch July 15, 2020 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cryptonemo cryptonemo cryptonemo approved these changes

@DrPeterVanNostrand DrPeterVanNostrand DrPeterVanNostrand left review comments

@dignifiedquire dignifiedquire Awaiting requested review from dignifiedquire

Assignees
No one assigned
Labels
cryptocomputelab CryptoComputeLab work
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@porcuquine @DrPeterVanNostrand @cryptonemo