-
Notifications
You must be signed in to change notification settings - Fork 208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bulletproofs (rangeproofs only) #23
Bulletproofs (rangeproofs only) #23
Conversation
include/secp256k1_commitment.h
Outdated
* If you need to convert to a format suitable for storage or transmission, use | ||
* secp256k1_pedersen_commitment_serialize and secp256k1_pedersen_commitment_parse. | ||
* | ||
* Furthermore, it is guaranteed to identical signatures will have identical |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(nit) s/to/two/
Discussions about ciphers can be annoying... Is there a specific reason to use chacha20? It's just used to derive randomness, why not rely on SHA256? I don't think that performance is a concern here. If performance is a concern, then I guess that AES is the canonical choice because of its hardware implementations (and constant-time software implementations are pretty good too).
I know @sipa has looked into these things, so I could imagine that all of this has been discussed extensively and I'm just late to the party.
It's also interesting for dicemix, which requires a PRG (or better a PRF) and needs much more randomness, each party needs O(|m| * n^2) bytes for n parties. So the performance of the PRF could be worth to think about and it would be preferable to use something which is already in libsecp256k1 of course.
Am 26. Mai 2018 00:22:25 MESZ schrieb Andrew Poelstra <notifications@github.com>:
…This is #16 without any of the arithmetic circuit stuff.
At this point there are probably still obvious code cleanliness things
to be dealt with, but I think it's ready for review. I'd like to get
this merged so that we can get the rangeproofs at least into Elements
soon.
cc @sipa @real-or-random @jonasnick can a couple of you review this?
I'm happy to split it into multple PRs if that makes sense.
You can view, comment on, or merge this pull request online at:
#23
-- Commit Summary --
* generator: add constant G and H generators
* add chacha20 function
* split Pedersen commitments out into their own module
* commitment: allow setting the blinding factor generator for Pedersen
commitments
* bulletproofs: add module, full support for rangeproofs
* bulletproofs: add benchmark
* bulletproofs: add rangeproof rewinding capability
-- File Changes --
M .gitignore (3)
M Makefile.am (8)
M configure.ac (50)
A include/secp256k1_bulletproofs.h (185)
A include/secp256k1_commitment.h (167)
M include/secp256k1_generator.h (6)
M include/secp256k1_rangeproof.h (157)
A src/bench_bulletproof.c (337)
M src/bench_rangeproof.c (9)
A src/modules/bulletproofs/Makefile.am.include (12)
A src/modules/bulletproofs/inner_product_impl.h (786)
A src/modules/bulletproofs/main_impl.h (249)
A src/modules/bulletproofs/rangeproof_impl.h (792)
A src/modules/bulletproofs/tests_impl.h (616)
A src/modules/bulletproofs/util.h (116)
A src/modules/commitment/Makefile.am.include (4)
A src/modules/commitment/main_impl.h (186)
A src/modules/commitment/pedersen_impl.h (38)
A src/modules/commitment/tests_impl.h (230)
M src/modules/generator/main_impl.h (26)
M src/modules/rangeproof/Makefile.am.include (2)
M src/modules/rangeproof/main_impl.h (187)
D src/modules/rangeproof/pedersen.h (22)
D src/modules/rangeproof/pedersen_impl.h (51)
M src/modules/rangeproof/rangeproof_impl.h (12)
M src/modules/rangeproof/tests_impl.h (318)
M src/scalar.h (3)
M src/scalar_4x64_impl.h (91)
M src/scalar_8x32_impl.h (100)
M src/scalar_low_impl.h (5)
M src/secp256k1.c (18)
M src/tests.c (70)
-- Patch Links --
https://github.com/ElementsProject/secp256k1-zkp/pull/23.patch
https://github.com/ElementsProject/secp256k1-zkp/pull/23.diff
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#23
|
It was performance motivated. The prover derives 64 * n_gates bytes of randomness, so there was a measurable performance improvement from using chacha. In the rewinding failure case, deriving randomness is a nontrivial portion of the computation (the whole thing is only a couple microseconds and a SHA2 compression is ~300 ns, so that's 10%). Interesting observation about AES hardware. In general we haven't done any hardware-specific optimizations in -zkp, though we have assembler implementations of some operations upstream. Another motivation was to keep things simple, and chacha is (a) dead simple and (b) we already had code for it laying around in multiple places. |
Okay I see, that makes sense. ChaCha20's software performance is certainly excellent and the way to go if hardware optimizations are not a goal, and they're indeed a doubly-edged sword: If you have AES-NI, AES is super fast and well, if you don't have it, then ChaCha20 is much faster. So ChaCha20 is a choice that does not prefer peers with a specific architecture [1]. And I think that's the right way to go in systems where everybody should be able to participate. (That's by the way interesting for online protocols such as DiceMix. Everybody needs to wait for the slowest peer every round, so if you want to optimize for a lower wall-clock time, you can even make the argument that it makes sense to optimize for slower machines. Though I don't think that matters a lot for protocols where bandwidth is the bottleneck and not computation. But that's another story.) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
More next week. :)
* import hashlib | ||
* C = EllipticCurve ([F (0), F (7)]) | ||
* G_bytes = '0479be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798483ada7726a3c4655da4fbfc0e1108a8fd17b448a68554199c47d08ffb10d4b8'.decode('hex') | ||
* H = C.lift_x(int(hashlib.sha256(G_bytes).hexdigest(),16)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need H = C.lift_x(F(int(hashlib.sha256(G_bytes).hexdigest(),16)))
here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting, this must be a new change in sage. Thanks for the fix!
x10 = LE32(seed32[6]); | ||
x11 = LE32(seed32[7]); | ||
x12 = idx; | ||
x13 = idx >> 32; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This right shift has undefined behavior if idx
/size_t
has 32 bits (or fewer). Same below.
(I guess this is rather a concern in the other file.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is still open.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
idx
is now a uint64_t
.
src/scalar_4x64_impl.h
Outdated
#define LE32(p) (p) | ||
#endif | ||
|
||
static void secp256k1_scalar_chacha20(secp256k1_scalar *r1, secp256k1_scalar *r2, const unsigned char *seed, size_t idx) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is related to the comment below: size_t idx
is somewhat strange. The code below is intended to handle foridx
values that are longer than 32 bits but then size_t
is not the right type to do this consistently on every platform, i.e., the 32-bit and 64-bit machines will behave differently (even if idx >> 32 == 0
on a 32-bit machine).
My suggestion is to use uint32_t
here and set x13 = 0
(and accept that we won't need more than 256 GiB of randomness. Another way is simply to use uint64_t
.
|
||
static void secp256k1_scalar_chacha20(secp256k1_scalar *r1, secp256k1_scalar *r2, const unsigned char *seed, size_t idx) { | ||
size_t n; | ||
size_t over_count = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wouldn't use size_t
for the two counter variables either (because the values do not express memory sizes). But in this case this is probably just personal taste. ;)
Does not compile for me.
|
dfeff0d
to
4ac6b21
Compare
Derp, I forgot to commit the circuit-removal from the benchmarks. Fixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I know all of this is just moved code but here are a few nits... Feel free to ignore.
include/secp256k1_commitment.h
Outdated
/** Initialize a context for usage with Pedersen commitments. */ | ||
void secp256k1_pedersen_context_initialize(secp256k1_context* ctx); | ||
|
||
/** Generate a pedersen commitment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/pedersen/Pedersen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll add a commit that fixes these typos and renames some things. I'll leave any re-ordering of arguments to future PRs because that'll break Elements.
include/secp256k1_commitment.h
Outdated
* In: ctx: pointer to a context object (cannot be NULL) | ||
* blinds: pointer to pointers to 32-byte character arrays for blinding factors. (cannot be NULL) | ||
* n: number of factors pointed to by blinds. | ||
* npositive: how many of the initial factors should be treated with a positive sign. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
indentation of the second column
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found initial
initially confusing here. ;)
include/secp256k1_commitment.h
Outdated
size_t npositive | ||
) SECP256K1_ARG_NONNULL(1) SECP256K1_ARG_NONNULL(2) SECP256K1_ARG_NONNULL(3); | ||
|
||
/** Verify a tally of pedersen commitments |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pedersen
include/secp256k1_commitment.h
Outdated
* In: ctx: pointer to a context object (cannot be NULL) | ||
* commits: pointer to array of pointers to the commitments. (cannot be NULL if pcnt is non-zero) | ||
* pcnt: number of commitments pointed to by commits. | ||
* ncommits: pointer to array of pointers to the negative commitments. (cannot be NULL if ncnt is non-zero) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the above function, n
was the prefix for "number", so I expected ncommits
to be the number of commitments, but this function apparently uses a cnt
suffix.
* generator_blind: array of asset blinding factors, `r` in the above paragraph | ||
* May not be NULL unless `n_total` is 0. | ||
* n_total: Total size of the above arrays | ||
* n_inputs: How many of the initial array elements represent commitments that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here the prefix is n_
even.
* generator_blind: array of asset blinding factors, `r` in the above paragraph | ||
* May not be NULL unless `n_total` is 0. | ||
* n_total: Total size of the above arrays | ||
* n_inputs: How many of the initial array elements represent commitments that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In secp256k1_pedersen_commit
, we had the positive summands first, then the negative ones. Maybe this makes sense like this, I don't know.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather not move arguments around unless there's a strong argument, as they're in use already.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to read the callback function in innerproduct_impl but I got lost in too many variables. The long comment is very nice but it's kind of hard to map the symbols in the comment to actual variable names, and all the cached/helper values add to my confusion.
I think a few comments to explain variables and to explain how some code lines relate to the verification formula would be helpful for further review.
secp256k1_scalar_set_b32(sec, data, NULL); | ||
memset(data, 0, 32); | ||
} | ||
/* sec * G + value * G2. */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/G2/H
include/secp256k1_bulletproofs.h
Outdated
#define SECP256K1_BULLETPROOF_CIRCUIT_VERSION 1 | ||
|
||
/* Maximum depth of 31 lets us validate an aggregate of 2^25 64-bit proofs */ | ||
#define SECP256K1_BULLETPROOF_MAX_DEPTH 60 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
31 or 60?
include/secp256k1_bulletproofs.h
Outdated
* Args: ctx: pointer to a context object (cannot be NULL) | ||
* In: blinding_gen: generator that blinding factors will be multiplied by (cannot be NULL) | ||
* n: number of NUMS generators to produce | ||
* precomp_n: for each NUMS generator, plus the blinding factor generator, how many multiples to precompute |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
n_precomp?
secp256k1_fe_normalize(&pointx); | ||
secp256k1_fe_get_b32(&out[bitveclen + i*32], &pointx); | ||
if (!secp256k1_fe_is_quad_var(&pt[i].y)) { | ||
out[i/8] |= (1ull << (i % 8)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ull
seems overkill
* | ||
*/ | ||
|
||
/* For the G and H generators, we choose the ith generator with a scalar computed from the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
New comment intentional? You may want to wrap formulas in ``.
unsigned char tmp[32] = { 0 }; | ||
secp256k1_generator gen; | ||
secp256k1_rfc6979_hmac_sha256_generate(&rng, tmp, 32); | ||
CHECK(secp256k1_generator_generate(ctx, &gen, tmp)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's certainly okay to bail out here but usually we handle (computationally) unreachable code more gracefully.
size_t n; | ||
secp256k1_ge *gens; | ||
secp256k1_ge *blinding_gen; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't secp256k1_bulletproof_generators
store precomp_n
too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I should just drop the precomp stuff for this PR since it isn't actually used (I had hacked it in when testing Jonas' multiexp code but never implemented it in an acceptable way).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dropped precomp.
size_t n; | ||
secp256k1_ge *gens; | ||
secp256k1_ge *blinding_gen; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An explanation of what is stored at what array index would have helped me a lot, including a comment that says that blinding_gen
points into gens[]
src/modules/bulletproofs/main_impl.h
Outdated
#include "modules/bulletproofs/rangeproof_impl.h" | ||
#include "modules/bulletproofs/util.h" | ||
|
||
secp256k1_bulletproof_generators *secp256k1_bulletproof_generators_create(const secp256k1_context *ctx, const secp256k1_generator *blinding_gen, size_t n, size_t precomp_n) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/secp256k1_bulletproof_generators_create/secp256k1_bulletproof_generators_generate
would be slightly more consistent with the naming in the generators module
size_t nbits, | ||
const secp256k1_generator* value_gen, | ||
const unsigned char* const* extra_commit, | ||
size_t *extra_commit_len |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const size_t *extra_commit_len
@@ -6,6 +6,9 @@ bench_verify | |||
bench_schnorr_verify | |||
bench_recover | |||
bench_internal | |||
bench_generator |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since you're adding bench_generator', you could also add the missing
bench_whitelist`
4706f75
to
8822988
Compare
Any news on this making it's way into Elements? Looking forward to it |
@real-or-random I replaced all the comments in the inner-product verification callback function. It should be possible to follow now. |
ec203a8
to
d1e9c44
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some preliminary comments with looking towards usage in Elements.
* Returns a list of generators, or NULL if allocation failed. | ||
* Args: ctx: pointer to a context object (cannot be NULL) | ||
* In: blinding_gen: generator that blinding factors will be multiplied by (cannot be NULL) | ||
* n: number of NUMS generators to produce |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any realistic upper-limit as per above constants, or can we go all the way up size_t
modulo memory?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really any upper limit. They take a while to generate, on the order of 1000/sec.
* generator_blind: array of asset blinding factors, `r` in the above paragraph | ||
* May not be NULL unless `n_total` is 0. | ||
* n_total: Total size of the above arrays | ||
* n_inputs: How many of the initial array elements represent commitments that |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd rather not move arguments around unless there's a strong argument, as they're in use already.
/** Standard secp256k1 generator G */ | ||
extern const secp256k1_generator secp256k1_generator_const_g; | ||
|
||
/** Alternate secp256k1 generator from Elements Alpha */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We really don't need to support Elements Alpha.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's useful to have a standard second generator though for non-CA applications.
d1e9c44
to
6290992
Compare
@instagibbs fixed your comments |
6290992
to
2d48ea5
Compare
We haven't been able to get Also noticing that none of the tests for |
…commitments We now use ecmult_const rather than ecmult_gen, which will slow down the generation of Pedersen commitments. However as far as I'm aware, this is never the bottleneck in proof generation.
1fab8eb
to
a1fc838
Compare
Rebased on #35 |
a1fc838
to
8e93c4c
Compare
8e93c4c
to
6fb7e05
Compare
As pointed out by drexl in IRC, we should make sure to read https://grin-tech.org/audits/jpa-audit-report.html and mimblewimble/secp256k1-zkp#37 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the missing deallocate is the only thing that applies to this PR (but someone else should have a second look).
int overflow; | ||
secp256k1_gej commitj; | ||
secp256k1_scalar_set_b32(&blinds[i], blind[i], &overflow); | ||
if (overflow || secp256k1_scalar_is_zero(&blinds[i])) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (overflow || secp256k1_scalar_is_zero(&blinds[i])) { | |
if (overflow || secp256k1_scalar_is_zero(&blinds[i])) { | |
secp256k1_scratch_deallocate_frame(scratch); |
cc @veorq, just for reference |
/* Correct for blinding factors and do the commitments */ | ||
CHECK(secp256k1_pedersen_blind_generator_blind_sum(ctx, value, (const unsigned char * const *) generator_blind, pedersen_blind, n_generators, n_inputs)); | ||
for (i = 0; i < n_generators; i++) { | ||
CHECK(secp256k1_pedersen_commit(ctx, &commit[i], pedersen_blind[i], value[i], &generator[i], &secp256k1_generator_const_h)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The last parameter should be secp256k1_generator_const_g
instead of secp256k1_generator_const_h
.
Related PR in Grin: mimblewimble/secp256k1-zkp#44
Fix fuzztarget ECDH to be symmetric
@apoelstra any chances of having this rebased and getting the great work and effort be finally merged? |
@dr-orlovsky yes, very good chance in the next week or two :) |
Closing this. Will retry in a more staged fashion. |
Closing in favor of #108. Note that the new proofs are not remotely compatible with the old ones. |
This is #16 without any of the arithmetic circuit stuff.
At this point there are probably still obvious code cleanliness things to be dealt with, but I think it's ready for review. I'd like to get this merged so that we can get the rangeproofs at least into Elements soon.
cc @sipa @real-or-random @jonasnick can a couple of you review this? I'm happy to split it into multple PRs if that makes sense.