Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why is chunkSize set in this way? #160

Open
luluzhou1 opened this issue May 6, 2024 · 0 comments
Open

Why is chunkSize set in this way? #160

luluzhou1 opened this issue May 6, 2024 · 0 comments

Comments

@luluzhou1
Copy link

luluzhou1 commented May 6, 2024

Issue
In src/engine_multiexp.js line 127-137,

const opPromises = [];
        for (let i=0; i<nPoints; i += chunkSize) {
            if (logger) logger.debug(`Multiexp start: ${logText}: ${i}/${nPoints}`);
            const n= Math.min(nPoints - i, chunkSize);
            const buffBasesChunk = buffBases.slice(i*sGIn, (i+n)*sGIn);
            const buffScalarsChunk = buffScalars.slice(i*sScalar, (i+n)*sScalar);
            opPromises.push(_multiExpChunk(buffBasesChunk, buffScalarsChunk, inType, logger, logText).then( (r) => {
                if (logger) logger.debug(`Multiexp end: ${logText}: ${i}/${nPoints}`);
                return r;
            }));
        }

If I understand correctly, the purpose here is to execute the _multiExpChunk function in parallel. However, I cannot understand why chunkSize is set like this: chunkSize = Math.floor(nPoints / (tm.concurrency /nChunks)); (line 122).
For instance, in one proof generation where tm.concurrency = 10, nChunks = 18, and nPoints = 1020160, the resulting chunkSize is 1836288, which is greater than nPoints, leading to opPromises containing only one element.

Possible solution:
change line 122 to
chunkSize = Math.floor(nPoints / (tm.concurrency * nChunks));
After the change, the time spent on Multiexp has been reduced to approximately 80% of the original runtime.
I'm not entirely sure if this is the best solution, as I do not fully understand the meaning of nChunks. Any explanation of the meanings of pTSizes, bitChunkSize, and nChunks would be greatly appreciated.

@luluzhou1 luluzhou1 changed the title Calculation of chunkSize Results in Suboptimal Parallelism in Multiexp Engine? Why is chunkSize set in this way? May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant