Skip to content
This repository has been archived by the owner on Jul 5, 2021. It is now read-only.

Parallel upload of multipart file #59

Open
imonarock opened this issue Nov 17, 2017 · 1 comment
Open

Parallel upload of multipart file #59

imonarock opened this issue Nov 17, 2017 · 1 comment
Milestone

Comments

@imonarock
Copy link

Is it possible to upload the parts in parallel with your uploader?

@MoriTanosuke
Copy link
Owner

No, the upload is single threaded at the moment. Multi thread would require to rewrite this part:

while (!completed && tries < 5) {
tries++;
try (InputStream inputSubStream = newInputSubstream(file, currentPosition, length)) {
inputSubStream.mark(-1);
checksum = TreeHashGenerator.calculateTreeHash(inputSubStream);
byte[] binaryChecksum = BinaryUtils.fromHex(checksum);
inputSubStream.reset();
String range = "bytes " + currentPosition + "-" + (currentPosition + length - 1) + "/*";
UploadMultipartPartRequest req = new UploadMultipartPartRequest()
.withChecksum(checksum)
.withBody(inputSubStream)
.withRange(range)
.withUploadId(uploadId)
.withVaultName(vaultName);
try {
UploadMultipartPartResult partResult = client.uploadMultipartPart(req);
log.info("Part {}/{} ({}) uploaded, checksum: {}, retries: {}", counter, total, range, partResult.getChecksum(), tries);
completed = true;
binaryChecksums.add(binaryChecksum);
} catch (Exception e) {
failedException = e;
}
}
}

@MoriTanosuke MoriTanosuke added this to the 1.0.0 milestone Apr 4, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants