Skip to content

Commit

Permalink
Fix S3InputStream's handling of large skips
Browse files Browse the repository at this point in the history
When the skip(n) method is called the MAX_SKIP_BYTES check is skipped,
resulting in the call potentially blocking for a long time.

Instead of delegating to the underlying stream, set the nextReadPosition
value. This allows the next read to decide if it is best to keep the existing
s3 object stream or open a new one.

This behavior matches the implementations for Azure and GCS.
  • Loading branch information
alexjo2144 authored and wendigo committed Dec 19, 2024
1 parent 7f0fa94 commit 6c52253
Showing 1 changed file with 4 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import java.io.IOException;
import java.io.InterruptedIOException;

import static java.lang.Math.clamp;
import static java.lang.Math.max;
import static java.util.Objects.requireNonNull;

Expand Down Expand Up @@ -126,14 +127,10 @@ public long skip(long n)
throws IOException
{
ensureOpen();
seekStream(false);

return reconnectStreamIfNecessary(() -> {
long skip = doSkip(n);
streamPosition += skip;
nextReadPosition += skip;
return skip;
});
long skipSize = clamp(n, 0, length != null ? length - nextReadPosition : Integer.MAX_VALUE);
nextReadPosition += skipSize;
return skipSize;
}

@Override
Expand Down

0 comments on commit 6c52253

Please sign in to comment.