Added a soft reference based shared cache for S3 reads

[malhotrashivam]

This PR adds a soft-reference based cache for recently fetched fragments from S3 for faster lookup.
This cache would be especially useful for smaller files which have fewer fragments and can fit in the cache.

Documentation:

• Removed the config parameter S3.maxFragmentSize since we won't use a buffer pool internally anymore.
• Updated defaults for following S3 specific instructions: maxConcurrentRequests, readAheadCount, fragmentSize, maxCacheSize.

[devinrsmith]

It's hard for me to evaluate this in a reasonable amount of time. I worry we're adding complexity and it's not obvious to me that things are correct. I'm posting this now, but I estimate I would need at least another half a day to go through it all.

[devinrsmith]

I'm trying to figure out what the advantage of acquire/release is as opposed to relying on java reachability to Request. Should we instead have a SoftReference<Request> structure somewhere?

[malhotrashivam]

The main challenge we were facing was to control when to cancel the request. This uses CleanupReference for that and the request holds a soft reference to the corresponding ByteBuffer. Once the ByteBuffer goes out of scope, the request can be cleared. This new acquire/release methodology was so that each context can control the lifecycle of the buffer and thus the corresponding request.

We have slightly changed the design now so that each context will hold {Request, ownershipToken}, where internally the ownershipToken will be the Buffer only.
So now the reachability of the buffer will directly control the lifecycle of the Request. I might not be able to explain the new design in words but the code is much cleaner and should be easier to follow.

[rcaudy]

.

[rcaudy]

When this fails, we can avoid get.

[rcaudy]

You found this more complicated. I still prefer to avoid an extra lookup. Maybe moot anyway, since we can't seem to trust replace right now.

        Request existingRequest = requests.get(key);
        while (true) {
            if (existingRequest != null) {
                final Request.AcquiredRequest acquired = existingRequest.tryAcquire();
                if (acquired != null) {
                    return acquired;
                }
            }
            if (newAcquiredRequest == null) {
                newAcquiredRequest = Request.createAndAcquire(fragmentIndex, context);
            }
            final boolean added;
            if (existingRequest == null) {
                added = (existingRequest = requests.putIfAbsent(key, newAcquiredRequest.request)) == null;
            } else {
                if (!(added = requests.replace(key, existingRequest, newAcquiredRequest.request))) {
                    existingRequest = requests.get(key);
                }
            }

[malhotrashivam]

Reverted back to remove + putIfAbsent pattern and added a TODO for this with issue #5486.

[chipkent]

Python LGTM

[rcaudy]

I can approve like this, but I had one suggestion/question.

[deephaven-internal]

Labels indicate documentation is required. Issues for documentation have been opened:

Community: deephaven/deephaven-docs-community#211