feat: Add shouldReturnRawInputStream option to Get requests

[JesseLovelace]

Adds support for shouldReturnRawInputStream, which will allow users to specify whether the client should auto-decompress gzipped content or not.

Note: read(StorageObject, Map<Option, ?>, long, OutputStream) and read(StorageObject, Map<Option, ?>, long, int) in HttpStorageRpc are set up to have different defaults to preserve their original behavior. Before, returnRawInputStream was always set to true, however read(StorageObject, Map<Option, ?>, long, OutputStream) had a bug that would cause that to be lost, so it would always turn out to be false (this is why blob.downloadTo and BlobReadChannel.read had different behaviors). That bug is fixed here, but these method signatures now have different defaults to preserve the original behavior. Users can use the new flag to change that behavior

Fixes #321

[frankyn]

Default value is different here so wondering how this will be reconciled.

[snippet-bot]

No region tags are edited in this PR.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

• Refresh this comment

[oripwk]

@JesseLovelace Does it work also with getContent() or only with downloadTo()?

[JesseLovelace]

@oripwk Should work with getContent, use getContent(Blob.BlobSourceOption.shouldReturnRawInputStream(true));

[oripwk]

@JesseLovelace when is it going to be released?

[oripwk]

Good, I see it's on 1.116.0

[marioaae]

Hi there, thanks for fixing this, unfortunately I haven't been able to make it work on my end. In Scala I'm trying to read files through a GZIPInputStream as follows:

val bucketName = "my-bucket-name"
val filePath = "path/to/file.gz"
val storage: Storage = StorageOptions.getDefaultInstance.getService

val blob: Blob = storage.get(BlobId.of(bucketName, fullPath), Storage.BlobGetOption.shouldReturnRawInputStream(true))
val reader: ReadChannel = blob.reader()
val inputStream: InputStream = Channels.newInputStream(reader)

Afterwards I'm wrapping it up in a GZIPInputStream to process it as follows: new GZIPInputStream(inputStream) but I get the following error:

java.util.zip.ZipException: Not in GZIP format

Still, if I only use the InputStream it works, so it seems to me that the file is being downloaded decompressed, I thought the shouldReturnRawInputStream(true) would download my file compressed in the original .gz format.

When I download the file from a different source (e.g. an SFTP server) where I know for sure no decompression is going on, wrapping it up in GZIPInputStream works properly.

Has anybody had the same problem?

Thanks!

[frankyn]

Hi @marioaae,

Could you clarify the state of your object metadata? In certain cases GCS will decompress an object on response when contentEncoding: gzip is set in object metadata.

[marioaae]

Hi @marioaae,

Could you clarify the state of your object metadata? In certain cases GCS will decompress an object on response when contentEncoding: gzip is set in object metadata.

Hi @frankyn,
I won't be able to control the metadata of that object, but in any case, when checking at the Google Console (UI) I can see that Content-Encoding is blank.