Read chunked files from URLs

[dmkl]

Hi @ananthakumaran,

we need to stream large files from a cloud storage. Is there an easy way to adopt zstream for feeding it read chunks? I'm not that advanced with Elixir yet. Can you give some hints on how to extend zstream for that purpose, please? Would it fit here at all?

[ananthakumaran]

I would likely need more info, whether zip/unzip, which cloud provider etc. But in most cases, you need to create the input/output as a stream. For example, if you want to download and zip large files from s3, you can use the aws library to fetch the files and then use zstream to zip. In general, I would suggest getting familiar with Elixir stream abstraction, then the solution is usually a matter of composing the streams correctly.

[dmkl]

Thank you! I've figured out how to feed Stream.resource as a zstream entry. It works well 👍🏻