Introduce experimental FileOutput interface for models that output File and Path types

[aron]

This PR is a proposal to add a new FileOutput type to the client SDK to abstract away file outputs from Replicate models.

It can be enabled by passing the use_file_output flag to the run() method (this will be moved to the constructor).

replicate.run("black-forest-labs/flux-schnell", input={}, use_file_output=True);

When enabled any URLs (and soon data-uris) will be converted into a FileOutput type. This is essentially an Iterable[bytes] | AsyncIterable[bytes] that has two additional fields, the attribute url referencing underlying URL and read() which will return bytes with the file data loaded into memory.

The intention here is to make it easier to work with file outputs and allows us to optimize the delivery of file assets to the client in future iterations.

Usage is as follows:

output = replicate.run(
  "black-forest-labs/flux-schnell",
  input={"prompt": "astronaut riding a rocket like a horse"},
  use_file_output=True,
);

For most basic cases you'll want to utilize either the url or read() fields depending on whether you want to directly consume the file or pass it on.

To access the file URL:

print(output.url) #=> "https://delivery.replicate.com/..."

To consume the file directly:

with open('output.bin', 'wb') as file:
    file.write(output.read())

Or for very large files they can be streamed:

with open(file_path, 'wb') as file:
    for chunk in output:
        file.write(chunk)

Each of these methods has an equivalent asyncio API.

async with aiofiles.open(filename, 'w') as file:
    await file.write(await output.aread())

async with aiofiles.open(filename, 'w') as file:
    await for chunk in output:
        await file.write(chunk)

For streaming responses from common frameworks, all support taking Iterator types:

Django

@condition(etag_func=None)
def stream_response(request):
    output = replicate.run("black-forest-labs/flux-schnell", input={...}, use_file_output =True)
    return HttpResponse(output, content_type='image/webp')

FastAPI

@app.get("/")
async def main():
    output = replicate.run("black-forest-labs/flux-schnell", input={...}, use_file_output =True)
    return StreamingResponse(output)

Flask

@app.route('/stream')
def streamed_response():
    output = replicate.run("black-forest-labs/flux-schnell", input={...}, use_file_output =True)
    return app.response_class(stream_with_context(output))

[mattt]

Great work, @aron! I made a couple changes and moved some things around to better match project conventions, but overall I think this is looking great.

[bfirsh]

I love it. This is really excellent.

A thought about the iterator semantics... are there any existing patterns for iterators on file handles returning chunks of data? Usually file handles return lines or single bytes if you iterate over them, and HTTPX responses have the iter_*() methods.

It feels like we should be consistent with one of those patterns rather than invent a new pattern. If it's the file handle pattern, which feels like the more universal thing to me, then we could also support the length argument to read() to read chunks.

Not a big deal anyway and we can always iterate (zing!) before release.

[aron]

@bfirsh this is great feedback thanks. I started off implementing io.IOBase but that felt more complex, than the httpx.Response which felt cleaner and supported asyncio nicely. I saw the iter_*() methods which provide decoding utilities, in favor of just providing the bytes and simplifying the interface.

If adding support for read(size) improves things we can do that.

[aron]

I've added support for data-uris. This puts it into parity with JS implementation.

Also I did a quick skim of common web frameworks Django, Flask and FastAPI, all support streaming HTTP responses with iterators so I've updated the PR description with examples. But this aspect feels pretty good with the current implementation.

[bfirsh]

OK brillant. If this is compatible with things that expect to get a binary file handle type thing, that's all I care about.

@mattt Do you have any thoughts about file handle and iterator semantics?

[mattt]

@bfirsh What we have now is in line with how I'd expect something like this to work. @aron did a good job surveying how this API is likely to be used.