Could most of the Channel API be provided by transport-agnostic code?

[oremanj]

(Happy to merge this into #719 if you'd like; I figured this was likely to spawn discussion and didn't want it to get lost in that thread.)

I was trying to write a not-just-in-memory channel implementation today, and it struck me that there's rather a lot of tricky logic that needs to be implemented separately for each kind of channel but isn't really dependent on the exact nature of the kind of channel.

I had the idea that we might refactor the interface so the abstract part is something like

class SendChannelTransport:
    def send_nowait(self, value) -> None: ...
    async def send(self, value) -> None: ...
    async def done_sending(self, *, poison=None) -> None: ...

class ReceiveChannelTransport:
    def receive_nowait(self) -> object: ...
    async def receive(self) -> object: ...
    async def done_receiving(self, *, poison=None) -> None: ...

(Different names for send-shutdown and receive-shutdown so it's possible for one class to implement both; not AsyncResource because these are intended to be extender-facing interfaces, not user-facing interfaces. Maybe they should even be in hazmat^Wpowertools^Wwiring^Wkernel^Wthat-thing-with-the-name.)

And then SendChannel and ReceiveChannel could be concrete classes that provide the existing channel interface, with the closure tracking and the smooth edges and so forth, when constructed using an appropriate ChannelTransport. The ChannelTransport classes could have a stricter contract with less state-keeping that would be easier to implement than the Channel contract; for example, if it were undefined to keep calling receive() after a previous call raised EndOfChannel, receive() could be written as "deserialize an object from the transport, throw EndOfChannel if it corresponds to the magic EOF marker".

I haven't gotten far enough into this to be totally confident of the feasibility, but I don't immediately see blockers, and it seems like if we can pull this off it would make it much much easier to implement real Channels (as opposed to the currently-tempting "things with send and receive methods but that don't support fanin/fanout") over sockets, pipes, etc.

[njsmith]

I don't think I'm quite seeing what you're imagining here. (Maybe because I'm sleepy.) Can you elaborate a little more? Do you have some sample code you could show?

[oremanj]

Yeah, there were some issues that became obvious when starting to implement this, like the inability to compose abort_fns. I was sleepy when I wrote it up too. :-) I think I'm going to instead go the route of trying to write a StreamChannel given serializing/deserializing methods, which should support the same kinds of deduplication without as much complexity.