Support directly pulling and pushing images from HTTP/cloud storage without a registry server

[PaulFurtado]

A registry server is such a thin layer over cloud provider/http storage of image layers, so it'd be awesome runtimes could pull directly from an HTTP server or cloud storage without needing to run a registry daemon.

We currently run a docker registry daemon on every one of out kubernetes nodes so that image pulling never requires a central registry service to be up and running (which itself, runs inside kubernetes for us). Given how simple the registry storage format is, I feel like it'd be pretty straightforward to directly support the registry storage backends in the runtime and it would reduce the operational complexity for internal registries.

[mtrmac]

Thanks for your report.

Yes, the read side of the docker/distribution protocol is pretty close to raw HTTP (#469 (comment) ).

There already exists https://github.com/NicolasT/static-container-registry to configure a HTTP server to serve compatible content; updating or extending that approach seems more practical to me than introducing some entirely new protocol (which would essentially happen by freezing and codifying the docker/distribution backend storage layout, independently from the evolution of that project), even if the differences are fairly minor.

But, ultimately, there needs to be some server code to provide the data, possibly require credentials for read access, and definitely require credentials for write access; so it’s not quite obvious to me that there is substantial difference between a docker/distribution server process, an nginx “ordinary static HTTP” server process, or a “cloud-native distributed storage with HTTP API access option” server process — but this is an area in which I have zero personal experience and I may well be wrong about the tradeoffs.

(That said, you can of course implement a new transport for containers/image to deal with whatever layout you need; as for accepting it upstream, it’s a question of it being likely enough to be generally useful.)

[PaulFurtado]

@mtrmac thanks for the response. I hadn't realized containers/image was the upstream project from this, otherwise I'd have filed this issue there. I had also looked for something like NicolasT's static-container-registry in the past, but turned up empty handed, so good to know that that is practical.

Given your assessment, it does seem like there'd be little chance that containers/image would merge a change that calcifies docker/distribution's backend implementation and that makes perfect sense to me. That said, as an alternative, it'd be awesome if there was an exec-based plugin framework for containers/image so that people could easily plug the daemon to introduce alternative ways to fetch the bytes (similar to how yum and apt-get support transport plugins)

Anyway, thanks!

[mtrmac]

Given your assessment, it does seem like there'd be little chance that containers/image would merge a change that calcifies docker/distribution's backend implementation and that makes perfect sense to me.

(Well, that depends on the pros/cons balance; the cons have been articulated, the pros are not clear to me — but that may well be me not having the experience.)

That said, as an alternative, it'd be awesome if there was an exec-based plugin framework for containers/image so that people could easily plug the daemon to introduce alternative ways to fetch the bytes (similar to how yum and apt-get support transport plugins)

FWIW that’s not likely to happen soon, because the transport interface of c/image is changing rather frequently, exactly to support new transports and features.