repo: Add an API for intercepting file commits and returning cached v…

[cgwalters]

…alues

Many people have observed that doing:

packagesystem --installroot=/var/tmp/blah
(cd blah && ostree commit ...)

is slow. There are a number of ways to optimize this. What
gnome-continuous does is to store separate subtrees for each
component, and union them. Then on commit, it uses a reverse
(device,inode) -> checksum mapping to avoid re-checksumming subtrees
that haven't changed.

For rpm-ostree, doing this sort of thing with RPM is...hard. It would
require changing how RPM writes to disk. One thing we can do is
support external cache mechanisms. And specifically one optimization
rpm-ostree can make is to create a mapping from
`(rpm version, filename) -> object checksum between the two trees.

[mbarnes]

This functionality is really cool to have built-in, but as a matter of API design I think I'd prefer a GTypeInterface, say OstreeFileCacheIface, with a single method:

char * (*get_checksum) (OstreeFileCache *cache,
                        OstreeRepo *repo,
                        const char *path)

Don't know how you imagined the callback getting set up, but having an interface definition would make it possible to use the GIOExtension framework, if we wanted to go with an automatic plugin-like approach.

[cgwalters]

I was thinking eventually the best thing would be to offer a convenient interface for callers to more fully control the commit process. Currently the ostree_repo_write_directory_to_mtree() is doing quite a lot and while you could do it all using the public API I think, it'd be really painful.

You'd have to reimplement things like opening a file stream, getting xattrs, calling oostree_repo_write_content() on it etc.

But yeah...having it be an interface would work. I think I'd prefer an API for providing an object implementing that interface over something where values get auto-loaded though via extensions because the latter is more for process-global things, but you may want to do two rpm-ostree things in one process.

[mbarnes]

Were you thinking of some kind of repo config option for this?

[cgwalters]

I don't think it's related to the repo config. Conceptually, I could have e.g. gnome-continuous content and rpm-ostree content in the same repo. They use different mechanisms to build software and have different forms of caching.

[cgwalters]

This is obsolete in favor of #167 - see coreos/rpm-ostree#209