-
Notifications
You must be signed in to change notification settings - Fork 280
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add alternative backends support (originally #365) #410
Conversation
if (path == NULL) | ||
return Qnil; | ||
|
||
return rb_str_new_utf8(path); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A common pattern used in rugged for this case is:
git_repository *repo;
const char *path;
Data_Get_Struct(self, git_repository, repo);
path = git_repository_path(repo);
return path ? rb_str_new_utf8(path) : Qnil;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Much nicer, I'll push an update.
I'm not sure what the test failures are, but they don't seem to be related to my changes. Anything I can do to get this to pass... (merge something in, etc...)? |
You could either rebase your changes against the latest |
@arthurschreiber Rebased, tests passed everywhere except on osx & ruby-head. There, it crashed with a malloc error... Is that something I need to fix, or just a random failure? |
@charypar Nope, that's an issue with ruby-head on OS X, not with rugged (as far as I can tell). I'm a bit concerned why one of @vmg's comitts is showing up here. Anyway, on the one hand I like the idea of having pluggable backends, on the other I really dislike the way custom backends have to somehow get the rugged header file into their projects. @vmg What do you think? |
Oh, Missed that commit, strange... I must have screwed up the rebase a little bit 😕 I agree the rugged header is a headache. I'd love a better solution than that, but can't think of one... |
Was there possibly a reset on master? When I did the rebase yesterday, that commit was the head of master, now it's not in the master history at all. 😕 |
@arthurschreiber @vmg any updates on this? |
At a risk of being annoying... Can I get some feedback/decision on this? :) |
Hey Victor, Sorry this is taking so long. I'm probably never going to use this @vmg can you take a look an just let me know what you think about |
1c28462
to
be7fb89
Compare
@arthurschreiber Fair enough, I understand. I guess all I can say is I keep seeing an appetite for pluggable backends in issues around github (various libgit2 repos, but also on the rugged-redis gem I built to use with the backend support). People seem interested in backing git with different things than disk, which is understandable given the move to deploying web services using Docker and other ways that don't mix well with local disk storage. All that said, I absolutely agree that the need for the header is nasty. Need to linking to rugged's version of libgit2 feels even worse. I'd love to figure out a better way, but the only thing I can think of is submoduling rugged into every backend implementation and version locking the dependency to a specific release. And I'm not sure that's any better. Still, I can't give up on the feature, and it would be a shame to fork rugged just for what is, on rugged's side, quite a small feature. |
return; | ||
|
||
cleanup: | ||
if (repo != NULL) git_repository_free(*repo); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You don't need those NULL checks. All the libgit2 git_*_free
functions handle this on their own.
Can you check the comments and rebase against the latest master? |
That should be all the problems fixed. Also rebased on top of current master. |
@arthurschreiber Still interested to get this in :) anything more I can do? |
I'm off the next few days, but I'll get this wrapped up by the end of this week. |
@arthurschreiber ping :) |
@arthurschreiber can we please wrap this up? Anything I can do? |
@vmg Can I merge this? This will break the |
@charypar can I encourage you to add the backend option to #clone_at? |
It's a bit strange that you can return to a repo using #init_at, but #new fails, as it can't find the path. |
@jamiehodge I can probably add support for The reasoning behind not supporting |
That makes sense. I guess I would then say that .new ought to be renamed to .working or the like... anyhow, it would be great if you could add speculative .clone_at support. Do any of the backends at rugged-backends support the pack format? I was considering writing a rugged-mysql wrapper this week. |
@vmg I'd like to get this merged. Is the API change of |
@arthurschreiber @vmg Any chance we could wrap this up? |
I know you guys are quite busy, but this has been open for four months now. Can we please get it in or otherwise decide on its fate? Sorry for bugging you repeatedly, I'm just worried it'll get way out of sync with master again. |
Hey @charypar! I'm super sorry this has been dragging on for so long! Can you do me one favor? I don't want to merge this in it's current state, as it breaks the existing API of If you do that, I'll merge this immediately, I promise! |
Sure, no problem. I'll look at it over the weekend. |
Ok, done. Should accept both an array and an options hash now. |
@charypar Thanks for being so patient with me! ❤️ |
Add alternative backends support (originally #365)
This is a second pull request for these changes (originally #365, which is now very out of date - still against
development
branch).I addressed the comments in the original PR. Sorry for the inactivity, this time I'm fully determined to get this in :)
/cc @arthurschreiber
Original description
Surfaces libgit2 backends support in rugged in a way that shouldn't
affect performance. The change enables using rugged in environments with ephemeral
file-systems (e.g. Heroku), or as an in-memory data versioning tool for something
like very complex undo/redo, for example. It's probably of no use for normal
code repos, but is very useful for using git repos as a storage format for other
things (something like gist, for example)
Sorry for the length of the description, it's not the simplest thing to explain. :)
Interface
The interface between rugged and the backend has two parts: 1)
Rugged::Backend
ruby class (empty) to inherit in the backend implementation acting as a wrapper for 2)
a
rugged_backend
C struct.The backend implementation has to provide two constructor functions that take
the repository path and return the ODB and RefDB backend structs.
Creating some backends requires special configuration (redis connection
settings for example) and the backend is supposed to wrap the
rugged_backend
struct with it's own struct containing the extra information needed. Example:
It also needs to provide the constructor functions which use that information to
call the actual backend constructors. Example:
The reason for this extra indirection is the
path
parameter, which is only knownat the moment of creating/opening a repo.
Returned backends are then passed to
git_odb_add_backend
andgit_refdb_set_backend
and eventually to
git_repository_wrap_odb
to create a repo. From then on, everythingworks the same way, except the storage itself.
The backend wrapper implementation is responsible for surfacing a ruby level API for
providing the extra configuration.
See rugged-redis for a complete example
of a backend.
API Changes
This directly affects two rugged methods:
Rugged::Repository#bare
andRugged::Repository#init_at
. For the former it's a breaking change replacing the optionalalternates
parameter with an options hash and moving alternates into it, but it's hopefully aminor change affecting a small percentage of clients.
I only added support for bare repositories, because there is no way to provide
a place for a working directory.
Usage example
Constructing a bare repo using an alternative backend looks something like this
Problems
One issue is that the created repositories don't have path. I fixed a crash
in
rb_git_repo_path
wheregit_repository_path
returnsNULL
, but it would benicer if it was possible to provide the path to the repo created by wrapping an ODB.
Second slight problem is that the backend implementation needs
rugged.h
, whichmakes writing the
extconf.rb
tricky, but possible.