Feature request: exclude files specified in .gitignore

[c0d3z3r0]

It would be great if there was an option "--exclude-gitignore" that enables exclusion of files specified in .gitignore.
When a .gitignore file exists in a directory, all rules in it could be converted to regex patterns at runtime like this:

*.o                => ^_pwd_/.*\.o$
*.pyc              => ^_pwd_/.*\.pyc$
file               => ^_pwd_/(.*/)?file$
/another_file      => ^_pwd_/another_file$

I have no clue how to handle gitignore excludes like "!/includefile"...

[ThomasWaldmann]

Considering there is a ton of other software having its own "ignore" or "exclude" list, i would find a --exclude-gitignore option a rather strange thing to have for a not-git-related software.

[c0d3z3r0]

hmm.. yeah, you've got a point there..

[ThomasWaldmann]

this is somehow closely related to #641, so I'll close this one - we can discuss there.

[tomaskikutis]

Great suggestion - I was thinking about the same. I agree that this isn't a git-related software and I don't suggest to call it --exclude-gitignore - but it would be good to support 3rd party ignore lists and .gitignore could simply be one of them. there's also .dockerignore for example. The flag could be something like --exclude-lists="gitignore,dockerignore"

[tomaskikutis]

Another thing to consider with git repos is that the list of files depends on a branch one is at. Thus if regular backups are enabled, sometimes you would be on branch A and it would backup files a1, a2, a3 and sometimes you would be on a branch B and backup tool would think that files a1, a2 and a3 were deleted. When in reality no changes were made.

I'm thinking the best approach is not to backup "active" git repositories at all and instead either use git bundle or a local remote that would be a bare repository.

[c0d3z3r0]

@tomaskikutis Not sure if this approach is a good idea. What if there are files one is currently working on (not commited yet)? These won't be backed up then.

Using git bundle would be difficult I guess, because borg restore works on filesystem basis. How and where would you save the bundles?

[tomaskikutis]

What if there are files one is currently working on (not commited yet)? These won't be backed up then.

I hope I'll never need my backups. In case I do, losing uncomitted code would be a very minor inconvenience.

Using git bundle would be difficult I guess, because borg restore works on filesystem basis. How and where would you save the bundles?

Backup location needs to be picked regardless if one backs up using bundle command, or local remotes. I've created a dedicated folder in my home diretory for git backups.

[c0d3z3r0]

What if there are files one is currently working on (not commited yet)? These won't be backed up then.

I hope I'll never need my backups. In case I do, losing uncomitted code would be a very minor inconvenience.

Well, that might work for you but not for everybody :-)

Using git bundle would be difficult I guess, because borg restore works on filesystem basis. How and where would you save the bundles?

Backup location needs to be picked regardless if one backs up using bundle command, or local remotes. I've created a dedicated folder in my home diretory for git backups.

I see, well, in that case you could have all your git repos ending with .git and just add a global exclude rule.

[tomaskikutis]

After using my approach for a while I realized it's not ideal since I still need to write custom code to get it working. A good generic solution would be if borgbackup supported custom backup strategies for certain folders. Strategies themselves could be 3rd party plugins if including this in core is not desired.

I imagine it working similar to --exclude-if-present. If a folder matches a rule, custom code would run and generate backup-ready version of that folder.

For example, with git repositories, it could use git bundle and only backup the output. The same strategy plugin could also provide custom code for restoring.