"Remote host doesn't support hardlinks" but manual hardlinks are possible

[lbckmnn]

My goal is to backup to a SSH host which has a smb share mounted.
When I try to create the profile, i get:

"Remote host doesn't support hardlinks"

If I leave the option "check if remote host support all neccessary commands" disabled, I can create the profile, but incremental backups are indeed not performed with hardlinks. Strangely enough, manual creation of hardlinks works just fine.
If i do touch a and ln a b on the smb share on the remote host, the files a and b have the same inode number.

If I choose a path which is not on the smb share, backintime does not complain about missing hardlink support.

Maybe this is not an issue of backintime but of rsync

local version of rsync: v3.2.3
remote version of rsync: version 3.1.3

Does anyone have any ideas what else I can do?

[rjdriscoll]

I believe I have a similar problem using a local NTFS formated USB hard disk. Each backup operation make new copies of my files with just one link when viewed with "ls -l." However when I use the linus command line tools I can make a hard link using "ln"

It looks as if the linux driver can make hard links but back in time doesn't make use of the fact, unike the "ln" command.

Running back in time with a local ext4 filesystem works as expected with multiple links when viewed with "ls -l"

[bebabi34]

similar problem here. i have made backups for many years without problems (last on 2021-08-16). now, the backup on usb drive formatted ext4 don't creates hard-links but copies files from within the same drive. i noticed also that the option "use full rsync" is no longer present in the gui. manual creation of ln on the drive works as LucasBec said.

debian stable (bullseye) 64bit
backintime 1.2.1

$ lsusb -t | grep -i storage
|__ Port 8: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 480M

fdisk -l
Device Boot Start End Sectors Size Id Type
/dev/sdd1 2048 3907028991 3907026944 1,8T 83 Linux

$ cat /etc/fstab
UUID=[...] /media/backup ext4 noauto 0 0

$ mount
/dev/sdd1 on /media/backup type ext4 (rw,relatime)

$ backintime-qt_polkit --debug
DEBUG: [common/backintime.py:583 argParse] Arguments: {'debug': True} | unknownArgs: []
Back In Time
Version: 1.2.1
Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type backintime --license for details.
DEBUG: [common/backintime.py:670 getConfig] config file: /root/.config/backintime/config
DEBUG: [common/backintime.py:671 getConfig] share path: /root/.local/share/backintime
DEBUG: [common/backintime.py:672 getConfig] profiles: 1=Profilo principale
DEBUG: [common/pluginmanager.py:90 PluginManager.load] Register plugin path /usr/share/backintime/plugins
DEBUG: [common/pluginmanager.py:106 PluginManager.load] Add plugin notifyplugin.py
DEBUG: [common/pluginmanager.py:106 PluginManager.load] Add plugin qt4plugin.py
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
DEBUG: [common/tools.py:799 keyringSupported] No keyring due to import error.
DEBUG: [common/mount.py:73 Mount.init] pw-cache is not running
DEBUG: [common/mount.py:81 Mount.init] Call command: /usr/bin/backintime pw-cache start
DEBUG: [common/tools.py:1231 readCrontab] Read 5 lines from users crontab
DEBUG: [common/config.py:1465 Config.removeOldCrontab] Clearing system Back In Time entries
DEBUG: [common/config.py:1499 Config.cronLine] Profile: Profilo principale | Automatic backup: Disabilitato
DEBUG: [common/config.py:1449 setupCron] Crontab didn't change. Skip writing.
DEBUG: [common/backintime.py:583 argParse] Arguments: {'debug': True, 'command': 'backup', 'func': <function backup at 0x7f4314b5bca0>} | unknownArgs: []
Back In Time
Version: 1.2.1
Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.
DEBUG: [common/backintime.py:670 getConfig] config file: /root/.config/backintime/config
DEBUG: [common/backintime.py:671 getConfig] share path: /root/.local/share/backintime
DEBUG: [common/backintime.py:672 getConfig] profiles: 1=Profilo principale
DEBUG: [common/pluginmanager.py:90 PluginManager.load] Register plugin path /usr/share/backintime/plugins
DEBUG: [common/pluginmanager.py:106 PluginManager.load] Add plugin notifyplugin.py
DEBUG: [common/pluginmanager.py:106 PluginManager.load] Add plugin qt4plugin.py
DEBUG: [common/snapshots.py:1751 Snapshots.flockExclusive] Set flock /tmp/backintime.lock
INFO: [common/snapshots.py:634 Snapshots.backup] Lock
DEBUG: [common/tools.py:1187 inhibitSuspend] Inhibit Suspend failed because BIT was started as root.
DEBUG: [common/tools.py:799 keyringSupported] No keyring due to import error.
DEBUG: [common/mount.py:73 Mount.init] pw-cache is not running
DEBUG: [common/mount.py:81 Mount.init] Call command: /usr/bin/backintime pw-cache start
INFO: [common/snapshots.py:665 Snapshots.backup] Take a new snapshot. Profile: 1 Profilo principale
INFO: [common/snapshots.py:987 takeSnapshot] Remove leftover 'new_snapshot' folder from last run
DEBUG: [common/snapshots.py:989 Execute.takeSnapshot] Call command "rsync -a --delete /tmp/tmp1ugefl9x/ /media/backup/backintime/desktop-deb/root/1/new_snapshot"
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
DEBUG: [common/snapshots.py:989 Execute.takeSnapshot] Command "rsync -a --delet..." returns 0
INFO: [common/snapshots.py:1022 Snapshots.takeSnapshot] Call rsync to take the snapshot
DEBUG: [common/snapshots.py:692 Snapshots.backup] Call command "rsync --recursive --times --devices --specials --hard-links --human-readable --links --acls --xattrs --perms --executability --group --owner --info=progress2 --no-inc-recursive --delete --delete-excluded -v -i --out-format=BACKINTIME: %i %n%L --link-dest=../../20210816-181114-647/backup --chmod=Du+wx --exclude=/media/backup --exclude=/root/.local/share/backintime --exclude=.local/share/backintime/mnt --include=[...]/root/ --include=/etc/ --include=/var/lib/synaptic/ --include=/var/lib/ --include=/var/ --include=/usr/lib/systemd/system/ --include=/usr/lib/systemd/ --include=/usr/lib/ --include=/usr/ --include=/usr/lib/systemd/system-sleep/ --include=/home/ --include=/usr/local/bin/ --include=/usr/local/ --include=/media/dati/ --include=/media/ --exclude=.gvfs --exclude=.cache/* --exclude=.thumbnails* --exclude=[Tt]rash* --exclude=.backup --exclude=*~ --exclude=.dropbox* --exclude=/proc/* --exclude=/sys/* --exclude=/dev/* --exclude=/run/* --exclude=/etc/mtab --exclude=/var/cache/apt/archives/.deb --exclude=lost+found/ --exclude=/tmp/* --exclude=/var/tmp/* --exclude=/var/backups/* --exclude=.Private --exclude=/media/dati/[...]/tmp --exclude=.Trash-* --exclude=.dtrash --exclude=/media/dati/[...]/appdata/VirtualBox/VMs --exclude=/home/swapfile --include=/root/** --include=/etc/** --include=/var/lib/synaptic/preferences --include=/usr/lib/systemd/system/bluetooth.service --include=/usr/lib/systemd/system-sleep/fancontrol --include=/home/** --include=/usr/local/bin/usb-no-wakeup.sh --include=/media/dati/** --exclude=* / /media/backup/backintime/desktop-deb/root/1/new_snapshot/backup"

[bebabi34]

tried with master branch, same effect.
i tried this:
cp -al previous_snapshot/backup/* new_snapshot/backup
touch new_snapshot/save_to_continue
then run backintime. it worked as expected.
the next run of BIT worked well.
i don't know what happened.

[texsit]

Hi,
I have a similar problem when I do a backup on an USB Hard disk formatted in NTFS. Every time i run a snapshot, BIT create a full copy and doesn't use the hardlinks for the files not modified. If I execute manually the creation of the hard links on the USB Hard disk (cp -al command) it works perfectly. Please can someone suggest what is the problem? In this way the software is unusable.

Thank you.

Best Regards.

[bebabi34]

happened again today. solution:

# rm -r new_snapshot
# mkdir -p new_snapshot/backup
# cp -al <_last_snapshot_>/backup/* new_snapshot/backup
# touch new_snapshot/save_to_continue

then run bit twice and delete first backup taken. 😕

[emtiu]

This is a tough one … we've got 4 people reporting the same error message on:

1. an ssh target with an smb mount
2. two NTFS target drives and
3. an ext4 target drive.

Are you all still affected by this problem?

[bebabi34]

after i fixed with the proposed workaroud, no more happened

[emtiu]

Hmm, this is apparently a case where the behavior of cp -l differs from that of rsync. Related to #988 and #944 in the sense that this was brought about by making "full rsync mode" the default (and eliminating the previous cp -l procedure).

[carmurio]

Hi, I have the same issue, and it seems to be happening only in remote hosts.

I have one local ntfs partition and the incremental backups are working fine. it is using hard links because the size of the folder is much smaller than the size of each backup put together.

But I tried to move my backups to another ntfs disk that I have in a different server in the network. I selected ssh and set up everything, but it doesn't allow it, it shows the message "Remote host doesn't support hardlinks".

Please, let me know if this is going to be solved, ntfs is the only fs that makes sense if you have linux and windows in the same network.

[ptilopteri]

* carmurio ***@***.***> [05-12-23 20:24]:

Hi, I have the same issue, and it seems to be happening only in remote hosts. I have one local ntfs partition and the incremental backups are working fine. it is using hard links because the size of the folder is much smaller than the size of each backup put together. But I tried to move my backups to another ntfs disk that I have in a different server in the network. I selected ssh and set up everything, but it doesn't allow it, it shows the message "Remote host doesn't support hardlinks". Please, let me know if this is going to be solved, ntfs is the only fs that makes sense if you have linux and windows in the same network.

I have cifs working quite well, also sshfs works excellently if ssh is installed on windoz.

…

-- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.org openSUSE Community Member facebook/ptilopteri Photos: http://wahoo.no-ip.org/piwigo paka @ IRCnet oftc

[aryoda]

@ALL Could you please show us the mount options of your SMB/NTFS remote drive?

Since you are using ssh I need the mount options of the target system.

I want to find out if some permission or timestamp options may always cause a full re-backup because rsync thinks the files have changed (so no hardlink is created if though technically the remote system would be able to do so).

[bebabi34]

sorry, i switched to ext4 fs.

[carmurio]

Hi, the target system was a usb hd drive mounted in a raspberry pi 3b, here is the record in the fstab file. However, my usb drive died and I already switched to a new drive formatted in zfs, which is used only for backups from my linux system. So I can't really test it anymore if you need it. #LABEL=usbntfs /mnt/usbhd auto rw,nosuid,nodev,nofail,uid=xxxxxx,gid=xxxxx 0 0 Regards, Carlos

…

On Sun, Sep 24, 2023 at 3:21 PM aryoda ***@***.***> wrote: @ALL <https://github.com/ALL> Could you please show us the mount options of your SMB/NTFS remote drive? Since you are using ssh I need the mount options of the target system. I want to find out if some permission or timestamp options may always cause a full re-backup because rsync thinks the files have changed (so no hardlink is created if though technically the remote system would be able to do so). — Reply to this email directly, view it on GitHub <#1164 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANHD2TQLBDKE7GSLCHG332LX4CB3XANCNFSM45SXKLXQ> . You are receiving this because you commented.Message ID: ***@***.***>

[aryoda]

@carmurio

auto

Can you still remember, if NTFS or NTFS-3g was used as filesystem type?

[aryoda]

@lbckmnn

Bug Analysis

My goal is to backup to a SSH host which has a smb share mounted. When I try to create the profile, i get:

"Remote host doesn't support hardlinks"

This error message is created in our checkRemoteCommands() function which is only used in case of ssh.

It checks if all required (shell) command and hardlinks are supported over ssh:

backintime/common/sshtools.py

Lines 964 to 967 in 5efb5d6

	if len(inodes) == 2 and inodes[0] != inodes[1]:
	raise MountException(
	_("Remote host {host} doesn't support hardlinks")
	.format(host=self.host))

If I leave the option "check if remote host support all neccessary commands" disabled, I can create the profile
but incremental backups are indeed not performed with hardlinks.

Yes, this prevents calling above function but then (without working hardlinks) you end up with full copies in every snapshot (because rsync treats them as "changed", see below in "suspected reasons").

If i do touch a and ln a b on the smb share on the remote host, the files a and b have the same inode number.

Yes, SMB (and also NTFS-3g) support hardlinks (if configured correctly).

Suspected reason

When using SMB or NTFS-3g via ssh

BiT introduced a new permission handling in v1.2.0 which als rsyncs the permissions (+ group + owner)
effectively adding the rsync options --perms --executability --group --owner.

These new options are also used in the failing checkRemoteCommands() function:

backintime/common/sshtools.py

Lines 768 to 769 in 5efb5d6

	rsync1 = tools.rsyncPrefix(
	self.config, no_perms=False, progress=False)

Since SMB (and NTFS-3g) apply user and permission mappings via the mount options no hardlinks created when the owner, group and permissions of the source file are different from the owner, group and permissions of the target
and they are different because user and permission mappings are applied via the mount options.

To check if this happens in your case please show us here your mount options of the remote server or even better connect via ssh and copy local file to the remote and check the permissions (with stats or ls -l).

When using SMB or NFTS-3g mounted locally (without ssh)

This setup was reported as failing in BiT by @rjdriscoll and @texsit while @ptilopteri reported it does work.

The reasons is the same: User and permission mappings in the mount options.

So could you please post your mount options here to verify this (use mount and find the line with your device which contains the actual mount options even if defaults apply).

BTW: In this constellation our function checkRemoteCommands() is not called so you do not see the "remote host doesn't support hardlinks" error message.

Possible other reasons:

Besides the mount options in case of SMB also a configuration file needs to be tweaked to support hardlinks (TODO: Which one?)

Workaround

Below workaround should work for all scenarios except a wrong SMB configuration.
Could you all please test it and give us feedback here?

https://github.com/bit-team/backintime#file-permissions-handling-and-therefore-possible-non-differential-backups

Fix

The fix depends on how we solve #988