rclone config with hetzner storage box append only

[demlak]

Hey there.. we are using the script with restic-rest-server since a year.. everything works good..

now i want to change the way the script uploads data..

we use a hetzner storage box, which is able to listen via ssh and that could be forced to use restic in this way:
https://forum.restic.net/t/hetzner-storage-box-supports-rclone-serve-restic-stdio/5943/6

on my linux clients, everything went fine with the new way.

since i am not a windows pro, i don't understand exactly how to change the scripts for using it like this: restic -o rclone.program='ssh -p23 uXXXXX@uXXXXX.your-storagebox.de forced-command' -r rclone: init

i was following the instructions for installing ssh key.. like described in the README.md.. and checking via .\PsExec.exe -s -i powershell.exe + ssh user@server was working

could you explain, what exactly i have to update in config.ps1 and the other files, to use it in that way via the task scheduler?

thx a lot

EDIT: Workaround here: #95 (comment)

[SeeJayEmm]

I think you need to do the following:

In config.ps1 set the $AdditionalBackupParameters to include your -o parameter.
In secrets.ps1 set the RESTIC_REPOSITORY env variable to "rclone:" and set the password to whatever your repo password is.

This is all guessing since I've never tried it.

[demlak]

thx a lot.. your hint leeds me to update my installation.. but still problems:

following works perfect:
.\restic.exe -o rclone.program='ssh -p 23 uxxxx@uxxxx-sub3.your-storagebox.de forced-command' -r rclone: check

Since my installation is old.. i just updated: backup.ps1, install.ps1, and windows.exclude. I also checked config for changes between the versions.. but $AdditionalBackupParameters seems to be the only new variable

This in config.ps1:

$AdditionalBackupParameters = @("--exclude-if-present", ".nobackup", "-o", "rclone.program='ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command'")

And this in secrets.ps1:

$Env:RESTIC_REPOSITORY='rclone: '

Results to this:

PS C:\restic> . .\config.ps1; . .\secrets.ps1; & $ResticExe check

using temporary cache in C:\WINDOWS\TEMP\restic-check-cache-1769111303
rclone: 2024/03/09 08:48:36 NOTICE: Config file "C:\\WINDOWS\\system32\\config\\systemprofile\\AppData\\Roaming\\rclone\\rclone.conf" not found - using defaults
Fatal: unable to open config file: <config/> does not exist
Is there a repository at the following location?
rclone:

as far as i understand the sourcecode of backup.ps1, the $AdditionalBackupParameters is only used on backup, so i also tried this:

PS C:\restic> . .\config.ps1; . .\secrets.ps1; & $ResticExe $AdditionalBackupParameters check

using temporary cache in C:\WINDOWS\TEMP\restic-check-cache-1624378344
Fatal: unable to open repository at rclone: : cmd.Start: exec: "ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command": executable file not found in %PATH%   

There seems to be something wrong with my syntax

[SeeJayEmm]

Do you have OpenSSH installed on your Windows machine? If you try to SSH from the command line does it work?

[demlak]

Hey.. ssh from commandline works.. like described in the first part of my last post..

finaly it seems i found the problem(s)..

it is about the brackets.. i changed:

$AdditionalBackupParameters = @("-o", "rclone.program='ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command'")

to

$AdditionalBackupParameters = @("-o", 'rclone.program="ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command"')

and now, this works:

. .\config.ps1; . .\secrets.ps1; & $ResticExe $AdditionalBackupParameters check

but well.. as stated in my last posting, the variable $AdditionalBackupParameters is used only on backup-command.. so everything else in the backup-script is not working.. for example says log after starting task in task-scheduler:

restic.exe : rclone: 2024/03/09 21:50:13 NOTICE: Config file 
"C:\\WINDOWS\\system32\\config\\systemprofile\\AppData\\Roaming\\rclone\\rclone.conf" not found - using defaults
In C:\restic\backup.ps1:82 Zeichen:14
+     $locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog
+              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (rclone: 2024/03... using defaults:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
Fatal: unable to open config file: <config/> does not exist
Is there a repository at the following location?
rclone: 

(But new snapshot was generated)

As far as i understand, i need to change the config.ps1 in that way, that $ResticExe is has the config-parameters included.
kinda like

$rcloneconnect = @("-o", 'rclone.program="ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command"')
$ResticExe = "$ResticExe" + $rcloneconnect

(i don't know about powershell-syntax) or i have to change backup.ps1 for adding the config-parameters at every use of $ResticExe?
It seems, default it is not possible to use restic-windows-backup in the way i want to use it.

[demlak]

Ok.. finaly i got it working.. here is a workaround:

1. Download and install/copy rclone.exe to c:\windows\system32. So it is reachable for SYSTEM-user
2. Configure and test your SSH, like described in the README.md at Topic "Backup over SFTP".
3. Add a new Variable to config.ps1 and replace the uxxxx with your credentials:

$RcloneConnect = @("-o", 'rclone.program="ssh -p 23 uxxxx@uxxxx.your-storagebox.de forced-command"')

4. Edit backup.ps1:
   After every $ResticExe add an $RcloneConnect.
   For Example Line 82:
   before:

$locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog

after:

$locks = & $ResticExe $RcloneConnect list locks --no-lock -q 3>&1 2>> $ErrorLog

5. In secrets.ps1 use rclone: as your repo

i won't call this a "solution" but a workaround, because we have to alter the backup.ps1. Ofcourse those changes are gone, after next update.

@kmwoley maybe you want to reflect this kind of solution/way to use restic in your great powershell-scripts? this is a great way to use restic append only via ssh, without restic rest-server

my skillz in powershell and git are too low to provide a good solution/pull-request, sorry =(

[kmwoley]

If someone could work up a pull request, I'll happily review and integrate it. I'm due for my annual updates to the script. :D

…

On Sat, Mar 9, 2024 at 1:48 PM demlak ***@***.***> wrote: Ok.. here is a workaround. 1. Configure and test your SSH, like described in the README.md at Topic "Backup over SFTP". 2. Add a new Variable to config.ps1 and replace the uxxxx with your credentials: $RcloneConnect = @("-o", 'rclone.program="ssh -p 23 ***@***.*** forced-command"') 1. Edit backup.ps1: After *every* $ResticExe add an $RcloneConnect. For Example Line 82: before: $locks = & $ResticExe list locks --no-lock -q 3>&1 2>> $ErrorLog after: $locks = & $ResticExe $RcloneConnect list locks --no-lock -q 3>&1 2>> $ErrorLog In secrets.ps1 use rclone: as your repo i won't call this a "solution" but a workaround, because we have to alter the backup.ps1. Ofcourse those changes are gone, after next update. @kmwoley <https://github.com/kmwoley> maybe you want to reflect this kind of solution/way to use restic in your great powershell-scripts? this is a great way to use restic append only via ssh, without restic rest-server — Reply to this email directly, view it on GitHub <#95 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOWG6BOOQLABAOCBNCCMULYXN7RBAVCNFSM6AAAAABENCCIAOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBWHE4DSMBWGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[woelfisch]

If someone could work up a pull request, I'll happily review and integrate it. I'm due for my annual updates to the script. :D

There you go: #96

Three additional options

• $AdditionalParameters will be used to set additional arguments to repository related restic calls
• $SelfUpdateParameters for restic self-update
• $AllowResticSelfUpdate so the self update can be disabled e.g. in air gapped deployments