python2: can't open file 'blocksync.py': [Errno 2] No such file or directory

[devZer0]

btw, i'm getting the error below and could workaround it by removing "python2" from the command

before

 cmd = ['python2', '/root/blocksync.py', 'server', dstpath, '-a', options.hashalg,

after

    cmd = ['blocksync.py', 'server', dstpath, '-a', options.hashalg,    

Errors:

# borg extract --stdout /mnt/repo2::sda_07_10_2022_8 |blocksync.py - localhost /dev/sdc

Dry run     : False
Local       : True
Block size  : 128.0 KB
Skipped     : 0 blocks
Hash alg    : sha512
Crypto alg  : aes128-cbc
Compression : False
Read cache  : True
SRC command : /usr/local/bin/blocksync.py - localhost /dev/sdc
DST command : python2 blocksync.py server /dev/sdc -a sha512 -b 131072 -k 0

python2: can't open file 'blocksync.py': [Errno 2] No such file or directory
ERROR: connecting to or invoking blocksync on the remote host!

# python2 blocksync.py
python2: can't open file 'blocksync.py': [Errno 2] No such file or directory

[shodanshok]

Thanks for reporting the issue. The error is related to the remote component not finding blocksync and I fail to understand how removing python2 fixed it. Rather, I think that it worked because you removed the absolute path when calling blocksync and so it was executed via the PATH environment. However, I can not see blocksync being called via the absolute path /root/blocksync.py in my code. Did you changed it when testing for stdin? Can you try to re-include python2 in the command?

[devZer0]

if i re-include python2 in the command, i get no such file or directory.

i'm on ubuntu 22.04

what's your distro ?

ps:

absolute path /root/blocksync.py in

sorry, that's an copy/past error from older buffer

[devZer0]

# cat /usr/local/bin/blocksync.py |grep blocksync.py
* Copy blocksync.py to the home directory on the remote host
    sudo python2 blocksync.py /dev/source user@remotehost /dev/dest
    cmd = ['python2', 'blocksync.py', 'server', dstpath, '-a', options.hashalg,
    #cmd = ['blocksync.py', 'server', dstpath, '-a', options.hashalg,
    
# python2 blocksync.py /dev/sda localhost /dev/sdc
python2: can't open file 'blocksync.py': [Errno 2] No such file or directory

# blocksync.py /dev/sda localhost /dev/sdc

Dry run     : False
Local       : True
Block size  : 128.0 KB
Skipped     : 0 blocks
Hash alg    : sha512
Crypto alg  : aes128-cbc
Compression : False
Read cache  : True
SRC command : /usr/local/bin/blocksync.py /dev/sda localhost /dev/sdc
DST command : python2 blocksync.py server /dev/sdc -a sha512 -b 131072 -k 0

python2: can't open file 'blocksync.py': [Errno 2] No such file or directory
ERROR: connecting to or invoking blocksync on the remote host!

[devZer0]

i come across https://stackoverflow.com/questions/9935151/popen-error-errno-2-no-such-file-or-directory-when-calling-shell-function

and added shell=True to

 p = subprocess.Popen(cmd, bufsize=0,
                         stdin=subprocess.PIPE, stdout=subprocess.PIPE,
                         close_fds=True, shell=True)

that fixed the problem

# python2 --version
Python 2.7.18`

the question is, why it behaves different on your system

[devZer0]

https://stackoverflow.com/questions/36477552/why-do-i-need-shell-true-here-in-order-for-pythons-subprocess-check-output-to-w

"shell=True obviously opens a shell but, less obviously, it also sets all the environment variables (such as PATH) -"

# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

[devZer0]

https://stackoverflow.com/questions/24306205/file-not-found-error-when-launching-a-subprocess-containing-piped-commands

"The FileNotFoundError happens because - in the absence of shell=True - Python tries to find an executable whose file name is the entire string you are passing in. You need to add shell=True to get the shell to parse and execute the string, or figure out how to rearticulate this command line to avoid requiring a shell."

i guess there must have been some change in python2 that makes it behave differently, or there may be differnt security settings on your machine

[shodanshok]

I am also using Ubuntu 22.04 and I can confirm the issue is related to path indeed: when removing python2, blocksync is executed via the PATH environment. However, removing python2 causes failure when blocksync.py is not in the PATH (eg: it is in a subdir of your home directory`).

I would like to avoid shell=True (due to its implications) and I hope to have fixed the issue in a cleaner manner, can you try with the latest commit applied?

Thanks.

[devZer0]

works find for me now. thanks for fixing !

# borg extract --stdout /mnt/repo2::sda_07_10_2022 | blocksync.py - localhost /dev/sdc

Dry run     : False
Local       : True
Block size  : 128.0 KB
Skipped     : 0 blocks
Hash alg    : sha512
Crypto alg  : aes128-cbc
Compression : False
Read cache  : True
SRC command : /usr/local/bin/blocksync.py - localhost /dev/sdc
DST command : /usr/local/bin/blocksync.py server /dev/sdc -a sha512 -b 131072 -k 0

Synching...
skipped: 0, same: 20555, diff: 3698, 24253/0, 227.9 MB/s