add zip plugin

[Darukutsu]

I don't like that built-in zip includes absolute paths by default. I also miss zstd as option. This could do probably any
compression. You can also password protect archive.

Related issues #1105.

[N-R-K]

You can generally just pipe the output of tar over to the compressor program without needing to create any intermediate file.

[N-R-K]

Why not just pass the full filename via argument instead of doing this. Also gpg might accept stdin as well. So you might be able to just pipe it in and avoid the intermediate file once again.

[KlzXS]

I only marked a single occurrence of each issue. Please change all of them.

[KlzXS]

CI is flagging this. Use "$alg".

[Darukutsu]

There are more variables should I put "" for all of them?
Also is it ok to amend commit? Don't know what's proper way when fixing issues like that.

[KlzXS]

Please do, I missed those. It'll keep complaining until all of them are quoted. In general, if it has $ it goes between "".

You can add more commits, amend, force push, whatever you like. PRs usually just get squashed anyway.

[KlzXS]

You know how every rule has an exception? This is it for the quoting rule: $(tr '\0' '\n' < "$selection") shouldn't be quoted. We want that to split at whitespace.

[Darukutsu]

what if filename has white space?

[Darukutsu]

anyway selection should only contain selected files nothing else right? So doesn't matter

[Darukutsu]

Ah, for loop isn't great for this since it will break if filename contains \n.

[Darukutsu]

well I will have to come with other solution or maybe I'm not right and already talking bullshits :D.

[N-R-K]

Just do it with xargs -0 and add a alg_to_extension() function or something to add the extension.

[KlzXS]

Yeah, it would be best if you can get it to work with xargs -0. Just keep in mind that you can't run functions inside xargs. That's a bash extension.

[Darukutsu]

Well boys, yesterday I tried many things like perl's sed and creating newfiles but that didn't seem ok to me. I come with this solution. Will this work in other shells? I tried to preserve original behavior, but it's kinda weird. I mean it works for me :D. I tested it for dirs and files with/without space/newline. I could omit something. Maybe I should include some tests.

I could possibly rewrite it in way in which I specify file extensions in choose_alg() and then use native tar --use-compress-program=COMMAND.
Or let user manually everytime specify extension in input like myarchive.tar.gz.

It works now in a way that users need to specify only name without extensions.

[jarun]

@N-R-K and @KlzXS is this ready to merge?

[KlzXS]

@jarun There are still some outstanding comments from me that should be addressed. CI is currently failing; I'd say we probably just silence it in this case. I myself haven't tried the plugin out; will probably do that at some point.

When those issues are address and if we confirm that it does work, we'll be ready to merge. Personally, I'd like to see the script simplified a bit, there's a lot of commands nested inside of xargs ... sh -c '' constructs, but I haven't had time to look into it and see if something can be done about that.

[Darukutsu]

There are some issues. I finally made somewhat testing script, so now it will be easier to fix them. I will provide script with fix tomorrow.

[Darukutsu]

When testing using script comment line 229. You still need to make sure output is ok. I tested using this.

  # Clear selection
  #printf "-" > "$NNN_PIPE"

test script

#!/usr/bin/env sh

g=$(tput setaf 2)
n=$(tput sgr0)

command=~/.config/nnn/plugins/zip

#tar
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rY\rY\rY\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 1${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rY\rY\rN\r\r"))& $command printf "\n%s\n\n" "${g}tar passed 1${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rY\rY\rN\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 2${n}" ; read&& printf "\n%s\n\n" "${g}tar passed 2${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rY\rN\rN\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 3${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rY\rN\rY\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 4${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rN\rY\rN\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 5${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rN\rY\rY\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 6${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rN\rN\rY\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 7${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "tar\rN\rN\rN\r\r"))& $command && printf "\n%s\n\n" "${g}tar passed 8${n}"

#tar.gz
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rY\rY\rY\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 1${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rY\rY\rN\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 2${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rY\rN\rN\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 3${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rY\rN\rY\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 4${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rN\rY\rN\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 5${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rN\rY\rY\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 6${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rN\rN\rY\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 7${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "gzip\r\rN\rN\rN\r\r"))& $command && printf "\n%s\n\n" "${g}gzip passed 8${n}"

#zip
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rY\rY\rY\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 1${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rY\rY\rN\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 2${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rY\rN\rN\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 3${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rY\rN\rY\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 4${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rN\rY\rN\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 5${n}" ; read
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rN\rY\rY\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 6${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rN\rN\rY\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 7${n}" ; read; read
(sleep 1 ; xdotool type --delay 100 $(printf "zip\r\rN\rN\rN\r\r"))& $command && printf "\n%s\n\n" "${g}zip passed 8${n}"

[Darukutsu]

Should I fix anything else?

[N-R-K]

Did some basic tests (didn't test encryption). Couple remarks:

0. Why does compressing a single file produce a tar? I'd expect it to be compressed directly without the tar archive.
1. Should we really hardcode 5 as default compression level? Each tool comes with it's own default and IMO we should let the tool decide for itself what the default should be instead of hardcoding 5.

[N-R-K]

Also, I'm really not a fan of this type of thing that's going on: -o -name "$filename.tar*". All this would go away if we just used stdin/out instead of making intermediate files. Here's an example of how to do stdin/out pipe to do tar.gz on selection:

$ xargs -0 tar -cvf - < "$selection" | gzip -c -9 > archive.tar.gz

Or if you want to remove the $pwd from the paths:

sed -z "s|^$PWD/||g" < "$selection" | xargs -0 tar -cvf - | zstd -c -19 > archive.tar.zst

EDIT: and if you don't want to refactor things to be stdin/out based then leave a TODO comment at the top about this.

[Darukutsu]

sed -z "s|^$PWD/||g" < "$selection" | xargs -0 tar -cvf - | zstd -c -19 > archive.tar.zst

does this work because after trying it i got something like this which is not what we want.

# run under .config/nnn
" tar.vim version v32
" Browsing tarfile /home/daru/.config/nnn/archive.tar.zst
" Select a file with cursor and press ENTER

plugins/x2sel
plugins/xdgdefault
plugins/zfs-snapdir
sessions/
sessions/@                              # my implementation does this
home/daru/.config/pgcli/                #pgcli/
home/daru/.config/pgcli/config          #pgcli/config
home/daru/.config/pgcli/log             #pgcli/log

also doing this piping to gpg and thus creating another elseif entry isn't very something... or maybe gpg should be separate script then, or don't include it at all.

Should we really hardcode 5 as default compression level? Each tool comes with it's own default and IMO we should let the tool decide for itself what the default should be instead of hardcoding 5.

As a workaround to prevent including more if-statements we could do "default_compression=-verbose since expanding it will make --verbose and it's unharmful behavior and should be same across all compression algorithms.

[jarun]

@Darukutsu CI is failing, can you check?

Also, this is open for a while, if this doesn't move, I'll close it.

[Darukutsu]

 ^-- SC2016 (info): Expressions don't expand in single quotes, use double quotes for that.

just like @KlzXS said

CI is currently failing; I'd say we probably just silence it in this case.

I couldn't come up with better solution. It's pretty much working for me(haven't spotted any issue, when using/testing), but that's not what means "production ready". Feel free to close. In either case thank you guys for perfectioning this script.

[KlzXS]

The main issue I have with the script is that it's a nightmare to maintain with all those nested substitutions, extracting and constructing names and a lot of ifs that run in parallel across multiple functions. Unfortunately, I don't have a solution to those issues.

If you're fine with such code merge it in.

@Darukutsu Thank you for giving it a shot. Personally, I'd have written this in something like Python as bash isn't really great at dealing with complex logic. You get ugly things like that find for removing files after encrypting them. Ideally you should know the name of the temporary file you just wrote to, but alas, you don't.

We don't really have examples of non-shell scripts, but a plugin can be any old executable file, we just happen to mostly do it in sh as that's available everywhere.

[KlzXS]

I think it's best that we close this for now. I've added it as a task in the Tracker. Someone can give it a shot in the future.

My suggestions for those attempting this task:

• Don't use POSIX sh. This is a complicated task and it's very limited in its functionality. At the very least use bash so that you can execute a function inside xargs. That should simplify the script significantly and make it easier to modify and maintain. Functions are your friends.
• You can try another scripting language but try to use something widespread and popular. I've mentioned Python above.
• If you try to support encryption, see if you can support both symmetric and asymmetric gpg. See gpge and gpgd for an example.
• If using gpg, see if you can remove the intermediate files directly, without having to resort to finding them after gpg's done its job.
• In general, try to make the script as performant as possible. It should spend most of its runtime doing useful work and as little as possible executing helper functions.
• nnn's core archiving solution has support for atool. See if you can use that to simplify the task.
• Consider if you want to support actions for extracting (Extract Here, Extract to..., Extract to archive_name/) available in many archiving solutions. Splitting that off into a separate plugin would be a good choice.

Now that I've written this it sounds more like a whole separate tool instead of a "simple" plugin. Make of that what you will.

All in all, this is functionality that would be nice to have in nnn in some form. Thanks to @Darukutsu for bringing it up.