Windows 32/64 binaries, 64 bit-HW accelerated

This release contains a large number of new features and therefore, as usual, may contain a new bugs.
So be careful, test on your own that the archived files are truly archived correctly.
Some features are introduced to facilitate my work, that is it's possible to obtain the same result with a combination of various other tools (awk, grep etc), but they are complicated to use and fine-tune.
While the *nix logic is do one thing, and do it well, that of zpaqfranz is the opposite
Do everything I need. Because, often, you don't have the tools you would like (e.g. ESXi, NAS and so on)

Better -stdin management

Now files added with -stdin should be deduplicated as well as those added manually (up to the previous version the deduplicator was less efficient, for a whole series of reasons long to explain)

New comparehex command

This command performs an operation that happens very frequently, namely comparing two lines (typically HASH codes) present in two files. If the codes are the same it exits with OK. You can also impose the expected length of the hash.
Attention, all non-HEX characters are discarded

Example:

zpaqfranz comparehex z:\1.txt z:\2.txt "GLOBAL SHA256:" 64

New count command

Counts within more than one file the occurrences of a certain string, and returns OK if the number is the expected one In essence, within a log file, it counts how many positive results there are.
If you don't select a string to search for, it will use the default one for OKs with the -big switch

Example:

zpaqfranz count z:\*.txt 3 "all OK"
zpaqfranz count z:\*.txt 10

Better Control-C handling

During program termination with Control-C press, a better management of the deletion of files created with -chunk is performed.
It might also perform rollback of .zpaq files, maybe in the future.
Apparently it should work in a portable way across various platforms, or at least I hope so.
The problem of terminating a multithreaded system based on a condition is not trivial, even from a performance perspective. I used "dirty tricks", we'll see in the future.

gettempdirectory

Improvement in the creation of temporary files, for the use of multiple zpaqfranz processes simultaneously
Temporary files are created inside different subfolders (marked with mtime) in order to reduce the risk of "collision" if
multiple zpaqfranz are launched simultaneously.

new work command with many verbs

Used to automate many convenient functions for backup logs.
Maybe for most users they are useless, but they make my work a lot easier to prepare easily readable log files.

Write something big                  work big "count the ok"
Write something big                  work big "COUNT THE OK"
Write something big                  work big forza inter
Pad number 123 to 00000123           work pad 123
Pad number 123 to 0123               work pad 123 -n 4
Extract path                         work filepathnotrailing z:\doc\2\3.eml
Write datetime                       work date -terse
Write year_month_day                 work date "%year_%month_%day" -terse
Write -----                          work printbar -terse
Write !!!!!                          work printbar "!" -terse

Newer -crc32 switch for t (test)

Run a triple CRC-32 check (!) against the filesystem
Use -find/replace to fix path (if needed); -ssd for M/T

Translation

The t command (without a path) do NOT read something from filesystem
You have to use the t command WITH a path (or use the -verify switch)

The brand new -crc32 switch is makes a triple CRC-32 comparing, READING from the filesystem.
In this case, the -find and -replace switches can be useful to transform the stored paths into physical paths (if they are different)

What's so exciting about it?
The first CRC-32 is the one calculated by the filesystem during the file addition phase
The second CRC-32 is the one RE-calculated from the blocks of the compressed file.
In other words, it's the CRC-32 of the file once extracted and written to the filesystem
The third CRC-32 is the one that is re-read from the filesystem

If these three CRC-32s match, it means that the file (during processing) has not been modified, and that the compressed file once decompressed will have the same CRC-32 that the file now has in the filesystem.
In short, that by DECOMPRESSING it will be correct

Obviously it is NOT equivalent to the p command (paranoid), to t -paranoid and not even to w, where files are really extracted.
But it's enormously faster, and can process in multithread with the -ssd switch

Triple check CRC-32:                 t z:\\1.zpaq -crc32");
3 check w/path manipulation:         t z:\\1.zpaq -crc32 -find \"x:/memme/\" -replace \"c:/nz/\"");

New -terse switch

Almost everywhere it transforms the output into a version without additional information
That is, it shows the lines typically for redirection to other programs.
If you think about the difference in output between the ls command on Linux and dir on Windows, it will be immediate.

New -csv and -csvhf switches for l (list)

Using these switches you can get an output more easily processable with other programs
In other words, if you want to extract the list of files in a zpaqfranz archive, for example to load it into an Excel file or wherever you want, you can use these switches, using \t instead of TAB.
Keep in mind that -csvhf stands for "csv header footer", that is any string indicated at the beginning and end

For example pippo pluto paperino can become "pippo","pluto","paperino"
Otherwise it would remain pippo","pluto","paperino

TAB-delimited CSV output             l z:\1.zpaq -terse -csv "\t"
Enclose fields between double quote  l z:\1.zpaq -terse -csv "\",\"" -csvhf "\""

-external in add

It is now possible to specify a command that is executed BEFORE the add function, save its output to a text file, and automatically add it to the archive.
It can be used for activities such as creating .zfs snapshots, but also - and above all - to perform an independent check, for truly paranoid people

Translation

Let's say you want to have a... squared certainty that your files are truly identical to the original ones (we are in the hypothesis of using some remote backup system, so you CANNOT compare the files with the local filesystem, because they are not there)

But zpaqfranz can have bugs, right?

You can use zpaqfranz for a long time with the -hashdeep switch, which will generate a file in the
hashdeep format
But it's still using zpaqfranz: any bugs could still affect its reliability
For delicate situations we use an external program, which therefore doesn't have the possible bugs of zpaqfranz, such as hashdeep (in the Windows version hashdeep64 is faster)

Now we will store in a virtual file (of zpaqfranz) VFILE-l-external.txt the list of sha256 hashes generated by hashdeep (or any other similar program).
In this case the string %files and $files is replaced with the files specified in zpaqfranz (they are different for Windows and Linux).
In this example, the command will be executed
c:\nz\hashdeep64 -r -c sha256 c:\nz >temporaryfile.txt
The temporary file will be compressed as usual and added to the version

zpaqfranz a z:\2.zpaq c:\nz -external "c:\nz\hashdeep64 -r -c sha256 %files"

Now let's send the .zpaq file to another computer "somehow".
Let's suppose now we extract the c:\nz folder (object of the example) into the z:\memme folder

zpaqfranz x z:\2.zpaq -to z:\memme

We will now find in the z:\memme folder a folder c (containing the content of the files) AND also the VFILE-l-external.txt file, which contains the output of hashdeep
Now we can use hashdeep "by hand" to test the identity of the stored hashes with those of the folder
If we are not so paranoid there is the zpaqfranz versum command that understands the hashdeep format and allows to manipulate paths
So pay attention to -find and -replace

zpaqfranz versum z:\memme\VFILE-l-external.txt -hashdeep -find "c:\nz" -replace "z:\memme\c\nz" -ssd

-external in the x extract command

You can directly extract the virtual external file, to have an output to redirect

Show external VFILE                  x z:\2.zpaq -external -silent

-silent, in this example, disables the output of zpaqfranz. You can then, if needed, easily redirect for example >mygoodoutput.txt

Various minor fixes

Increased the maximum number of versions shown by the i command

-symlink in add (on Windows)

Ignores symlinks completely. The magical world of NTFS is so complex that every now and then
a more or less unknown functionality pops up. Anyway for one of my clients I need to ignore them, so I do it. It will probably be evolved in the future.

Always show output if there's the -big switch

By limiting the output with various switches the final OK output (gigantic) is still written
It's for the recognition (with the count command) of logs that went well
In essence, it reduces the verbosity of the output, but maintaining the final result

Execution dates are stored in backups

And the testbackup command shows the last one (if it exists)
This way you can understand if an automated backup procedure works
I had to use very dirty tricks to maintain backwards compatibility with older zpaqfranz, but it should work

newer franzomips CPU

Added some new CPUs to the franzomips list

It's possible to set file dates with -touch X

During the a add phase, you can force the timestamp of added files with the X switch
where X represents a date or a date and time
It also works for files added with -stdin