Progress bar

[aarsenca]

Hi,

I tried imgclone today to backup the SD card (16 GB, about 75 % full, 5 partitions) from my Rpi4 (ethernet) to my Win10 laptop (wireless). It took a bit more than one hour. It completed successfully. I did not try a restore.

The feature that I would like to see is some kind of progess bar. In my case, the bulk of the data is on partition 5 and the only message is that imgclone is copying data but it gives no indication that it actually is. I was able to see that the time stamp on the img file (on the laptop) was changing, so I assumed it was being written, but that's all.

It does not have to be a progress bar as such, but some kind of reassurance that it is actually copying the large partition would be nice.

Thanks.

[tom-2015]

I know but didn't have time yet to implement yet.

[tom-2015]

Progress output is now added with the -p command line option.

[aarsenca]

Great! I'll give it a try when I have a chance. Thanks!

[aarsenca]

So I tested the -p option. It prints a basic progress % at some interval. That would be enough for me except that it stopped updating at 18 %, then several minutes later it showed 53 % and then again several minutes later, 81 % and nothing else until it completed. More constant updates (without holes) would be an improvement. Thanks.

[aarsenca]

Just to be sure, I rebooted the Rpi and started imgclone again. This time. the holes were at 18 % and 36 %. It stayed on 36 % until it completed.

[tom-2015]

I've used the same progress code as the original piclone program. There is a delay of 1 - 10 seconds between every update depending on how much data is to be copied. But every update it runs the du command scanning for how much space all files are using in the partition. I think the du command is too slow resulting in jumping from 18 to 53...

[aarsenca]

For the first few minutes, until it reaches 18 %, it updates frequently, actually a few times for each 1 % progress. Then the pattern changes completely as it only updates a few times between 18 % and 100 %. I don't understand how the pattern can change so drasticly. It would be interesting if more people could test it and report how it goes for them.

[tom-2015]

Maybe a lot of small files get copied after 18% slowing down the du command.

[aarsenca]

That could explain it. Again, it would be interesting to read from other users on this issue.

[tom-2015]

I've replaced the du command by df command which seems to work faster but maybe a little less accurate.

[aarsenca]

I will be away for the next few days. I will definitely test it when I'm back home. Thanks!

[aarsenca]

I tested the new code. It is much better. The % complete updated from 1 % to 100 % without any hole. Problem solved.

It may be related to the new code or not, but the run finished with this message:

umount: /tmp/tmp.4eOYzqH18G: target is busy.
Could not unmount partition.

Is there a safe way to unmount this temporary partition?

I should add that I am backing up the Rpi to a folder on my laptop. This is how I start the run:

sudo mount -t cifs //192.168.1.xxx/Users/Me/Rpi4_Backups /media/pi/RPi4_Backups -o username=Me,password=1234567
cd /media/pi/RPi4_Backups
sudo imgclone -p -d RPi4_aammjj.img

Here is the output of: lsof | grep '/media/pi/RPi4_Backups'

lsof 3231 pi cwd DIR 0,39 4096 20547673299996738 /media/pi/RPi4_Backups
grep 3232 pi cwd DIR 0,39 4096 20547673299996738 /media/pi/RPi4_Backups
lsof 3233 pi cwd DIR 0,39 4096 20547673299996738 /media/pi/RPi4_Backups
bash 5067 pi cwd DIR 0,39 4096 20547673299996738 /media/pi/RPi4_Backups

Killing (sudo pkill "process number") the four processes does not solve the issue. The first three processes are assigned a new number and 5067 remains alive.

Thanks!

[tom-2015]

Seems there is still some disk I/O after the copy command finished which makes the umount command fail.
umount should wait until all disk I/O has finished I guess, will try to fix this.

[aarsenca]

That's possible, but in the minutes that followed the run, I tried many times to manually unmount the temporary partition. I always had the same message (target is busy). I only had this message for the last of five partitions, which is by far the largest one.

I ended up rebooting because I could see that this was affecting the CPU load. After the reboot, everything was back to normal and the tmp partition was gone.

[22878120]

May I suggest to show the progress using

sys_printf ("pv %s/. > %s/.", src_mnt, dst_mnt);

instead of

sys_printf ("cp -ax %s/. %s/.", src_mnt, dst_mnt);