File timestamps are not preserved on upload anymore

[danielb42]

⚠️ Before posting ⚠️

• This is a bug, not a question or an enhancement.
• I've searched for similar issues and didn't find a duplicate.
• I've written a clear and descriptive title for this issue, not just "Bug" or "Crash".
• I agree to follow Nextcloud's Code of Conduct.

Steps to reproduce

1. Upload a file with Nextcloud/Android v3.22.1.

Expected behaviour

Timestamps of a file (i.e. creation date/time) should be preserved when uploading.

Actual behaviour

Since the recent update, files uploaded to the server have the upload time set as creation timestamp, erasing the original creation metadata. This is bad, and only one of the more innocuous consequences is having "sort by newest" display files uploaded at the same time in a shuffled way..

Android version

12

Device brand and model

Galaxy A41

Stock or custom OS?

Stock

Nextcloud android app version

3.22.1

Nextcloud server version

24.0.6

Using a reverse proxy?

Yes

Android logs

No response

Server error logs

No response

Additional information

No response

[Byter3]

I concuer, I want to bump this too. It just caused me hedache with around 500 images now all over the timeline.

[AlvaroBrey]

I cannot seem to reproduce this; the files I upload retain their creation timestamp as reported by the server. Can you tell me exact steps (as in, how are you checking the timestamp, etc)?

[Byter3]

@AlvaroBrey

I have set up the Nc client on my phone to auto upload, the photos taken with the phone, but only when the device is on Wifi. So in the evening, when I got back home, it started upload the photos. Now my photos are all over the place in terms of time, if I check them on Nextcloud webgui, there is no order to the photos. ie, pictures taken in the dark is behind pictures when the sun was still up, so it indivisble, that the timestamps got confused.
Also a I had tremendeaus issues with the new client, while it started the auto upload, the client crashed multiple times. And restarted the upload multiple times. It wasn't visible which file is currently being uploaded. The progress bar in the notification area poped up and dissappeared in fraction of a second. I never seen, such a strange behaviour with older version of the Nextcloud clien app on my phone.

So maybe this strange behaviour causing the time stamp issue(?)

Also I just wan't to flag it here as well since you closed my other topic, that the new version of nextcloud app, drains an ungodly amount of my battery. This wasn't the case with older version of the app.

[danielb42]

Hi @AlvaroBrey, sure, but I'm afraid there are no special steps involved. My NC webinterface, and also the files downloaded through NC desktop client simply carry the upload timestamp now instead of the real creation timestamp..

So, for a visual example: Yesterday I took a bunch of photos outside, say between 14:00 und 17:00. Those were not uploaded immediately as "WiFi only" is checked. When I got home at 18:23 and my phone connected to my WiFi, all the pics were uploaded through regular auto-upload, nothing special. Up to version 3.22.1, each file would have the original creation timestamp (anything between 14-17h) and this would show correctly in NC webinterface and also when downloading the files onto a computer. And now.. all files pretend to be created at 18:23, as shown in webinterface as well as after downloading.

[AlvaroBrey]

Ok, so both of you are using autoupload and uploading later when connected to wifi, that gives me some clues. Can you check if a manually uploaded file preserves timestamp correctly?

[danielb42]

Negative, manual uploads carry upload timestamp as well.

Inspecting on the server directly.. original timestamp of the file (on my phone) is as indicated in its filename (Jul 2020). Manual upload was (today) 18:38.

# stat 20200704_175548.jpg
  File: 20200704_175548.jpg
  Size: 2976631         Blocks: 5816       IO Block: 4096   regular file
Device: fd02h/64770d    Inode: 5514201     Links: 1
Access: (0644/-rw-r--r--)  Uid: (   33/www-data)   Gid: (   33/www-data)
Access: 2022-10-24 18:38:31.777826633 +0200
Modify: 2022-10-24 18:38:28.000000000 +0200
Change: 2022-10-24 18:38:31.733827167 +0200
 Birth: -

[AlvaroBrey]

Dang, that makes it harder for me, as I was testing manual uploads and they were preserving the correct timestamps. Will have to dig further.

[AlvaroBrey]

Does the nextcloud app have the All Files permission, or does it have the media read-only one?

[AndyXheli]

@AlvaroBrey i can also confirm this. Auto upload or no auto upload same issue. When i click in update files and i select a few photos that i took yesterday once the files upload to the server the photos say modified today at the time I uploaded the files but they should not say that it should keep the date and time when the file was taken on the phone etc. I tired the same steps via iOS nextcloud app and it kept the time and data

Test Senario

1. Open Nextcloud Android App
2. Click on + icon
3. Select Upload content from other apps
4. Select Samsung Gallery
5. Select 5 to 10 photos that where taken a few days back
6. Upload them to server
7. Open Server portal go into the folder and check the file dates they say a few seconds ago "but it should say the date and time the file was taken. "That how its working under Nextcloud iOS

I also noticed that the app keeps looping on the files. the file are already uploaded and it keeps wanting to sync them again. I can send you a video if you ping me on cloud nextcloud com chat

[danielb42]

Does the nextcloud app have the All Files permission, or does it have the media read-only one?

It has All Files permission.

[Byter3]

Does the nextcloud app have the All Files permission, or does it have the media read-only one?

It has all the permissions it can get.
Also didnát mentioned the phone: Samsunbg Galaxy S21 Ultra, Android 12

[danielb42]

btw @ people suffering from this, here's a quirky little bash snippet to fix the timestamps according to the filenames.
Assuming photos are named YYYYMMDD_hhmmss.jpg, as Samsung DCIM does for example.

cd MyNextcloudPhotoFolder

for PIC in $(ls -1 ????????_??????.jpg); do
  Y=${PIC:0:4}
  M=${PIC:4:2}
  D=${PIC:6:2}
  h=${PIC:9:2}
  m=${PIC:11:2}
  s=${PIC:13:2}
  touch -m -t $Y$M$D$h$m.$s $PIC
done

[hbfrank]

This bug is caused by the new (wrong) logic of upload action, i.e., make a copy of the local file and then upload the copy whose lastModificationTimestamp is of the time-of-copying.

---

BTW, the OC_X_OC_MTIME_HEADER of upload file in library/src/main/java/com/owncloud/android/lib/resources/files/ChunkedFileUploadRemoteOperation.java:

moveMethod.addRequestHeader(OC_X_OC_MTIME_HEADER, String.valueOf(file.lastModified() / 1000));

is different from in its super class library/src/main/java/com/owncloud/android/lib/resources/files/UploadFileRemoteOperation.java.

putMethod.addRequestHeader(OC_X_OC_MTIME_HEADER, lastModificationTimestamp);

so this bug only appears on files uploaded by ChunkedFileUploadRemoteOperation, i.e., file size larger than the CHUNK_SIZE_WIFI (10240000 bytes) or CHUNK_SIZE_MOBILE (1024000 bytes).

[AndyXheli]

Same

{"reqId":"kiUGDj41laPvPNM5btA7","level":3,"time":"2022-11-07T02:24:43-06:00","remoteAddr":"15.1.5.3","user":"admin","app":"webdav","method":"PUT","url":"/remote.php/dav/files/admin//Documents/Backups/.backupd741f552-6192-4f35-92b2-a85359a87c2e.tmp","message":"X-OC-MTime header must be a valid positive integer","userAgent":"Mozilla/5.0 (Android) Nextcloud-android/3.22.3","version":"25.0.1.1","exception":{"Exception":"InvalidArgumentException","Message":"X-OC-MTime header must be a valid positive integer","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Node.php","line":424,"function":"sanitizeMtime","class":"OCA\\DAV\\Connector\\Sabre\\MtimeSanitizer","type":"::"},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":378,"function":"sanitizeMtime","class":"OCA\\DAV\\Connector\\Sabre\\Node","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php","line":151,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1098,"function":"createFile","class":"OCA\\DAV\\Connector\\Sabre\\Directory","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":504,"function":"createFile","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":360,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/nextcloud/remote.php","line":171,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/MtimeSanitizer.php","Line":37,"message":"X-OC-MTime header must be a valid positive integer","exception":[],"CustomMessage":"X-OC-MTime header must be a valid positive integer"},"id":"636922759cf5e"}

[AlvaroBrey]

Thanks for the research @hbfrank ! That timestamp is indeed incorrect.

However, I still can't reproduce this even with chunked files; I guess it's a matter of whether the specific device changes the modification date when copying the file or not.

In #11007, there should soon be a QA APK with this patched. Can any of the affected users use it to verify it's fixed? thanks!

[AlvaroBrey]

Ping for affected users, especially @danielb42. APK is here: #11007 (comment)

Since I can't reproduce it, and the bug in the code was found and patched, if I don't see any reports of the contrary I'll consider this fixed when that PR is merged.

[AndyXheli]

@AlvaroBrey Just tested the APK and it looks good to me. If you use Signal on your phone setup backups to a nextlcoud folder and i think you'll be able to reproduce it. Every time my Signal app dose a backup i get the error logs i changed to folder dir to point to the QA app and did a backup no errors under server logs

[danielb42]

@AlvaroBrey looks good!

# stat 20221109_182511.jpg
  File: 20221109_182511.jpg
  Size: 1719793         Blocks: 3360       IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 7209167     Links: 1
Access: (0644/-rw-r--r--)  Uid: (   33/www-data)   Gid: (   33/www-data)
Access: 2022-11-09 18:30:21.774251583 +0100
Modify: 2022-11-09 18:25:15.000000000 +0100
Change: 2022-11-09 18:30:21.738252122 +0100

Everything (timestamp-wise) as it should be.