Fixed a problem with file names when moving from external storage to nextcloud (with ObjectStore S3 enabled)

[hopleus]

Summary

Fixed a problem with file names when moving from external storage to nextcloud (with ObjectStore S3 enabled)

Use case

The file is copied from the external storage to the local one (extDisk: test.docx => localDisk: test.docx.part)
An entry is created about a new file (test.docx.part) in the database, the oc_filecache table. The new file contains a file id equal to the one that is saved as the filename in S3
Renames the file in the local storage from test.docx.part to test.docx
The information in the database is updated (new path, new storage ID) to record the source file (the one that we move from the external storage)
, the record in the database about the new (temporary) file (test.docx.part) is deleted.
Result: The old file id remains in the database, but with new information (path, name, storage). And S3 stores a file with a temporary ID that has not been changed after all the manipulations.

Checklist

• Code is properly formatted
• Sign-off message is added to all commits
• Tests (unit, integration, api and/or acceptance) are included
• Screenshots before/after for front-end changes
• Documentation (manuals or wiki) has been updated or is not required
• Backports requested where applicable (ex: critical bugfixes)

[hopleus]

/backport stable28

[hopleus]

/backport stable27

[hopleus]

/backport stable26

[hopleus]

@joshtrichards Hi. Could you run the tests again?

P.S. Last time cypress crashed with 503 errors and I didn't understand why

[icewind1991]

I don't think this is the correct approach to solve the issue, it ends up creating a cache entry for the .part file that isn't used, it also impacts copies, where it will write to the wrong urn. And the method of passing the fileid to writeStream is very fragile to other changes.

The best I can think of is having ObjectStoreStorage overwrite moveFromStorage where it can then write the files to the correct urn without making new cache entries for them and instead updating the existing source cache entries to move them to the new storage.

[icewind1991]

Great job with figuring out the root cause of the issue though!

[hopleus]

I don't think this is the correct approach to solve the issue, it ends up creating a cache entry for the .part file that isn't used, it also impacts copies, where it will write to the wrong urn. And the method of passing the fileid to writeStream is very fragile to other changes.

The best I can think of is having ObjectStoreStorage overwrite moveFromStorage where it can then write the files to the correct urn without making new cache entries for them and instead updating the existing source cache entries to move them to the new storage.

I've changed the logic, check when the opportunity arises

[icewind1991]

I can confirm that this solves the problem but the extra copy added will cause issues.

I've created an alternative approach that forgoes the default copyFromStorage logic to ensure it writes directly to the correct urn: #46013

Please verify that it resolves the issue.

[juliushaertl]

I think the alternative from @icewind1991 looks a bit safer, could you double check if that also resolves your issue?

[kesselb]

Thank you very much @hopleus 🙏

The alternative approach was just merged.

Thanks again for bringing it to our attention and your thorough analysis of the root cause.