Problems when dealing with large amount of files

[icestraw]

When using the iOS client, I frequently encounter performance problems:

1. When the main app is running, It seems that it is always running "PROPFIND" method to fetch Favorite Folders' data structures. However, once the data grows, the "PROPFIND" method can never be done. Background fetch is okay, but sometimes it makes UI stuck. And, when switching app repeatedly (e.g. using the bottom bar of iPhone X to swipe left and right) from background to the front, it usually tries to reindex from the beginning. Such behaviour can mostly cause the app stuck or crash.

2. When using the File Provider Extension of the client, if I enter a folder with many items, it will always shows "Content Unavailable" error message. As I discover, this usually happens when the PROPFIND result body size is larger than 500KB. According to Apple, the memory usage of the file provider is really limited, and the best practice should be pagination. However, I don't know how to do pagination with webdav method or how hard it is. Is that feasible? If not, can we just reduce the memory usage of enumeration process? Apple's limit seems to be 4MB.

Expected behaviour

Described above

Actual behaviour

Described above

Steps to reproduce

Described above

iOS version

iOS 12.2

App version

2.23.3.4

Server configuration

Operating system:

Ubuntu 64 18.04 LTS

Web server:

Apache2

Database:

MariaDB

PHP version:

7.2

Nextcloud version: (see Nextcloud admin page)

16.0.0 beta 3

[marinofaggiana]

Hi @icestraw,

When the main app is running, It seems that it is always running "PROPFIND" ....

Yes, I have only this possibility for "favorite" &/| "Offline" (for now)

When using the File Provider Extension of the client ....

Correct but for now we don't have a PROPFIND with the pagination, read here:

nextcloud/server#10880

[icestraw]

P1:

Is the recursive folder traversal process necessary? Or just on-demand loading with depth 1

Can the traversal process be faster, no UI stuck or has pause&continue mechanism? Currently it works on main thread and just one request once at a time. (sharedSynchronize->_operationSynchronizeQueue.maxConcurrentOperationCount = 1;)

P2:

Yes we're excited once this server side feature is implemented😀

[marinofaggiana]

@icestraw the webdav driver (old owncloud) is not optimized for run in background mode ... for now this is a big issue.

[cryptosven76]

I can’t even sync if i let the app run in the foreground i have a few folders with ~40k files in them, it never finishes downloading all of them. Does anybody know how to fix this ?
On my laptop (macOS) it downloads everything in minutes , on my iPad it never finishes.

[icestraw]

@cryptosven76
As I know there is no way. Do not sync it on iOS, usually folders like this I cannot even open.

[cryptosven76]

Is there a fix planned for this ? as this bug makes Nextcloud pretty useless for me :-(

[icestraw]

@cryptosven76
Nope, the mobile client is really not stable enough.
I hope maybe iOS 13's SMB feature can be an alternative (actually it's not syncing), but it's still really buggy now ( beta 3 ).

[srkunze]

Is there an update on this?

[nook24]

Will this ever be addressed?

[zeratax]

this still makes nextcloud basically unusable on ios :(