Nextcloud Client not Chunking (uploads over 100 MB fail)

[metheis]

Expected behaviour

The Nextcloud Client should chunk large files to 10MB so that large files can be uploaded even when the max upload size is smaller than the file size.

Actual behaviour

When placing files larger than 100 MB into a synced folder, I am hitting cloudflare's 100 MB upload limit, even though I expected the nextcloud client to chunk requests to 10 MB in size per request. Cloudflare allows for uploads up to 100 MB per request. This issue has only happened over the past month or two.

Steps to reproduce

1. Setup a nextcloud server with a domain in Cloudflare's network. Set the DNS record to proxy through Cloudflare
2. Setup the nextcloud client and connect to the server
3. Try to upload a file larger than 100 MB, and the upload will fail due to hitting the upload limit.

Client configuration

Client version: 3.4.1

Operating system: Windows 11 22000.493

OS language: Englush (US)

Installation path of client: C:\Program Files\Nextcloud

Server configuration

Nextcloud version: 22.2.3

Logs

Please use Gist (https://gist.github.com/) or a similar code paster for longer
logs.

1. Client logfile:
   The file I tried to upload is: mt_20220203.tar.gz
   https://gist.github.com/metheis/d5e59e705f501438af82cf9ebfe2f7b1

2. Web server error log:

3. Server logfile: nextcloud log (data/nextcloud.log):

[metheis]

Keeping all else the same, I tried using an older client (3.3.6) and I'm able to upload large files. This definitely seems to have been introduced with recent client versions

[metheis]

Merging this issue with #4278

[metheis]

Fix:

Hi everybody, thanks for finding this. I found simply setting the maximum chunk size to 50 MB (half of Cloudflare's 100 MB upload size limit) worked to resolve this issue.

I put together a short guide to fix this issue with the latest stable release (3.4.4, but should work on any client v3.4+). I tried to make it as accessible as possible to follow.

Windows Fix

Press Win+R on your keyboard to open the Run application. Past the following in the dialog box:

%APPDATA%\Nextcloud\nextcloud.cfg

This will either ask you to pick an application to open nextcloud.cfg or will open in your default text editor (unless you have something else set to open .cfg files). If it asks you to pick an application, feel free to use Notepad or any other editor.

Add the following line under the [General] section:

maxChunkSize=50000000

Save the file, quit Nextcloud desktop, and start it again.

MacOS Fix

Open a Finder window and press Command+Shift+G on your keyboard. This will bring up a 'Go to folder' window. Paste the following in the dialog box:

$HOME/Library/Preferences/Nextcloud

Open the nextcloud.cfg file. If you do not have a default editor for .cfg files, feel free to open the file with TextEdit.

Add the following line under the [General] section:

maxChunkSize=50000000

Save the file, quit Nextcloud desktop, and start it again.

Linux Fix

Open a terminal window and edit the following file:

nano $HOME/.config/Nextcloud/nextcloud.cfg

Add the following line under the [General] section:

maxChunkSize=50000000

Save the file (Ctl+o, Ctl+x), then quit Nextcloud desktop, and start it again.

[cainnz]

@metheis hello there, I am running nextcloud in a docker container and behind a nginx proxy manager via Cloudflare. Question: where do I find the nextcloud.cfg inside nextcloud docker? I look everywhere but I cannot find it. I also look at my mounted "config" volume and cant find the file, if not, where should I entered the "maxChunkSize=5000000" so files above 100MB gets uploaded? thanks in advance

[brian8544]

@metheis hello there, I am running nextcloud in a docker container and behind a nginx proxy manager via Cloudflare. Question: where do I find the nextcloud.cfg inside nextcloud docker? I look everywhere but I cannot find it. I also look at my mounted "config" volume and cant find the file, if not, where should I entered the "maxChunkSize=5000000" so files above 100MB gets uploaded? thanks in advance

In case you still have this issue; since it's a client-side configuration option, for Windows it will be in: (%AppData%\Roaming\Nextcloud\nextcloud.cfg).

[EnderArchery]

I'm running into this problem again with nextcloud-client 3.6.0

[MilesTEG1]

Hello,
I have this issue too, with macos client 3.8.2

[Potato-007]

thank you for the fix, it's still a problem with the default config using macOS client 3.10.81

[primetime-sudo]

Fix:

Hi everybody, thanks for finding this. I found simply setting the maximum chunk size to 50 MB (half of Cloudflare's 100 MB upload size limit) worked to resolve this issue.

I put together a short guide to fix this issue with the latest stable release (3.4.4, but should work on any client v3.4+). I tried to make it as accessible as possible to follow.

Windows Fix

Press Win+R on your keyboard to open the Run application. Past the following in the dialog box:

%APPDATA%\Nextcloud\nextcloud.cfg

This will either ask you to pick an application to open nextcloud.cfg or will open in your default text editor (unless you have something else set to open .cfg files). If it asks you to pick an application, feel free to use Notepad or any other editor.

Add the following line under the [General] section:

maxChunkSize=50000000

Save the file, quit Nextcloud desktop, and start it again.

MacOS Fix

Open a Finder window and press Command+Shift+G on your keyboard. This will bring up a 'Go to folder' window. Paste the following in the dialog box:

$HOME/Library/Preferences/Nextcloud

Open the nextcloud.cfg file. If you do not have a default editor for .cfg files, feel free to open the file with TextEdit.

Add the following line under the [General] section:

maxChunkSize=50000000

Save the file, quit Nextcloud desktop, and start it again.

Linux Fix

Open a terminal window and edit the following file:

nano $HOME/.config/Nextcloud/nextcloud.cfg

Add the following line under the [General] section:

maxChunkSize=50000000

Save the file (Ctl+o, Ctl+x), then quit Nextcloud desktop, and start it again.

This was the fix for me in March 2024, Windows client 3.12.3

[Doooder]

Fix:

.....

MacOS Fix

Open a Finder window and press Command+Shift+G on your keyboard. This will bring up a 'Go to folder' window. Paste the following in the dialog box:

$HOME/Library/Preferences/Nextcloud

Open the nextcloud.cfg file. If you do not have a default editor for .cfg files, feel free to open the file with TextEdit.

Add the following line under the [General] section:

maxChunkSize=50000000

Save the file, quit Nextcloud desktop, and start it again.

I cannot seem to find the nextcloud.cfg on any of my Macs. They are Apple Silicone if that makes a difference. So if anyone knows where I can fine this setting on my Mac it would be much appreciated!

[Doooder]

Ok, I'm just gonna put this here in case someone else with a Mac comes here with the same question.

Viewing the hidden ~/Library folder

• The hidden '~/Library' folder is located in the 'Finder's' top menu.

• To view these, go into 'Finder' and click on 'Go' on the menu bar.

• Next, hold down the 'Option' key on your keyboard.

• You'll now see the hidden '~/Library' listed as 'Library' on the' Go' menu.

• Click on it to see the content.

[S1ngl3-x]

It's still an issue on linux, but works fine on android.

The upvoted answer rescued me after 3 days of blind troubleshooting.

[S1ngl3-x]

It seems like it can be fixed on the server side for all clients as well. Add line to config.php

'max_chunk_size' => 78643200, // Set to approximately 75MB

So far all good on my side.

[boray-tw]

@metheis hello there, I am running nextcloud in a docker container and behind a nginx proxy manager via Cloudflare. Question: where do I find the nextcloud.cfg inside nextcloud docker? I look everywhere but I cannot find it. I also look at my mounted "config" volume and cant find the file, if not, where should I entered the "maxChunkSize=5000000" so files above 100MB gets uploaded? thanks in advance

Thanks @cainnz for the clue of Cloudflare.

Cloudflare stated that the maximum body size limit behind Cloudflare proxy has been 100MB for free tier since 2020 or earlier. The upload limit for each tier is as follows.

Free	Pro	Business	Enterprise
100 MB	100 MB	200 MB	500 MB (default)

To bypass the limit, @metheis's solution still works on Nextcloud client 3.13.3.

Another solution is to unproxy DNS records that point to Nextcloud server(s).

By the way, if someone cannot upload files larger than 200 MB or 500 MB, they may hit the limit of Cloudflare as well.

[hermeneuticlens]

Can confirm this is still an issue for Linux and Windows. Deployed NextCloud Server official Docker. With default settings, any files over 100MB fail to upload, server says "files not found", which is a very unintuitive error message. Added the lines to the local config file and the issues are fixed for both Linux and Windows.

[S1ngl3-x]

Perhaps conceptually correct solution would be to expose server ENV Variable "max_chunk_size"? Where documentation would mention it's needed to change it to a specific value for some proxies like cloudflared=78643200?

[Permanently]

Alternatively, there could just be an option under the Advanced section under Nextcloud settings - seems a bit too common of an issue to be having text configuration files as the solution

[punkyard]

maxChunkSize=50000000 is not fixing it for me

[Potato-007]

maxChunkSize is only one of many things that needs to be adjusted.

you have to continue editing the php config on computer running nextcloud server!

in my case on macOS this is the path, but it will be differ on yours

sudo pico /usr/local/etc/php/8.3/php.ini

...change
memory_limit = 512M
upload_max_filesize = 16G
post_max_size = 16G

...if using opcachen, then uncomment and change from 8 to 64
opcache.interned_strings_buffer

[punkyard]

hi @Potato-007
and thank you so much for your reply

there is no web server installed on my local machine (macos monterey) so there is no php in /usr/local/etc
my NC instance is online on debian, i'm using NC desktop and it won't sync files bigger than 10Mb

[Potato-007]

the reason for your trouble @punkyard is that php is not configured properly - as in my previous post on server side.

php is not in use on client side. contact the admin who manages the server

[punkyard]

hi @Potato-007
I'm the admin and we are using AiO latest 960
do you mean my .yaml is not well configured?

or Shall I warn the AiO about this?

[Potato-007]

I have no experience with the AIO version of Nextcloud (I'm using a manual installation on macOS) but properly configured PHP is essential for big file upload, and Nextcloud is based on php. I hope you find a way to configure it and then share the steps here with others

chatgpt says:

Nextcloud AIO (All-in-One) is designed for a simplified Nextcloud installation and management, but it doesn't offer detailed configuration options like custom PHP settings out of the box. However, Nextcloud itself relies heavily on PHP, and with a more manual Nextcloud setup, PHP is configurable. In AIO, the containerized approach limits direct access to PHP configurations. To modify PHP settings, you might need to adjust the Docker container or override the configurations indirectly.

[Potato-007]

also, make sure to read the related guide:

https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/big_file_upload_configuration.html