File with plus (+) character stored on external WebDAV storage disappear

[PaulBol]

I noticed an issue that seems similar to #7762 but the behavior is slightly different and more worrisome from my perspective. Whenever I put a file with a plus character in its file name in a folder it disappears.

Any ideas what I can try to avoid the problem or to collect more details?

Steps to reproduce

1. Mount external WebDAV storage, I'm using https://www.magentacloud.de/
2. Enable encryption on external storage
3. Upload file "Plus+file.txt" to main folder on WebDAV storage
4. Refresh file list - "Plus+file.txt" is not listed anymore

Expected behaviour

"Plus+file.txt" should be listed, available to download and sync to local drive
This is what happens when adding the same file to an internal folder

Actual behaviour

"Plus+file.txt" is not listed anymore
However, it does exist on the WebDAV storage
Also when trying to upload again a conflict is shown

Server configuration

Operating system: Not exactly sure, shared web server at Host Europe

Web server: Not exactly sure, shared web server at Host Europe

Database: mysql 5.6.43

PHP version: 7.2.18

Nextcloud version: (see Nextcloud admin page) 16.0.1.1

Updated from an older Nextcloud/ownCloud or fresh install: Updated from ownCloud but external storage was only added after updating

Where did you install Nextcloud from: Nextcloud website

Signing status:

Signing status

Results
=======
- core
	- INVALID_HASH
		- .htaccess

Raw output
==========
Array
(
    [core] => Array
        (
            [INVALID_HASH] => Array
                (
                    [.htaccess] => Array
                        (
                            [expected] => f30494d88fc2d7caf0be8b27629cb246483cea0203696e44fd376bff19db51bb89c9f4076b1f849627ef3cb011f4a20088116e7b0d10ce467a3dde5db71e108b
                            [current] => 78d327103dcad8520b5068521a03f20126938f25ea3b69192cc38e882248cae47a17938de718a062e77dc4d79574a83e957e827c3872cd643f5c59f3674c995f
                        )

                )

        )

)

List of activated apps:

App list

SSO & SAML authentication 
2.2.0
Official 
  


Accessibility 
1.2.0
Official 
 


Collaborative tags 
1.6.0
Official 
 


Comments 
1.6.0
Official 
 


Default encryption module 
2.4.0
Official 
 


Deleted files 
1.6.0
Official 
 


External storage support 
1.7.0
Official 
 


Federation 
1.6.0
Official 
 


File sharing 
1.8.0
Official 
 


First run wizard 
2.5.0
Official 
 


Log Reader 
2.1.0
Official 
 


Monitoring 
1.6.0
Official 
 


Nextcloud announcements 
1.5.0
Official 
 


Notifications 
2.4.1
Official 
 


Password policy 
1.6.0
Official 
 


Privacy 
1.0.0

 

Recommendations 
0.4.0

 

Right click 
0.13.0

 


Share by mail 
1.6.0
Official 
 


Support 
1.0.0
Official 
 


Theming 
1.7.0
Official 
 


Update notification 
1.6.0
Official 
 


Usage survey 
1.4.0
Official 
 


Versions 
1.9.0
Official 
 


Video player 
1.5.0
Official 
 


Viewer 
1.0.0

 


Activity 
2.9.1
Official 


Auditing / Logging 
1.6.0
Official 



Gallery 
18.3.0
Official 



LDAP user and group backend 
1.6.0
Official 



PDF viewer 
1.5.0
Official 



Text editor 
2.8.0
Official 

Nextcloud configuration:

Config report

<?php
$CONFIG = array (
  'instanceid' => 'REMOVED',
  'passwordsalt' => 'REMOVED',
  'secret' => 'REMOVED',
  'trusted_domains' => 
  array (
    0 => 'REMOVED',
  ),
  'datadirectory' => '/is/htdocs/REMOVED/owncloud/data',
  'dbtype' => 'mysql',
  'version' => '16.0.1.1',
  'dbname' => 'REMOVED',
  'dbhost' => 'localhost',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'REMOVED',
  'dbpassword' => 'REMOVED',
  'logtimezone' => 'UTC',
  'installed' => true,
  'theme' => '',
  'loglevel' => 2,
  'maintenance' => false,
  'mail_smtpmode' => 'sendmail',
  'mail_from_address' => 'REMOVED',
  'mail_domain' => 'REMOVED',
  'updater.secret' => 'REMOVED',
  'mysql.utf8mb4' => false,
  'trashbin_retention_obligation' => 'auto, 90',
  'versions_retention_obligation' => 'auto, 720',
  'upgrade.disable-web' => false,
  'overwrite.cli.url' => 'https://REMOVED',
  'mail_smtpsecure' => 'ssl',
  'mail_sendmailmode' => 'pipe',
  'mail_smtpauth' => 1,
  'mail_smtphost' => 'REMOVED',
  'mail_smtpport' => '25',
  'mail_smtpname' => 'REMOVED',
  'mail_smtppassword' => 'REMOVED',
);

Are you using external storage, if yes which one: WebDAV

Are you using encryption: yes

Are you using an external user-backend, if yes which one:

Client configuration

Browser: Firefox, Edge and likely any other browser

Operating system: Windows 10

Logs

Web server error log

Web server error log

error log is empty

Nextcloud log (data/nextcloud.log)

Nextcloud log

no records added while problem is observed

Browser log

Browser log

n/a

[PaulBol]

I increased the log level to capture some additional data.

It looks to me like the file name gets decoded twice. Plus%2Bfile.txt is stored as Plus+file.txt but later Nextcloud is looking for Plus file.txt

{"reqId":"6Xr7MHobfICjMLrsC3NQ","level":0,"time":"2019-06-03T09:36:42+00:00","remoteAddr":"IP","user":"userid","app":"OC\\Files\\Cache\\Scanner","method":"PROPFIND","url":"\/remote.php\/dav\/files\/userid\/MyFolder\/Plus%2Bfile.txt","message":"!!! Path 'Plus file.txt' is not accessible or present !!!","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}
{"reqId":"zNaJwQ4vwLL9cfrT2d30","level":0,"time":"2019-06-03T09:36:44+00:00","remoteAddr":"IP","user":"userid","app":"OC\\Files\\Cache\\Scanner","method":"GET","url":"\/index.php\/core\/preview?fileId=111956&c=7f9b-2068-58a681a01248a&x=250&y=250&forceIcon=0","message":"!!! Path 'Plus file.txt' is not accessible or present !!!","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}
{"reqId":"thf8Kt98jfNy6AvxZvvd","level":0,"time":"2019-06-03T09:36:51+00:00","remoteAddr":"IP","user":"--","app":"cron","method":"GET","url":"\/cron.php","message":"Run OC\\Command\\CommandJob job with ID 639","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}
{"reqId":"thf8Kt98jfNy6AvxZvvd","level":0,"time":"2019-06-03T09:36:51+00:00","remoteAddr":"IP","user":"--","app":"cron","method":"GET","url":"\/cron.php","message":"Finished OC\\Command\\CommandJob job with ID 639 in 0 seconds","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}
{"reqId":"Dhx1oQiR7xzbbTRkWHmz","level":0,"time":"2019-06-03T09:36:52+00:00","remoteAddr":"IP","user":"userid","app":"OC\\Files\\Cache\\Scanner","method":"PROPFIND","url":"\/remote.php\/dav\/files\/userid\/MyFolder","message":"!!! Path 'Plus file.txt' is not accessible or present !!!","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}
{"reqId":"Dhx1oQiR7xzbbTRkWHmz","level":0,"time":"2019-06-03T09:36:52+00:00","remoteAddr":"IP","user":"userid","app":"OC\\Files\\Cache\\Scanner","method":"PROPFIND","url":"\/remote.php\/dav\/files\/userid\/MyFolder","message":"!!! Path 'Plus file.txt' is not accessible or present !!!","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}
{"reqId":"Dhx1oQiR7xzbbTRkWHmz","level":0,"time":"2019-06-03T09:36:52+00:00","remoteAddr":"IP","user":"userid","app":"OC\\Files\\Cache\\Scanner","method":"PROPFIND","url":"\/remote.php\/dav\/files\/userid\/MyFolder","message":"!!! Path 'Plus file.txt' is not accessible or present !!!","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}
{"reqId":"8K1OeP1PdFG6R1GujENk","level":0,"time":"2019-06-03T09:36:53+00:00","remoteAddr":"IP","user":"userid","app":"OC\\Files\\Cache\\Scanner","method":"GET","url":"\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FMyFolder","message":"!!! Path 'Plus file.txt' is not accessible or present !!!","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko\/20100101 Firefox\/67.0","version":"16.0.1.1"}

[PaulBol]

I looked through the code to figure out where Plus+file.txt becomes Plus file.txt. It happens in the line

$file = urldecode($file);

in lib/private/Files/Storage/DAV.php public function opendir($path).

The external WebDAV server apparently returns $file='Plus+file.txt' which gets decoded to Plus file.txt.

Replacing the line with

$file = urldecode(str_replace('+', '%2B', $file));

and further changing

return str_replace('%2F', '/', rawurlencode($path));

to

return str_replace('%2B', '+', str_replace('%2F', '/', rawurlencode($path)));

in protected function encodePath($path) solves the problem for me.

However, is the fix necessary for WebDAV in general or is the https://www.magentacloud.de/ WebDAV implementation broken?

[kesselb]

cc @rullzer @icewind1991

[PaulBol]

Can anyone comment on this (potential) issue?

[jospoortvliet]

Hi,

Thanks for your bug report and sorry to see you have this problem. I notice nobody has had time to respond... As this more of a corner case, it will not be high priority so it can take a lot of time before somebody has time for it... If you are familiar with PHP, help is welcome!

If you need this fixed urgently and can't do it yourself, or need it to test Nextcloud for use in your business, we would be happy to make time for you. See nextcloud.com/enterprise for more information about Nextcloud Enterprise and options for help with a proof of concept.

[bendem]

As this more of a corner case, it will not be high priority so it can take a lot of time before somebody has time for it...

Not so corner case, multiple people have reported this issue 3 years ago, which was closed even though users made clear the problem was not fixed.

[szaimen]

Is this Issue still valid? If not, please close this issue. Thanks! :)

[PaulBol]

I gave up on WebDAV when it became clear that the issue won't be fixed and I got the impression that WebDAV support is not a priority.

Therefore I can't confirm that the bug still exists. However, in the history https://github.com/nextcloud/server/commits/master/lib/private/Files/Storage/DAV.php I see no attempt to address it.

[ghost]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

[bendem]

Not stale, just waiting to be fixed

[szaimen]

Does this only happen for magentacloud webdav connections for you or also for other webdav connetions?

[ghost]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

[GPS2288]

Does this only happen for magentacloud webdav connections for you or also for other webdav connetions?

I can confirm that this happens with my own QNAP webdav server, too.
Please reopen the issue.

[wahboh]

Kindly reopen this case. It is not stale, just unattended, which surprises me after so long time! The bug seems frequent on every Nextcloud installation I have tested. '+' i frequent in Windows filenames and the Nextcloud behaviour makes Nextcloud practically unusable.
Every filename with '+' crashes the sync.