Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error downloading or uploading large files #9801

Closed
ludovi-com opened this issue Jun 8, 2018 · 1 comment
Closed

Error downloading or uploading large files #9801

ludovi-com opened this issue Jun 8, 2018 · 1 comment

Comments

@ludovi-com
Copy link

ludovi-com commented Jun 8, 2018

Steps to reproduce

Sometimes, randomly, downloading or uploading large files

Expected behaviour

Synchronise the file

Actual behaviour

Skip the file and returns error : Doctrine\DBAL\Exception\UniqueConstraintViolationException: An exception occurred while executing 'INSERT INTO oc_file_locks (key,lock,ttl) SELECT ?,?,? FROM oc_file_locks WHERE key = ? HAVING COUNT(*) = 0' with params ["files/b8d1b7bbcdb48962b43c548833754976", 1, 1528461715, "files/b8d1b7bbcdb48962b43c548833754976"]: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'files/b8d1b7bbcdb48962b43c548833754976' for key 'lock_key_index'

Server configuration

Operating system: CentOS Linux release 7.5.1804 (Core)

Web server: Apache (httpd-2.4.6-80.el7.centos.x86_64)

Database: Ver 8.0.11 for Linux on x86_64 (MySQL Community Server - GPL)

PHP version: PHP 7.2.6

Nextcloud version: 13.0.3

Updated from an older Nextcloud/ownCloud or fresh install: Fresh install

Where did you install Nextcloud from: Download from NextCloud website

Signing status:

Signing status
No errors have been found.

List of activated apps:

App list
Enabled:
  - activity: 2.6.1
  - bruteforcesettings: 1.0.3
  - comments: 1.3.0
  - dav: 1.4.7
  - federatedfilesharing: 1.3.1
  - federation: 1.3.0
  - files: 1.8.0
  - files_sharing: 1.5.0
  - files_texteditor: 2.5.1
  - files_trashbin: 1.3.0
  - files_versions: 1.6.0
  - files_videoplayer: 1.2.0
  - firstrunwizard: 2.2.1
  - gallery: 18.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.1.0
  - nextcloud_announcements: 1.2.0
  - notifications: 2.1.2
  - oauth2: 1.1.1
  - password_policy: 1.3.0
  - provisioning_api: 1.3.0
  - serverinfo: 1.3.0
  - sharebymail: 1.3.0
  - survey_client: 1.1.0
  - systemtags: 1.3.0
  - theming: 1.4.5
  - twofactor_backupcodes: 1.2.3
  - updatenotification: 1.3.0
  - workflowengine: 1.3.0
Disabled:
  - admin_audit
  - encryption
  - files_external
  - files_pdfviewer
  - user_external
  - user_ldap

Nextcloud configuration:

Config report
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***",
        "dbtype": "mysql",
        "version": "13.0.3.2",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_smtpmode": "php",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpsecure": "ssl",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "htaccess.RewriteBase": "\/nextcloud",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "enable_previews": true,
        "preview_max_x": 1280,
        "preview_max_y": 720,
        "enabledPreviewProviders": [
            "OC\\Preview\\Movie",
            "OC\\Preview\\Image"
        ],
        "enable_movie_transcode": true
    }
}

Are you using external storage, if yes which one: local (Setting data folder to a binded mount point)

Are you using encryption: No

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

LDAP configuration (delete this part if not used)

LDAP config
[Symfony\Component\Console\Exception\CommandNotFoundException]  
  There are no commands defined in the "ldap" namespace.

Client configuration

Browser: Error encountered in various browsers (Safari, Firefox, Chrome) and desktop clients

Operating system: Error encountered in various Operating Systems (Windows 10 and MacOS 10.13)

Logs

Web server error log

Web server error log
Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Nextcloud log
{"reqId":"WxmuIkfOtVyixusuLkozrQAAAAk","level":2,"time":"2018-06-07T22:13:54+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"--","app":"core","method":"POST","url":"\/nextcloud\/login","message":"Login failed: 'ludovico' (Remote IP: '77.43.1.139')","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1 Safari\/605.1.15","version":"13.0.3.2"}
{"reqId":"WxoNXmWYmFj6zHmZkfo4wgAAAAg","level":3,"time":"2018-06-08T05:00:15+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/apps\/files_trashbin\/ajax\/preview.php?file=%2FNextcloud.mp4.d1528401419&c=1528401419000","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_wIVnIZ): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.1 Safari\/605.1.15","version":"13.0.3.2"}
{"reqId":"WxoNaGNy7xdk-yjksn7tzwAAAAc","level":3,"time":"2018-06-08T05:00:24+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"POST","url":"\/nextcloud\/index.php\/apps\/files_trashbin\/ajax\/delete.php","message":"filesize(): stat failed for \/var\/www\/html\/nextdata\/nextdata\/\/ludovico\/files_trashbin\/files\/FILE019.MOV.d1528409613 at \/var\/www\/html\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#166","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.1 Safari\/605.1.15","version":"13.0.3.2"}
{"reqId":"WxoNaGNy7xdk-yjksn7tzwAAAAc","level":3,"time":"2018-06-08T05:00:24+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"POST","url":"\/nextcloud\/index.php\/apps\/files_trashbin\/ajax\/delete.php","message":"filesize(): stat failed for \/var\/www\/html\/nextcloud\/nextdata\/nextdata\/\/ludovico\/files_trashbin\/files\/FILE019.MOV.d1528401438 at \/var\/www\/html\/nextcloud\/lib\/private\/Files\/Storage\/Local.php#166","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.1 Safari\/605.1.15","version":"13.0.3.2"}
{"reqId":"WxoNg3QiHCnVq7u2m5QC@wAAAAU","level":2,"time":"2018-06-08T05:00:51+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"--","app":"core","method":"POST","url":"\/nextcloud\/login","message":"Login failed: 'root' (Remote IP: '***REMOVED SENSITIVE VALUE***')","userAgent":"Mozilla\/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit\/605.1.15 (KHTML, like Gecko) Version\/11.1.1 Safari\/605.1.15","version":"13.0.3.2"}
{"reqId":"Wxo0c7bjqnIscLQ9skMsJgAAAAQ","level":3,"time":"2018-06-08T07:47:00+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"francesco","app":"PHP","method":"GET","url":"\/nextcloud\/core\/preview.png?file=%2FNextcloud.mp4&c=1f631ae1ed17006fe4bff343c746a370&x=56&y=56&forceIcon=0","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_W6wM1W): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/67.0.3396.79 Safari\/537.36","version":"13.0.3.2"}
{"reqId":"Wxo1MfF5dfWvvx8bPPbq4gAAAAY","level":3,"time":"2018-06-08T07:50:25+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_vXDTcD): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxo1MfF5dfWvvx8bPPbq4gAAAAY","level":3,"time":"2018-06-08T07:50:40+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_7WOse0): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxo1RMMLUoGt9@8dbOYoSQAAAAc","level":3,"time":"2018-06-08T07:50:42+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_FQxLfU): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxo1QzZM7gZEXNSS1id0rQAAAAA","level":3,"time":"2018-06-08T07:50:43+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_JmDp8O): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxo1RMMLUoGt9@8dbOYoSQAAAAc","level":3,"time":"2018-06-08T07:50:54+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_DVUFf3): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxo1QzZM7gZEXNSS1id0rQAAAAA","level":3,"time":"2018-06-08T07:51:05+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_LS2e1Z): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxo1YSd0amOCaDOaaR-EnwAAAAs","level":3,"time":"2018-06-08T07:51:12+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_amGoXN): No such file or directory at \/var\/www\/html\/nextcloud\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxo1YSd0amOCaDOaaR-EnwAAAAs","level":3,"time":"2018-06-08T07:51:31+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"PHP","method":"GET","url":"\/nextcloud\/index.php\/apps\/files\/api\/v1\/thumbnail\/128\/128\/Drone\/18-05-26\/DJI_0193.MOV","message":"unlink(\/mnt\/data\/php_tmp\/oc_tmp_2AU5Jj): No such file or directory at \/var\/www\/html\/lib\/private\/Preview\/Movie.php#111","userAgent":"Mozilla\/5.0 (iOS) Nextcloud-iOS\/2.21.0","version":"13.0.3.2"}
{"reqId":"Wxprg10xFcHfuwCM62iX-gAAAAE","level":4,"time":"2018-06-08T11:41:55+00:00","remoteAddr":"***REMOVED SENSITIVE VALUE***","user":"ludovico","app":"webdav","method":"GET","url":"\/nextcloud\/remote.php\/dav\/files\/Drone\/18-05-18\/FILE002.MOV","message":"Exception: {\"Exception\":\"Doctrine\\\\DBAL\\\\Exception\\\\UniqueConstraintViolationException\",\"Message\":\"An exception occurred while executing 'INSERT INTO `oc_file_locks` (`key`,`lock`,`ttl`) SELECT ?,?,? FROM `oc_file_locks` WHERE `key` = ? HAVING COUNT(*) = 0' with params [\\\"files\\\\\\\/b8d1b7bbcdb48962b43c548833754976\\\", 1, 1528461715, \\\"files\\\\\\\/b8d1b7bbcdb48962b43c548833754976\\\"]:\\n\\nSQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'files\\\/b8d1b7bbcdb48962b43c548833754976' for key 'lock_key_index'\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/html\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/DBALException.php(128): Doctrine\\\\DBAL\\\\Driver\\\\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException))\\n#1 \\\/var\\\/www\\\/html\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Connection.php(1015): Doctrine\\\\DBAL\\\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOMySql\\\\Driver), Object(Doctrine\\\\DBAL\\\\Driver\\\\PDOException), 'INSERT INTO `oc...', Array)\\n#2 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/DB\\\/Connection.php(216): Doctrine\\\\DBAL\\\\Connection->executeUpdate('INSERT INTO `oc...', Array, Array)\\n#3 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/DB\\\/Adapter.php(114): OC\\\\DB\\\\Connection->executeUpdate('INSERT INTO `oc...', Array)\\n#4 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/DB\\\/Connection.php(254): OC\\\\DB\\\\Adapter->insertIfNotExist('*PREFIX*file_lo...', Array, Array)\\n#5 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Lock\\\/DBLockingProvider.php(119): OC\\\\DB\\\\Connection->insertIfNotExist('*PREFIX*file_lo...', Array, Array)\\n#6 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Lock\\\/DBLockingProvider.php(164): OC\\\\Lock\\\\DBLockingProvider->initLockField('files\\\/b8d1b7bbc...', 1)\\n#7 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Common.php(709): OC\\\\Lock\\\\DBLockingProvider->acquireLock('files\\\/b8d1b7bbc...', 1)\\n#8 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(592): OC\\\\Files\\\\Storage\\\\Common->acquireLock('files\\\/Drone', 1, Object(OC\\\\Lock\\\\DBLockingProvider))\\n#9 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/View.php(1924): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->acquireLock('files\\\/Drone', 1, Object(OC\\\\Lock\\\\DBLockingProvider))\\n#10 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/View.php(2038): OC\\\\Files\\\\View->lockPath('\\\/Drone', 1)\\n#11 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/View.php(1134): OC\\\\Files\\\\View->lockFile('\\\/Drone\\\/18-05-18...', 1)\\n#12 \\\/var\\\/www\\\/html\\\/nextcloud\\\/lib\\\/private\\\/Files\\\/View.php(987): OC\\\\Files\\\\View->basicOperation('fopen', '\\\/Drone\\\/18-05-18...', Array, 'r')\\n#13 \\\/var\\\/www\\\/html\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php(335): OC\\\\Files\\\\View->fopen('Drone\\\/18-05-18\\\/...', 'r')\\n#14 \\\/var\\\/www\\\/html\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(88): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->get()\\n#15 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpGet(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#16 \\\/var\\\/www\\\/html\\\/nextcloud\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#17 \\\/var\\\/www\\\/html\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(479): Sabre\\\\Event\\\\EventEmitter->emit('method:GET', Array)\\n#18 \\\/var\\\/www\\\/html\\\/nextcloud\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#19 \\\/var\\\/www\\\/html\\\/nextcloud\\\/apps\\\/dav\\\/lib\\\/Server.php(287): Sabre\\\\DAV\\\\Server->exec()\\n#20 \\\/var\\\/www\\\/html\\\/nextcloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(35): OCA\\\\DAV\\\\Server->exec()\\n#21 \\\/var\\\/www\\\/html\\\/nextcloud\\\/remote.php(164): require_once('\\\/var\\\/www\\\/html\\\/n...')\\n#22 {main}\",\"File\":\"\\\/var\\\/www\\\/html\\\/nextcloud\\\/3rdparty\\\/doctrine\\\/dbal\\\/lib\\\/Doctrine\\\/DBAL\\\/Driver\\\/AbstractMySQLDriver.php\",\"Line\":66}","userAgent":"Mozilla\/5.0 (Macintosh) mirall\/2.3.3 (build 84) (Nextcloud)","version":"13.0.3.2"}

Browser log

Browser log
Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...
@MorrisJobke
Copy link
Member

In more recent versions (13.0.4 and 12.0.9) we reduced drastically the amount of lock statements which should avoid most of the problems in here. Thus I will close it. If there is still stuff that breaks have a look at #9305 and it's linked issues.

Another workaround: use Redis as locking backend

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants