Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DeadlockException executing UPDATE oc_filecache #22482

Closed
ghost opened this issue Aug 29, 2020 · 118 comments
Closed

DeadlockException executing UPDATE oc_filecache #22482

ghost opened this issue Aug 29, 2020 · 118 comments
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 25-feedback bug needs info stale Ticket or PR with no recent activity

Comments

@ghost
Copy link

ghost commented Aug 29, 2020 •
edited by kesselb
Loading

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.

Report

Getting the following error when deleting files. I even setup a new Nextcloud 19.0.2 server and still same issue. Seems to be related with photo files more then docs.

Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing
'UPDATE `oc_filecache` SET `size` = ? WHERE (`fileid` = ?) AND ((`size` <> ?) OR (`size` IS NULL))'
with params [237591435, 213130, 237591435]:
SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction

I've already ran

sudo -u www-data php occ files:scan-app-data
sudo -u www-data php occ files:scan --all
sudo -u www-data php occ files:cleanup

Already checked the oc_filecache table; and no issues there based on the output of mysql.

image

image

image
@ghost ghost added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Aug 29, 2020
@solracsf
Copy link
Member

solracsf commented Aug 29, 2020 •
edited
Loading

There are several issues like this like #15047, but it clears seems a new problem introduced in 19.0.2 as it affects all our instances with different setups after upgrading from 19.0.1.

We can reproduce it but uploading many files (30) to a folder then select them all and delete. Some of them will produce a Deadlock.

Filelocking is enabled on Redis. I'm using MariaBD 10.4.

Detailed output 
{
  "reqId": "1IyGszyr4rk3uCA4FIoF",
  "level": 4,
  "time": "2020-08-27T16:06:11+02:00",
  "remoteAddr": "176.177.105.226",
  "user": "nextcloud",
  "app": "webdav",
  "method": "DELETE",
  "url": "/remote.php/dav/files/nextcloud/Photos/hvkjbhvbkljmknbm.png",
  "message": {
    "Exception": "Doctrine\\DBAL\\Exception\\DeadlockException",
    "Message": "An exception occurred while executing 'UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))' with params [1598537171, \"5f47bdd345199\", 1]:\n\nSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction",
    "Code": 0,
    "Trace": [
      {
        "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php",
        "line": 169,
        "function": "convertException",
        "class": "Doctrine\\DBAL\\Driver\\AbstractMySQLDriver",
        "type": "->",
        "args": [
          "An exception occurred while executing 'UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))' with params [1598537171, \"5f47bdd345199\", 1]:\n\nSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction",
          {
            "errorInfo": [
              "40001",
              1213,
              "Deadlock found when trying to get lock; try restarting transaction"
            ],
            "__class__": "Doctrine\\DBAL\\Driver\\PDOException"
          }
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php",
        "line": 145,
        "function": "wrapException",
        "class": "Doctrine\\DBAL\\DBALException",
        "type": "::",
        "args": [
          {
            "__class__": "Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"
          },
          {
            "errorInfo": [
              "40001",
              1213,
              "Deadlock found when trying to get lock; try restarting transaction"
            ],
            "__class__": "Doctrine\\DBAL\\Driver\\PDOException"
          },
          "An exception occurred while executing 'UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))' with params [1598537171, \"5f47bdd345199\", 1]:\n\nSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction"
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php",
        "line": 1063,
        "function": "driverExceptionDuringQuery",
        "class": "Doctrine\\DBAL\\DBALException",
        "type": "::",
        "args": [
          {
            "__class__": "Doctrine\\DBAL\\Driver\\PDOMySql\\Driver"
          },
          {
            "errorInfo": [
              "40001",
              1213,
              "Deadlock found when trying to get lock; try restarting transaction"
            ],
            "__class__": "Doctrine\\DBAL\\Driver\\PDOException"
          },
          "UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))",
          {
            "1": 1598537171,
            "2": "5f47bdd345199",
            "3": 1
          }
        ]
      },
      {
        "file": "/var/www/nextcloud/lib/private/DB/Connection.php",
        "line": 214,
        "function": "executeUpdate",
        "class": "Doctrine\\DBAL\\Connection",
        "type": "->",
        "args": [
          "UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))",
          [
            1598537171,
            "5f47bdd345199",
            1
          ],
          [
            1,
            2,
            1
          ]
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php",
        "line": 203,
        "function": "executeUpdate",
        "class": "OC\\DB\\Connection",
        "type": "->",
        "args": [
          "UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, :dcValue1), `etag` = :dcValue2 WHERE (`storage` = :dcValue3) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))",
          {
            "dcValue1": 1598537171,
            "dcValue2": "5f47bdd345199",
            "dcValue3": 1
          },
          {
            "dcValue1": 1,
            "dcValue2": 2,
            "dcValue3": 1
          }
        ]
      },
      {
        "file": "/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
        "line": 216,
        "function": "execute",
        "class": "Doctrine\\DBAL\\Query\\QueryBuilder",
        "type": "->",
        "args": []
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/Cache/Propagator.php",
        "line": 101,
        "function": "execute",
        "class": "OC\\DB\\QueryBuilder\\QueryBuilder",
        "type": "->",
        "args": []
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/Cache/Updater.php",
        "line": 166,
        "function": "propagateChange",
        "class": "OC\\Files\\Cache\\Propagator",
        "type": "->",
        "args": [
          "files/Photos/hvkjbhvbkljmknbm.png",
          1598537171
        ]
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/View.php",
        "line": 329,
        "function": "remove",
        "class": "OC\\Files\\Cache\\Updater",
        "type": "->",
        "args": [
          "files/Photos/hvkjbhvbkljmknbm.png"
        ]
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/View.php",
        "line": 1173,
        "function": "removeUpdate",
        "class": "OC\\Files\\View",
        "type": "->",
        "args": [
          {
            "cache": {
              "__class__": "OC\\Files\\Cache\\Cache"
            },
            "scanner": null,
            "watcher": null,
            "propagator": {
              "__class__": "OC\\Files\\Cache\\Propagator"
            },
            "updater": {
              "__class__": "OC\\Files\\Cache\\Updater"
            },
            "__class__": "OCA\\Files_Trashbin\\Storage"
          },
          "files/Photos/hvkjbhvbkljmknbm.png"
        ]
      },
      {
        "file": "/var/www/nextcloud/lib/private/Files/View.php",
        "line": 718,
        "function": "basicOperation",
        "class": "OC\\Files\\View",
        "type": "->",
        "args": [
          "unlink",
          "/Photos/hvkjbhvbkljmknbm.png",
          [
            "delete"
          ]
        ]
      },
      {
        "file": "/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php",
        "line": 455,
        "function": "unlink",
        "class": "OC\\Files\\View",
        "type": "->",
        "args": [
          "/Photos/hvkjbhvbkljmknbm.png"
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php",
        "line": 183,
        "function": "delete",
        "class": "OCA\\DAV\\Connector\\Sabre\\File",
        "type": "->",
        "args": []
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
        "line": 295,
        "function": "delete",
        "class": "Sabre\\DAV\\Tree",
        "type": "->",
        "args": [
          "files/nextcloud/Photos/hvkjbhvbkljmknbm.png"
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
        "line": 89,
        "function": "httpDelete",
        "class": "Sabre\\DAV\\CorePlugin",
        "type": "->",
        "args": [
          {
            "__class__": "Sabre\\HTTP\\Request"
          },
          {
            "__class__": "Sabre\\HTTP\\Response"
          }
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 474,
        "function": "emit",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          "method:DELETE",
          [
            {
              "__class__": "Sabre\\HTTP\\Request"
            },
            {
              "__class__": "Sabre\\HTTP\\Response"
            }
          ]
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 251,
        "function": "invokeMethod",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          {
            "__class__": "Sabre\\HTTP\\Request"
          },
          {
            "__class__": "Sabre\\HTTP\\Response"
          }
        ]
      },
      {
        "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 319,
        "function": "start",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/var/www/nextcloud/apps/dav/lib/Server.php",
        "line": 320,
        "function": "exec",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php",
        "line": 35,
        "function": "exec",
        "class": "OCA\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/var/www/nextcloud/remote.php",
        "line": 167,
        "args": [
          "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php",
    "Line": 34,
    "Previous": {
      "Exception": "Doctrine\\DBAL\\Driver\\PDOException",
      "Message": "SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction",
      "Code": "40001",
      "Trace": [
        {
          "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php",
          "line": 1054,
          "function": "execute",
          "class": "Doctrine\\DBAL\\Driver\\PDOStatement",
          "type": "->",
          "args": []
        },
        {
          "file": "/var/www/nextcloud/lib/private/DB/Connection.php",
          "line": 214,
          "function": "executeUpdate",
          "class": "Doctrine\\DBAL\\Connection",
          "type": "->",
          "args": [
            "UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))",
            [
              1598537171,
              "5f47bdd345199",
              1
            ],
            [
              1,
              2,
              1
            ]
          ]
        },
        {
          "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php",
          "line": 203,
          "function": "executeUpdate",
          "class": "OC\\DB\\Connection",
          "type": "->",
          "args": [
            "UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, :dcValue1), `etag` = :dcValue2 WHERE (`storage` = :dcValue3) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))",
            {
              "dcValue1": 1598537171,
              "dcValue2": "5f47bdd345199",
              "dcValue3": 1
            },
            {
              "dcValue1": 1,
              "dcValue2": 2,
              "dcValue3": 1
            }
          ]
        },
        {
          "file": "/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
          "line": 216,
          "function": "execute",
          "class": "Doctrine\\DBAL\\Query\\QueryBuilder",
          "type": "->",
          "args": []
        },
        {
          "file": "/var/www/nextcloud/lib/private/Files/Cache/Propagator.php",
          "line": 101,
          "function": "execute",
          "class": "OC\\DB\\QueryBuilder\\QueryBuilder",
          "type": "->",
          "args": []
        },
        {
          "file": "/var/www/nextcloud/lib/private/Files/Cache/Updater.php",
          "line": 166,
          "function": "propagateChange",
          "class": "OC\\Files\\Cache\\Propagator",
          "type": "->",
          "args": [
            "files/Photos/hvkjbhvbkljmknbm.png",
            1598537171
          ]
        },
        {
          "file": "/var/www/nextcloud/lib/private/Files/View.php",
          "line": 329,
          "function": "remove",
          "class": "OC\\Files\\Cache\\Updater",
          "type": "->",
          "args": [
            "files/Photos/hvkjbhvbkljmknbm.png"
          ]
        },
        {
          "file": "/var/www/nextcloud/lib/private/Files/View.php",
          "line": 1173,
          "function": "removeUpdate",
          "class": "OC\\Files\\View",
          "type": "->",
          "args": [
            {
              "cache": {
                "__class__": "OC\\Files\\Cache\\Cache"
              },
              "scanner": null,
              "watcher": null,
              "propagator": {
                "__class__": "OC\\Files\\Cache\\Propagator"
              },
              "updater": {
                "__class__": "OC\\Files\\Cache\\Updater"
              },
              "__class__": "OCA\\Files_Trashbin\\Storage"
            },
            "files/Photos/hvkjbhvbkljmknbm.png"
          ]
        },
        {
          "file": "/var/www/nextcloud/lib/private/Files/View.php",
          "line": 718,
          "function": "basicOperation",
          "class": "OC\\Files\\View",
          "type": "->",
          "args": [
            "unlink",
            "/Photos/hvkjbhvbkljmknbm.png",
            [
              "delete"
            ]
          ]
        },
        {
          "file": "/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php",
          "line": 455,
          "function": "unlink",
          "class": "OC\\Files\\View",
          "type": "->",
          "args": [
            "/Photos/hvkjbhvbkljmknbm.png"
          ]
        },
        {
          "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php",
          "line": 183,
          "function": "delete",
          "class": "OCA\\DAV\\Connector\\Sabre\\File",
          "type": "->",
          "args": []
        },
        {
          "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
          "line": 295,
          "function": "delete",
          "class": "Sabre\\DAV\\Tree",
          "type": "->",
          "args": [
            "files/nextcloud/Photos/hvkjbhvbkljmknbm.png"
          ]
        },
        {
          "file": "/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
          "line": 89,
          "function": "httpDelete",
          "class": "Sabre\\DAV\\CorePlugin",
          "type": "->",
          "args": [
            {
              "__class__": "Sabre\\HTTP\\Request"
            },
            {
              "__class__": "Sabre\\HTTP\\Response"
            }
          ]
        },
        {
          "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
          "line": 474,
          "function": "emit",
          "class": "Sabre\\DAV\\Server",
          "type": "->",
          "args": [
            "method:DELETE",
            [
              {
                "__class__": "Sabre\\HTTP\\Request"
              },
              {
                "__class__": "Sabre\\HTTP\\Response"
              }
            ]
          ]
        },
        {
          "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
          "line": 251,
          "function": "invokeMethod",
          "class": "Sabre\\DAV\\Server",
          "type": "->",
          "args": [
            {
              "__class__": "Sabre\\HTTP\\Request"
            },
            {
              "__class__": "Sabre\\HTTP\\Response"
            }
          ]
        },
        {
          "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
          "line": 319,
          "function": "start",
          "class": "Sabre\\DAV\\Server",
          "type": "->",
          "args": []
        },
        {
          "file": "/var/www/nextcloud/apps/dav/lib/Server.php",
          "line": 320,
          "function": "exec",
          "class": "Sabre\\DAV\\Server",
          "type": "->",
          "args": []
        },
        {
          "file": "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php",
          "line": 35,
          "function": "exec",
          "class": "OCA\\DAV\\Server",
          "type": "->",
          "args": []
        },
        {
          "file": "/var/www/nextcloud/remote.php",
          "line": 167,
          "args": [
            "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"
          ],
          "function": "require_once"
        }
      ],
      "File": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php",
      "Line": 119,
      "Previous": {
        "Exception": "PDOException",
        "Message": "SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction",
        "Code": "40001",
        "Trace": [
          {
            "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php",
            "line": 117,
            "function": "execute",
            "class": "PDOStatement",
            "type": "->",
            "args": [
              null
            ]
          },
          {
            "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php",
            "line": 1054,
            "function": "execute",
            "class": "Doctrine\\DBAL\\Driver\\PDOStatement",
            "type": "->",
            "args": []
          },
          {
            "file": "/var/www/nextcloud/lib/private/DB/Connection.php",
            "line": 214,
            "function": "executeUpdate",
            "class": "Doctrine\\DBAL\\Connection",
            "type": "->",
            "args": [
              "UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))",
              [
                1598537171,
                "5f47bdd345199",
                1
              ],
              [
                1,
                2,
                1
              ]
            ]
          },
          {
            "file": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php",
            "line": 203,
            "function": "executeUpdate",
            "class": "OC\\DB\\Connection",
            "type": "->",
            "args": [
              "UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, :dcValue1), `etag` = :dcValue2 WHERE (`storage` = :dcValue3) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'd01bb67e7b71dd49fd06bad922f521c9'))",
              {
                "dcValue1": 1598537171,
                "dcValue2": "5f47bdd345199",
                "dcValue3": 1
              },
              {
                "dcValue1": 1,
                "dcValue2": 2,
                "dcValue3": 1
              }
            ]
          },
          {
            "file": "/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
            "line": 216,
            "function": "execute",
            "class": "Doctrine\\DBAL\\Query\\QueryBuilder",
            "type": "->",
            "args": []
          },
          {
            "file": "/var/www/nextcloud/lib/private/Files/Cache/Propagator.php",
            "line": 101,
            "function": "execute",
            "class": "OC\\DB\\QueryBuilder\\QueryBuilder",
            "type": "->",
            "args": []
          },
          {
            "file": "/var/www/nextcloud/lib/private/Files/Cache/Updater.php",
            "line": 166,
            "function": "propagateChange",
            "class": "OC\\Files\\Cache\\Propagator",
            "type": "->",
            "args": [
              "files/Photos/hvkjbhvbkljmknbm.png",
              1598537171
            ]
          },
          {
            "file": "/var/www/nextcloud/lib/private/Files/View.php",
            "line": 329,
            "function": "remove",
            "class": "OC\\Files\\Cache\\Updater",
            "type": "->",
            "args": [
              "files/Photos/hvkjbhvbkljmknbm.png"
            ]
          },
          {
            "file": "/var/www/nextcloud/lib/private/Files/View.php",
            "line": 1173,
            "function": "removeUpdate",
            "class": "OC\\Files\\View",
            "type": "->",
            "args": [
              {
                "cache": {
                  "__class__": "OC\\Files\\Cache\\Cache"
                },
                "scanner": null,
                "watcher": null,
                "propagator": {
                  "__class__": "OC\\Files\\Cache\\Propagator"
                },
                "updater": {
                  "__class__": "OC\\Files\\Cache\\Updater"
                },
                "__class__": "OCA\\Files_Trashbin\\Storage"
              },
              "files/Photos/hvkjbhvbkljmknbm.png"
            ]
          },
          {
            "file": "/var/www/nextcloud/lib/private/Files/View.php",
            "line": 718,
            "function": "basicOperation",
            "class": "OC\\Files\\View",
            "type": "->",
            "args": [
              "unlink",
              "/Photos/hvkjbhvbkljmknbm.png",
              [
                "delete"
              ]
            ]
          },
          {
            "file": "/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php",
            "line": 455,
            "function": "unlink",
            "class": "OC\\Files\\View",
            "type": "->",
            "args": [
              "/Photos/hvkjbhvbkljmknbm.png"
            ]
          },
          {
            "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php",
            "line": 183,
            "function": "delete",
            "class": "OCA\\DAV\\Connector\\Sabre\\File",
            "type": "->",
            "args": []
          },
          {
            "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
            "line": 295,
            "function": "delete",
            "class": "Sabre\\DAV\\Tree",
            "type": "->",
            "args": [
              "files/nextcloud/Photos/hvkjbhvbkljmknbm.png"
            ]
          },
          {
            "file": "/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
            "line": 89,
            "function": "httpDelete",
            "class": "Sabre\\DAV\\CorePlugin",
            "type": "->",
            "args": [
              {
                "__class__": "Sabre\\HTTP\\Request"
              },
              {
                "__class__": "Sabre\\HTTP\\Response"
              }
            ]
          },
          {
            "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
            "line": 474,
            "function": "emit",
            "class": "Sabre\\DAV\\Server",
            "type": "->",
            "args": [
              "method:DELETE",
              [
                {
                  "__class__": "Sabre\\HTTP\\Request"
                },
                {
                  "__class__": "Sabre\\HTTP\\Response"
                }
              ]
            ]
          },
          {
            "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
            "line": 251,
            "function": "invokeMethod",
            "class": "Sabre\\DAV\\Server",
            "type": "->",
            "args": [
              {
                "__class__": "Sabre\\HTTP\\Request"
              },
              {
                "__class__": "Sabre\\HTTP\\Response"
              }
            ]
          },
          {
            "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
            "line": 319,
            "function": "start",
            "class": "Sabre\\DAV\\Server",
            "type": "->",
            "args": []
          },
          {
            "file": "/var/www/nextcloud/apps/dav/lib/Server.php",
            "line": 320,
            "function": "exec",
            "class": "Sabre\\DAV\\Server",
            "type": "->",
            "args": []
          },
          {
            "file": "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php",
            "line": 35,
            "function": "exec",
            "class": "OCA\\DAV\\Server",
            "type": "->",
            "args": []
          },
          {
            "file": "/var/www/nextcloud/remote.php",
            "line": 167,
            "args": [
              "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"
            ],
            "function": "require_once"
          }
        ],
        "File": "/var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php",
        "Line": 117
      }
    },
    "CustomMessage": "--"
  },
  "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0",
  "version": "19.0.2.2"
}

Cc @nextcloud/server-triage

@solracsf solracsf changed the title Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing 'UPDATE `oc_filecache 19.0.2: DeadlockException executing UPDATE oc_filecache Aug 29, 2020
@ghost
Copy link
Author

ghost commented Aug 29, 2020 •
edited by ghost
Loading

@acsfer Yup I only get it then I select alot of file within a folder to delete. Idk why this is happening but this should be looked at and fixed!

I got worried that my database got corrupt and I build a new server and still the same issue so definitely not a my server instance your anyone else that's having the issue out there.

cc @canadaduane @jcfischer

@linuxrrze

This comment has been minimized.

Sign in to view
@toxpal
Copy link

toxpal commented Aug 30, 2020

Came here to say that I have the same problem with ALL instances of NC right after upgrading to 19.0.2. Even performed a fresh installation of 19.0.2 - the problem is still here.

After some debugging, I found it only happens if I try to delete multiple files at once. If I delete the same files one after one, everything works perfectly. But select them all - and deadlock message is here...

@toxpal toxpal mentioned this issue Aug 30, 2020
Closed
@toxpal
Copy link

toxpal commented Aug 30, 2020

Another update: the issue exists even after adding "'filelocking.enabled' => false" line to config file.

@nickvergessen
Copy link
Member

Yeah the message is not from the file locking feature, but from a database lock

@nickvergessen
Copy link
Member

Cc @icewind1991 @rullzer anything changed in filesystem recently with regards to that?

@solracsf solracsf removed the 0. Needs triage Pending check for reproducibility or if it fits our roadmap label Aug 30, 2020
@APTX
Copy link

APTX commented Aug 31, 2020

I get various forms of DeadlockException almost every time I sync files (last time I wrote a comment about it: #10473 (comment)). I guess it's finally happening with MySQL as well.

There are quite a few issues related to these errors (#11136, #15047, #21537). These errors are not dangerous by themselves. The error means that 2 database transactions could not be executed at the same time and one of them got cancelled ( see https://en.wikipedia.org/wiki/Deadlock ). Retrying that transaction should be enough, but Nextcloud does not seem to handle these errors at all.

DeadlockExceptions can only happen when multiple requests are made to the Nextcloud server at the same time, so testing requires adding/removing multiple files, not just one. It should also only happen on queries that read/modify multiple rows.

My most recent DeadlockException:

Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing 'UPDATE "oc_filecache" SET "size" = GREATEST(?, "size" + ?) WHERE ("storage" = ?) AND ("path_hash" IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', 'f55b8b9ffac161f2bbd566bd5a4c8075', 'd642cac0349fb48bcc52d75171a3ce7c')) AND ("size" > '-1')' with params [-1, 3691355, 2]: SQLSTATE[40P01]: Deadlock detected: 7 ERROR: deadlock detected DETAIL: Process 2427 waits for ShareLock on transaction 44023088; blocked by process 2425. Process 2425 waits for ShareLock on transaction 44023091; blocked by process 2427. HINT: See server log for query details. CONTEXT: while rechecking updated tuple (325,6) in relation "oc_filecache"

@ghost
Copy link
Author

ghost commented Aug 31, 2020

@APTX Thank you for the detailed info, hopefully the dev team can figure something out.

@simonspa
Copy link
Contributor

Not sure this helps, but these are the queries (along with the back traces) which seem to trigger this for me:

UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?) 

[webdav] Fatal: Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing 'UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, ?), `etag` = ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', '5ef60ea8991f2576a90bb23baa5e4399', '48056cb550ab7052ab094f43fbb60058', 'fa7f16366733c3a3ea5b328fd5b52fa3', '6f979bb2b50146cf44f3065c1d67f59d', 'c11f4c49a04a182bbc89e93f52d09f32', '8b89822309373a5aae36013d86a47681'))' with params [1598854372, "5f4c94e4f3cb9", 9]:

SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction at <<closure>>

 0. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 169
    Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException("An exception oc ... n", Doctrine\DBAL\Dr ... ]})
 1. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 145
    Doctrine\DBAL\DBALException::wrapException(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "An exception oc ... n")
 2. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php line 1063
    Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "UPDATE `oc_file ... )", {1: 1598854372,2: "5f4c94e4f3cb9",3: 9})
 3. /var/www/nextcloud/lib/private/DB/Connection.php line 214
    Doctrine\DBAL\Connection->executeUpdate("UPDATE `oc_file ... )", [1598854372,"5f4c94e4f3cb9",9], [1,2,1])
 4. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php line 203
    OC\DB\Connection->executeUpdate("UPDATE `oc_file ... )", {dcValue1: 15988 ... 9}, {dcValue1: 1,dcValue2: 2,dcValue3: 1})
 5. /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php line 216
    Doctrine\DBAL\Query\QueryBuilder->execute()
 6. /var/www/nextcloud/lib/private/Files/Cache/Propagator.php line 101
    OC\DB\QueryBuilder\QueryBuilder->execute()
 7. /var/www/nextcloud/lib/private/Files/Cache/HomePropagator.php line 49
    OC\Files\Cache\Propagator->propagateChange("files/ ... 6", 1598854372, 0)
 8. /var/www/nextcloud/lib/private/Files/Cache/Updater.php line 166
    OC\Files\Cache\HomePropagator->propagateChange("files ... 6", 1598854372)
 9. /var/www/nextcloud/lib/private/Files/View.php line 329
    OC\Files\Cache\Updater->remove("files/<file> ... 6")
10. /var/www/nextcloud/lib/private/Files/View.php line 1173
    OC\Files\View->removeUpdate(OCA\Files_Trashb ... }}, "files/<file> ... 6")
11. /var/www/nextcloud/lib/private/Files/View.php line 718
    OC\Files\View->basicOperation("unlink", "/<file> ... 6", ["delete"])
12. /var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php line 455
    OC\Files\View->unlink("/<file> ... 6")
13. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php line 183
    OCA\DAV\Connector\Sabre\File->delete()
14. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 295
    Sabre\DAV\Tree->delete("files/<file> ... 6")
15. /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->httpDelete(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
16. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 474
    Sabre\DAV\Server->emit("method:DELETE", [Sabre\HTTP\Requ ... }])
17. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 251
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
18. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 319
    Sabre\DAV\Server->start()
19. /var/www/nextcloud/apps/dav/lib/Server.php line 320
    Sabre\DAV\Server->exec()
20. /var/www/nextcloud/apps/dav/appinfo/v2/remote.php line 35
    OCA\DAV\Server->exec()
21. /var/www/nextcloud/remote.php line 167
    require_once("/var/www/nextcl ... p")

DELETE /remote.php/dav/files/...
from <ip> by <user> at 2020-08-31T08:12:53+02:00

UPDATE `oc_filecache` SET `size` = ? 

[webdav] Fatal: Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing 'UPDATE `oc_filecache` SET `size` = ? WHERE (`fileid` = ?) AND ((`size` <> ?) OR (`size` IS NULL))' with params [93564, 2024096, 93564]:

SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction at <<closure>>

 0. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 169
    Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException("An exception oc ... n", Doctrine\DBAL\Dr ... ]})
 1. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 145
    Doctrine\DBAL\DBALException::wrapException(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "An exception oc ... n")
 2. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php line 1063
    Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "UPDATE `oc_file ... )", {1: 93564,2: "** ... 4})
 3. /var/www/nextcloud/lib/private/DB/Connection.php line 214
    Doctrine\DBAL\Connection->executeUpdate("UPDATE `oc_file ... )", [93564,"*** sens ... 4], [2,1,2])
 4. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php line 203
    OC\DB\Connection->executeUpdate("UPDATE `oc_file ... )", {dcValue1: "***  ... 4}, {dcValue1: 1,dcValue2: 2,dcValue3: 2})
 5. /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php line 216
    Doctrine\DBAL\Query\QueryBuilder->execute()
 6. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 363
    OC\DB\QueryBuilder\QueryBuilder->execute()
 7. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 898
    OC\Files\Cache\Cache->update("*** sensitive parameters replaced ***")
 8. /var/www/nextcloud/lib/private/Files/Cache/HomeCache.php line 43
    OC\Files\Cache\Cache->calculateFolderSize("files/<file> ... t", OC\Files\Cache\CacheEntry {})
 9. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 830
    OC\Files\Cache\HomeCache->calculateFolderSize("files/<file> ... t", null)
10. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 842
    OC\Files\Cache\Cache->correctFolderSize("files/<file> ... t")
11. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 842
    OC\Files\Cache\Cache->correctFolderSize("files/<file> ... t")
12. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 842
    OC\Files\Cache\Cache->correctFolderSize("files/<file> ... s")
13. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 842
    OC\Files\Cache\Cache->correctFolderSize("files/<file> ... 5")
14. /var/www/nextcloud/lib/private/Files/Cache/Updater.php line 217
    OC\Files\Cache\Cache->correctFolderSize("files/<file> ... 4")
15. /var/www/nextcloud/apps/files_trashbin/lib/Trashbin.php line 289
    OC\Files\Cache\Updater->renameFromStorage(OCA\Files_Trashb ... }}, "files/<file> ... 4", "files_trashbin/ ... 2")
16. /var/www/nextcloud/apps/files_trashbin/lib/Trash/LegacyTrashBackend.php line 108
    OCA\Files_Trashbin\Trashbin::move2trash("<file> ... 4")
17. /var/www/nextcloud/apps/files_trashbin/lib/Trash/TrashManager.php line 103
    OCA\Files_Trashbin\Trash\LegacyTrashBackend->moveToTrash(OCA\Files_Trashb ... }}, "files/<file> ... 4")
18. /var/www/nextcloud/apps/files_trashbin/lib/Storage.php line 192
    OCA\Files_Trashbin\Trash\TrashManager->moveToTrash(OCA\Files_Trashb ... }}, "files/<file> ... 4")
19. /var/www/nextcloud/apps/files_trashbin/lib/Storage.php line 99
    OCA\Files_Trashbin\Storage->doDelete("files/<file> ... 4", "unlink")
20. /var/www/nextcloud/lib/private/Files/View.php line 1161
    OCA\Files_Trashbin\Storage->unlink("files/<file> ... 4")
21. /var/www/nextcloud/lib/private/Files/View.php line 718
    OC\Files\View->basicOperation("unlink", "/<file> ... 4", ["delete"])
22. /var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php line 455
    OC\Files\View->unlink("/<file> ... 4")
23. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php line 183
    OCA\DAV\Connector\Sabre\File->delete()
24. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 295
    Sabre\DAV\Tree->delete("files/ ... 4")
25. /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->httpDelete(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
26. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 474
    Sabre\DAV\Server->emit("method:DELETE", [Sabre\HTTP\Requ ... }])
27. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 251
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
28. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 319
    Sabre\DAV\Server->start()
29. /var/www/nextcloud/apps/dav/lib/Server.php line 320
    Sabre\DAV\Server->exec()
30. /var/www/nextcloud/apps/dav/appinfo/v2/remote.php line 35
    OCA\DAV\Server->exec()
31. /var/www/nextcloud/remote.php line 167
    require_once("/var/www/nextcl ... p")

DELETE /remote.php/dav/files/...
from <ip> by <user> at 2020-08-31T08:12:52+02:00

UPDATE `oc_filecache` SET `storage` = ?, `path_hash` = MD5(CONCAT(?, SUBSTR(`path`, ?))), `path` = CONCAT(?, SUBSTR(`path`, ?)) 

[webdav] Fatal: Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing 'UPDATE `oc_filecache` SET `storage` = ?, `path_hash` = MD5(CONCAT(?, SUBSTR(`path`, ?))), `path` = CONCAT(?, SUBSTR(`path`, ?)) WHERE (`storage` = ?) AND (`path` LIKE ?)' with params [9, "files_trashbin\/files\/08.d1598854370", 59, "files_trashbin\/files\/08.d1598854370", 59, 9, "files\/<file>\/%"]:

SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction at <<closure>>

 0. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 169
    Doctrine\DBAL\Driver\AbstractMySQLDriver->convertException("An exception oc ... n", Doctrine\DBAL\Dr ... ]})
 1. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php line 145
    Doctrine\DBAL\DBALException::wrapException(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "An exception oc ... n")
 2. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Connection.php line 1063
    Doctrine\DBAL\DBALException::driverExceptionDuringQuery(Doctrine\DBAL\Driver\PDOMySql\Driver {}, Doctrine\DBAL\Dr ... ]}, "UPDATE `oc_file ... )", {1: 9,2: "files_ ... "})
 3. /var/www/nextcloud/lib/private/DB/Connection.php line 214
    Doctrine\DBAL\Connection->executeUpdate("UPDATE `oc_file ... )", [9,"files_trashb ... "], [1,2,1,2,1,1,2])
 4. /var/www/nextcloud/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Query/QueryBuilder.php line 203
    OC\DB\Connection->executeUpdate("UPDATE `oc_file ... )", {dcValue1: "file ... "}, {dcValue1: 2,dcV ... 2})
 5. /var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php line 216
    Doctrine\DBAL\Query\QueryBuilder->execute()
 6. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 647
    OC\DB\QueryBuilder\QueryBuilder->execute()
 7. /var/www/nextcloud/lib/private/Files/Cache/Cache.php line 585
    OC\Files\Cache\Cache->moveFromCache(OC\Files\Cache\HomeCache {}, "files/<file> ... 8", "files_trashbin/files/08.d1598854370")
 8. /var/www/nextcloud/lib/private/Files/Cache/Updater.php line 199
    OC\Files\Cache\Cache->move("files/<file> ... 8", "files_trashbin/files/08.d1598854370")
 9. /var/www/nextcloud/apps/files_trashbin/lib/Trashbin.php line 289
    OC\Files\Cache\Updater->renameFromStorage(OCA\Files_Trashb ... }}, "files/<file> ... 8", "files_trashbin/files/08.d1598854370")
10. /var/www/nextcloud/apps/files_trashbin/lib/Trash/LegacyTrashBackend.php line 108
    OCA\Files_Trashbin\Trashbin::move2trash("<file> ... 8")
11. /var/www/nextcloud/apps/files_trashbin/lib/Trash/TrashManager.php line 103
    OCA\Files_Trashbin\Trash\LegacyTrashBackend->moveToTrash(OCA\Files_Trashb ... }}, "files/<file> ... 8")
12. /var/www/nextcloud/apps/files_trashbin/lib/Storage.php line 192
    OCA\Files_Trashbin\Trash\TrashManager->moveToTrash(OCA\Files_Trashb ... }}, "files/<file> ... 8")
13. /var/www/nextcloud/apps/files_trashbin/lib/Storage.php line 118
    OCA\Files_Trashbin\Storage->doDelete("files/<file> ... 8", "rmdir")
14. /var/www/nextcloud/lib/private/Files/View.php line 1161
    OCA\Files_Trashbin\Storage->rmdir("files/<file> ... 8")
15. /var/www/nextcloud/lib/private/Files/View.php line 350
    OC\Files\View->basicOperation("rmdir", "/<file> ... 8", ["delete"])
16. /var/www/nextcloud/apps/dav/lib/Connector/Sabre/Directory.php line 308
    OC\Files\View->rmdir("/<file> ... 8")
17. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php line 183
    OCA\DAV\Connector\Sabre\Directory->delete()
18. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 295
    Sabre\DAV\Tree->delete("files/<file> ... 8")
19. /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->httpDelete(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
20. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 474
    Sabre\DAV\Server->emit("method:DELETE", [Sabre\HTTP\Requ ... }])
21. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 251
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
22. /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 319
    Sabre\DAV\Server->start()
23. /var/www/nextcloud/apps/dav/lib/Server.php line 320
    Sabre\DAV\Server->exec()
24. /var/www/nextcloud/apps/dav/appinfo/v2/remote.php line 35
    OCA\DAV\Server->exec()
25. /var/www/nextcloud/remote.php line 167
    require_once("/var/www/nextcl ... p")

DELETE /remote.php/dav/files/<file>
from <ip> by <user> at 2020-08-31T08:12:50+02:00

@someone-somenet-org
Copy link

+1 on tthe issue. I seem to be unable to delete folders that contain subfolders since the update.

Doing so will make the linux desktop sync-client throw errors at me for hours at which point I just killed the sync client for now...

Doing a scan --all now with the hope it will fix things.

@solracsf solracsf mentioned this issue Sep 3, 2020
Closed
@nickvergessen nickvergessen mentioned this issue Sep 3, 2020
Closed
@ghost

This comment has been minimized.

Sign in to view
@solracsf
Copy link
Member

solracsf commented Sep 3, 2020

#22563 may fix this. Feedback welcome.

@linuxrrze
Copy link

Just applied patch 4a3f6a7 to nextcloud 19.0.2 and restarted apache2.

Trying to delete multiple files at once still gives the same error messages.

@cy2201

This comment has been minimized.

Sign in to view
@solracsf
Copy link
Member

solracsf commented Sep 5, 2020 •
edited
Loading 

MariaDB [(none)]> SELECT @@GLOBAL.tx_isolation, @@tx_isolation;
+-----------------------+----------------+
| @@GLOBAL.tx_isolation | @@tx_isolation |
+-----------------------+----------------+
| READ-COMMITTED        | READ-COMMITTED |
+-----------------------+----------------+

Output of MariaDB SHOW ENGINE INNODB STATUS\G:

------------------------
LATEST DETECTED DEADLOCK
------------------------
2020-09-04 12:45:28 0x7fce70f63700
*** (1) TRANSACTION:
TRANSACTION 91989232, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 4 lock struct(s), heap size 1128, 4 row lock(s), undo log entries 1
MySQL thread id 257055, OS thread handle 140524754528000, query id 6196914 10.1.0.4 nextcloud Updating
UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, 1599216328), `etag` = '5f521ac8effdb' WHERE (`storage` = 1) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', '56c303da11d07c61688947bb3296276a'))
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 13131 page no 245 n bits 136 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 91989232 lock_mode X locks rec but not gap waiting
Record lock, heap no 46 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 800000000000ba29; asc        );;
 1: len 6; hex 0000057ba4e3; asc    {  ;;
 2: len 7; hex 27000001a209a0; asc '      ;;
 3: len 8; hex 8000000000000001; asc         ;;
 4: len 10; hex 66696c65732f54657374; asc files/Test;;
 5: len 30; hex 353663333033646131316430376336313638383934376262333239363237; asc 56c303da11d07c61688947bb329627; (total 32 bytes);
 6: len 8; hex 8000000000000002; asc         ;;
 7: len 4; hex 54657374; asc Test;;
 8: len 8; hex 8000000000000002; asc         ;;
 9: len 8; hex 8000000000000001; asc         ;;
 10: len 8; hex 80000000004f9b3e; asc      O >;;
 11: len 8; hex 800000005f521ac8; asc     _R  ;;
 12: len 8; hex 800000005f521ac8; asc     _R  ;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 35663532316163386565346637; asc 5f521ac8ee4f7;;
 16: len 4; hex 8000001f; asc     ;;
 17: SQL NULL;

*** (2) TRANSACTION:
TRANSACTION 91989219, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
4 lock struct(s), heap size 1128, 3 row lock(s), undo log entries 2
MySQL thread id 257058, OS thread handle 140524635174656, query id 6196920 10.1.0.5 nextcloud Updating
UPDATE `oc_filecache` SET `size` = '5217086' WHERE (`fileid` = 2) AND ((`size` <> '5217086') OR (`size` IS NULL))
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 13131 page no 245 n bits 136 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 91989219 lock_mode X locks rec but not gap
Record lock, heap no 46 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 800000000000ba29; asc        );;
 1: len 6; hex 0000057ba4e3; asc    {  ;;
 2: len 7; hex 27000001a209a0; asc '      ;;
 3: len 8; hex 8000000000000001; asc         ;;
 4: len 10; hex 66696c65732f54657374; asc files/Test;;
 5: len 30; hex 353663333033646131316430376336313638383934376262333239363237; asc 56c303da11d07c61688947bb329627; (total 32 bytes);
 6: len 8; hex 8000000000000002; asc         ;;
 7: len 4; hex 54657374; asc Test;;
 8: len 8; hex 8000000000000002; asc         ;;
 9: len 8; hex 8000000000000001; asc         ;;
 10: len 8; hex 80000000004f9b3e; asc      O >;;
 11: len 8; hex 800000005f521ac8; asc     _R  ;;
 12: len 8; hex 800000005f521ac8; asc     _R  ;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 35663532316163386565346637; asc 5f521ac8ee4f7;;
 16: len 4; hex 8000001f; asc     ;;
 17: SQL NULL;

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 13131 page no 10 n bits 160 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 91989219 lock_mode X locks rec but not gap waiting
Record lock, heap no 3 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 8000000000000002; asc         ;;
 1: len 6; hex 0000057ba4f0; asc    {  ;;
 2: len 7; hex 2f000001a10794; asc /      ;;
 3: len 8; hex 8000000000000001; asc         ;;
 4: len 5; hex 66696c6573; asc files;;
 5: len 30; hex 343562393633333937616134306434613030363365306438356534666537; asc 45b963397aa40d4a0063e0d85e4fe7; (total 32 bytes);
 6: len 8; hex 8000000000000001; asc         ;;
 7: len 5; hex 66696c6573; asc files;;
 8: len 8; hex 8000000000000002; asc         ;;
 9: len 8; hex 8000000000000001; asc         ;;
 10: len 8; hex 80000000004ad20b; asc      J  ;;
 11: len 8; hex 800000005f521ac8; asc     _R  ;;
 12: len 8; hex 800000005f521217; asc     _R  ;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 35663532316163386566666462; asc 5f521ac8effdb;;
 16: len 4; hex 8000001f; asc     ;;
 17: SQL NULL;

*** WE ROLL BACK TRANSACTION (1)

@ghost
Copy link
Author

ghost commented Sep 5, 2020

now im having sync issues i deleted these files but somehow still in the system. this bug is very bad

image

@ghost

This comment has been minimized.

Sign in to view
@ghost

This comment has been minimized.

Sign in to view
@skjnldsv skjnldsv added the 1. to develop Accepted and waiting to be taken care of label Sep 9, 2020
@mblo
Copy link

mblo commented Sep 9, 2020

Same problems here, on 19.0.2 and mysql. This bug also occurs when a Windows sync client reads files from a server (w/o any write operations). This is a production system, so I cannot install a patch, but I'm happy to provide more (debugging) information if required.

@szaimen szaimen added the 0. Needs triage Pending check for reproducibility or if it fits our roadmap label Nov 26, 2022
@nextcloud nextcloud unlocked this conversation Nov 26, 2022
@nextcloud-command
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

@nextcloud-command nextcloud-command added the stale Ticket or PR with no recent activity label Jan 13, 2023
@solracsf
Copy link
Member

v24.0.9, today (v25 is unusable in production for now, too much buggy).
image

@solracsf solracsf removed needs info stale Ticket or PR with no recent activity labels Jan 20, 2023
@simonspa
Copy link
Contributor

Still happening on 25.0.3. NC log:

{"reqId":"PNGe4jxUV1R9Lb4UfaOs","level":3,"time":"2023-01-19T11:41:25+01:00","remoteAddr":"87.190.6.181","user":"myusern","app":"no app in context","method":"PUT","url":"/remote.php/dav/files/myuser/file.jpg","message":"An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","userAgent":"Mozilla/5.0 (Linux) mirall/3.5.4-20220806.084713.fea986309-1.0~focal1 (Nextcloud, ubuntu-5.4.0-136-generic ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.3.2","exception":{"Exception":"Doctrine\\DBAL\\Exception\\DeadlockException","Message":"An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","Code":1213,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1780,"function":"convert","class":"Doctrine\\DBAL\\Driver\\API\\MySQL\\ExceptionConverter","type":"->","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDO\\Exception"},{"__class__":"Doctrine\\DBAL\\Query"}]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1719,"function":"handleDriverException","class":"Doctrine\\DBAL\\Connection","type":"->","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDO\\Exception"},{"__class__":"Doctrine\\DBAL\\Query"}]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1173,"function":"convertExceptionDuringQuery","class":"Doctrine\\DBAL\\Connection","type":"->","args":[{"__class__":"Doctrine\\DBAL\\Driver\\PDO\\Exception"},"UPDATE `oc_filecache` SET `mtime` = ?, `storage_mtime` = ? WHERE (`fileid` = ?) AND (((`mtime` <> ?) OR (`mtime` IS NULL)) OR ((`storage_mtime` <> ?) OR (`storage_mtime` IS NULL)))",["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"],[2,2,1,2,2]]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":294,"function":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->","args":["UPDATE `oc_filecache` SET `mtime` = ?, `storage_mtime` = ? WHERE (`fileid` = ?) AND (((`mtime` <> ?) OR (`mtime` IS NULL)) OR ((`storage_mtime` <> ?) OR (`storage_mtime` IS NULL)))",["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"],[2,2,1,2,2]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":354,"function":"executeStatement","class":"OC\\DB\\Connection","type":"->","args":["UPDATE `oc_filecache` SET `mtime` = :dcValue4, `storage_mtime` = :dcValue5 WHERE (`fileid` = :dcValue1) AND (((`mtime` <> :dcValue2) OR (`mtime` IS NULL)) OR ((`storage_mtime` <> :dcValue3) OR (`storage_mtime` IS NULL)))",["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"],[1,2,2,2,2]]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":281,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Cache.php","line":382,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Updater.php","line":256,"function":"update","class":"OC\\Files\\Cache\\Cache","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Updater.php","line":138,"function":"correctParentStorageMtime","class":"OC\\Files\\Cache\\Updater","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":322,"function":"update","class":"OC\\Files\\Cache\\Updater","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":942,"function":"writeUpdate","class":"OC\\Files\\View","type":"->","args":[{"__class__":"OCA\\Files_Trashbin\\Storage","cache":null,"scanner":{"__class__":"OC\\Files\\Cache\\Scanner"},"watcher":null,"propagator":null,"updater":{"__class__":"OC\\Files\\Cache\\Updater"}},"*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Versions/LegacyVersionsBackend.php","line":93,"function":"copy","class":"OC\\Files\\View","type":"->","args":["/files/file.jpg","/files_versions/file.jpg.v1521974585"]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Versions/VersionManager.php","line":92,"function":"createVersion","class":"OCA\\Files_Versions\\Versions\\LegacyVersionsBackend","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Node\\File"}]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Storage.php","line":238,"function":"createVersion","class":"OCA\\Files_Versions\\Versions\\VersionManager","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Node\\File"}]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Hooks.php","line":56,"function":"store","class":"OCA\\Files_Versions\\Storage","type":"::","args":["/file.jpg"]},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Hook.php","line":106,"function":"write_hook","class":"OCA\\Files_Versions\\Hooks","type":"::","args":[["/file.jpg",true]]},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":448,"function":"emit","class":"OC_Hook","type":"::","args":["OC_Filesystem","write",["/file.jpg",true]]},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":178,"function":"emitPreHooks","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[true]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1137,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":492,"function":"updateFile","class":"Sabre\\DAV\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:PUT",[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":360,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/remote.php","line":171,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php","Line":37,"Previous":{"Exception":"Doctrine\\DBAL\\Driver\\PDO\\Exception","Message":"SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","Code":1213,"Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","line":94,"function":"new","class":"Doctrine\\DBAL\\Driver\\PDO\\Exception","type":"::","args":[{"__class__":"PDOException","errorInfo":["40001",1213,"Deadlock found when trying to get lock; try restarting transaction"]}]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1163,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDO\\Statement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":294,"function":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->","args":["UPDATE `oc_filecache` SET `mtime` = ?, `storage_mtime` = ? WHERE (`fileid` = ?) AND (((`mtime` <> ?) OR (`mtime` IS NULL)) OR ((`storage_mtime` <> ?) OR (`storage_mtime` IS NULL)))",["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"],[2,2,1,2,2]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":354,"function":"executeStatement","class":"OC\\DB\\Connection","type":"->","args":["UPDATE `oc_filecache` SET `mtime` = :dcValue4, `storage_mtime` = :dcValue5 WHERE (`fileid` = :dcValue1) AND (((`mtime` <> :dcValue2) OR (`mtime` IS NULL)) OR ((`storage_mtime` <> :dcValue3) OR (`storage_mtime` IS NULL)))",["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"],[1,2,2,2,2]]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":281,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Cache.php","line":382,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Updater.php","line":256,"function":"update","class":"OC\\Files\\Cache\\Cache","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Updater.php","line":138,"function":"correctParentStorageMtime","class":"OC\\Files\\Cache\\Updater","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":322,"function":"update","class":"OC\\Files\\Cache\\Updater","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":942,"function":"writeUpdate","class":"OC\\Files\\View","type":"->","args":[{"__class__":"OCA\\Files_Trashbin\\Storage","cache":null,"scanner":{"__class__":"OC\\Files\\Cache\\Scanner"},"watcher":null,"propagator":null,"updater":{"__class__":"OC\\Files\\Cache\\Updater"}},"*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Versions/LegacyVersionsBackend.php","line":93,"function":"copy","class":"OC\\Files\\View","type":"->","args":["/files/file.jpg","/files_versions/file.jpg.v1521974585"]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Versions/VersionManager.php","line":92,"function":"createVersion","class":"OCA\\Files_Versions\\Versions\\LegacyVersionsBackend","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Node\\File"}]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Storage.php","line":238,"function":"createVersion","class":"OCA\\Files_Versions\\Versions\\VersionManager","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Node\\File"}]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Hooks.php","line":56,"function":"store","class":"OCA\\Files_Versions\\Storage","type":"::","args":["/file.jpg"]},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Hook.php","line":106,"function":"write_hook","class":"OCA\\Files_Versions\\Hooks","type":"::","args":[["/file.jpg",true]]},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":448,"function":"emit","class":"OC_Hook","type":"::","args":["OC_Filesystem","write",["/file.jpg",true]]},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":178,"function":"emitPreHooks","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[true]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1137,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":492,"function":"updateFile","class":"Sabre\\DAV\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:PUT",[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":360,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/remote.php","line":171,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php","Line":26,"Previous":{"Exception":"PDOException","Message":"SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","Code":"40001","Trace":[{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","line":92,"function":"execute","class":"PDOStatement","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php","line":1163,"function":"execute","class":"Doctrine\\DBAL\\Driver\\PDO\\Statement","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/DB/Connection.php","line":294,"function":"executeStatement","class":"Doctrine\\DBAL\\Connection","type":"->","args":["UPDATE `oc_filecache` SET `mtime` = ?, `storage_mtime` = ? WHERE (`fileid` = ?) AND (((`mtime` <> ?) OR (`mtime` IS NULL)) OR ((`storage_mtime` <> ?) OR (`storage_mtime` IS NULL)))",["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"],[2,2,1,2,2]]},{"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php","line":354,"function":"executeStatement","class":"OC\\DB\\Connection","type":"->","args":["UPDATE `oc_filecache` SET `mtime` = :dcValue4, `storage_mtime` = :dcValue5 WHERE (`fileid` = :dcValue1) AND (((`mtime` <> :dcValue2) OR (`mtime` IS NULL)) OR ((`storage_mtime` <> :dcValue3) OR (`storage_mtime` IS NULL)))",["*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***","*** sensitive parameters replaced ***"],[1,2,2,2,2]]},{"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php","line":281,"function":"execute","class":"Doctrine\\DBAL\\Query\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Cache.php","line":382,"function":"execute","class":"OC\\DB\\QueryBuilder\\QueryBuilder","type":"->","args":[]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Updater.php","line":256,"function":"update","class":"OC\\Files\\Cache\\Cache","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Updater.php","line":138,"function":"correctParentStorageMtime","class":"OC\\Files\\Cache\\Updater","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":322,"function":"update","class":"OC\\Files\\Cache\\Updater","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/View.php","line":942,"function":"writeUpdate","class":"OC\\Files\\View","type":"->","args":[{"__class__":"OCA\\Files_Trashbin\\Storage","cache":null,"scanner":{"__class__":"OC\\Files\\Cache\\Scanner"},"watcher":null,"propagator":null,"updater":{"__class__":"OC\\Files\\Cache\\Updater"}},"*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Versions/LegacyVersionsBackend.php","line":93,"function":"copy","class":"OC\\Files\\View","type":"->","args":["/files/file.jpg","/files_versions/file.jpg.v1521974585"]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Versions/VersionManager.php","line":92,"function":"createVersion","class":"OCA\\Files_Versions\\Versions\\LegacyVersionsBackend","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Node\\File"}]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Storage.php","line":238,"function":"createVersion","class":"OCA\\Files_Versions\\Versions\\VersionManager","type":"->","args":[{"__class__":"OC\\User\\User"},{"__class__":"OC\\Files\\Node\\File"}]},{"file":"/var/www/nextcloud/apps/files_versions/lib/Hooks.php","line":56,"function":"store","class":"OCA\\Files_Versions\\Storage","type":"::","args":["/file.jpg"]},{"file":"/var/www/nextcloud/lib/private/legacy/OC_Hook.php","line":106,"function":"write_hook","class":"OCA\\Files_Versions\\Hooks","type":"::","args":[["/file.jpg",true]]},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":448,"function":"emit","class":"OC_Hook","type":"::","args":["OC_Filesystem","write",["/file.jpg",true]]},{"file":"/var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":178,"function":"emitPreHooks","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[true]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":1137,"function":"put","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":492,"function":"updateFile","class":"Sabre\\DAV\\Server","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPut","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:PUT",[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/dav/lib/Server.php","line":360,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/nextcloud/remote.php","line":171,"args":["/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php","Line":92}},"CustomMessage":"--"},"id":"63cacaeae4c5b"}

Corresponding MariaDB transaction log:

------------------------
LATEST DETECTED DEADLOCK
------------------------
2023-01-19 11:41:25 0x7fcb940b2700
*** (1) TRANSACTION:
TRANSACTION 1204878026, ACTIVE 0 sec starting index read
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1128, 2 row lock(s)
MySQL thread id 427866, OS thread handle 140490079287040, query id 42206920 localhost nextcloud_user Updating
UPDATE `oc_filecache` SET `mtime` = '1674124885', `storage_mtime` = '1674124885' WHERE (`fileid` = 3401087) AND (((`mtime` <> '1674124885') OR (`mtime` IS NULL)) OR ((`storage_mtime` <> '1674124885') OR (`storage_mtime` IS NULL)))
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 572 page no 94093 n bits 136 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 1204878026 lock_mode X locks rec but not gap waiting
Record lock, heap no 65 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 800000000033e57f; asc      3  ;;
 1: len 6; hex 000047d0fac8; asc   G   ;;
 2: len 7; hex 5200000e75264c; asc R   u&L;;
 3: len 8; hex 8000000000000007; asc         ;;
 4: len 30; hex 66696c65735f76657273696f6e732f466f746f73206d6f62696c2f323031; asc files_versions/file; (total 34 bytes);
 5: len 30; hex 396666323134306465313937616439396563626565333365343530356634; asc 9ff2140de197ad99ecbee33e4505f4; (total 32 bytes);
 6: len 8; hex 800000000033e57e; asc      3 ~;;
 7: len 2; hex 3033; asc 03;;
 8: len 8; hex 8000000000000001; asc         ;;
 9: len 8; hex 8000000000000002; asc         ;;
 10: len 8; hex 800000000160dae1; asc      `  ;;
 11: len 8; hex 8000000063c91e55; asc     c  U;;
 12: len 8; hex 8000000063c91e55; asc     c  U;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 36336339316535353036393533; asc 63c91e5506953;;
 16: len 4; hex 8000001f; asc     ;;
 17: len 0; hex ; asc ;;

*** (2) TRANSACTION:
TRANSACTION 1204878024, ACTIVE 0 sec updating or deleting
mysql tables in use 1, locked 1
10 lock struct(s), heap size 1128, 9 row lock(s), undo log entries 4
MySQL thread id 427865, OS thread handle 140512338847488, query id 42206916 localhost nextcloud_user Updating
UPDATE `oc_filecache` SET `mtime` = GREATEST(`mtime`, 1674124885), `etag` = '63c91e5506953', `size` = CASE WHEN `size` > '-1' THEN GREATEST(`size` + '1060361', -1) ELSE `size` END WHERE (`storage` = 7) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '9692aae50022f45f1098646939b287b1', '367130d1ca65778842a4fe2955fa77ae', '6122674d8331fe3d0acab8a556281f0d', '9ff2140de197ad99ecbee33e4505f4a7'))
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 572 page no 94093 n bits 136 index PRIMARY of table `nextcloud`.`oc_filecache` trx id 1204878024 lock_mode X locks rec but not gap
Record lock, heap no 65 PHYSICAL RECORD: n_fields 18; compact format; info bits 0
 0: len 8; hex 800000000033e57f; asc      3  ;;
 1: len 6; hex 000047d0fac8; asc   G   ;;
 2: len 7; hex 5200000e75264c; asc R   u&L;;
 3: len 8; hex 8000000000000007; asc         ;;
 4: len 30; hex 66696c65735f76657273696f6e732f466f746f73206d6f62696c2f323031; asc files_versions/file; (total 34 bytes);
 5: len 30; hex 396666323134306465313937616439396563626565333365343530356634; asc 9ff2140de197ad99ecbee33e4505f4; (total 32 bytes);
 6: len 8; hex 800000000033e57e; asc      3 ~;;
 7: len 2; hex 3033; asc 03;;
 8: len 8; hex 8000000000000001; asc         ;;
 9: len 8; hex 8000000000000002; asc         ;;
 10: len 8; hex 800000000160dae1; asc      `  ;;
 11: len 8; hex 8000000063c91e55; asc     c  U;;
 12: len 8; hex 8000000063c91e55; asc     c  U;;
 13: len 4; hex 80000000; asc     ;;
 14: len 8; hex 8000000000000000; asc         ;;
 15: len 13; hex 36336339316535353036393533; asc 63c91e5506953;;
 16: len 4; hex 8000001f; asc     ;;
 17: len 0; hex ; asc ;;

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 572 page no 39799 n bits 344 index fs_id_storage_size of table `nextcloud`.`oc_filecache` trx id 1204878024 lock_mode X locks rec but not gap waiting
Record lock, heap no 189 PHYSICAL RECORD: n_fields 3; compact format; info bits 0
 0: len 8; hex 800000000033e57f; asc      3  ;;
 1: len 8; hex 8000000000000007; asc         ;;
 2: len 8; hex 800000000150acd8; asc      P  ;;

*** WE ROLL BACK TRANSACTION (1)

@Sieboldianus
Copy link

Sieboldianus commented Jan 31, 2023 •
edited
Loading

I am also seeing these issues more often again since upgrading from 24.0.9 to 25.0.3

Doctrine\DBAL\Exception\DeadlockException: An exception occurred while executing a query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction

I should have waited with the major upgrade, given all the issues with v25 :(

After a few seconds, the client syncs again and then it usually works. I am on 32GB Xeon Server Hardware, with Raidz2 on ZFS.

@JosefWN
Copy link

JosefWN commented Jan 31, 2023

I've also had this issue, and perhaps coincidentally I'm also on ZFS running raidz2.

@obel1x
Copy link

obel1x commented Feb 16, 2023

@andyxheli @simonspa
Looks like its related to the same behaviour as #6899. See my proposal to reduce deadlocks by changing indices on table oc_filecache. Can you check #6899 (comment) and leave a thumb up there if it works?

@solracsf
Copy link
Member

See #37820

@ghost
Copy link
Author

ghost commented Apr 21, 2023

Thanks @solracsf keeping fingers 🤞. Hopefully this dose the trick.

@JosefWN
Copy link

JosefWN commented Apr 21, 2023 •
edited
Loading

Without any knowledge of the code, would it be possible to use semaphores or such to prevent deadlocks from ever occurring? Reducing the probability of a deadlock occurring sounds like a mitigating measure, which is good, but perhaps not enough to write the issue off altogether?

EDIT: Say you batch upload some 10 000 files in the Web UI and the issue occurs in 1% of the uploaded files, or even 0.1%, it will be quite messy to find out which portion of the files that is in order to maintain data integrity.

@obel1x
Copy link

obel1x commented May 2, 2023

just left a comment to this here to explain why this happes imo: #29985 (comment)

@szaimen
Copy link
Contributor

szaimen commented Jun 17, 2023

Hi, there have been patches to address this issue that will land in 26.0.3. Can you please verify if 26.0.3-rc1 improves the situation around this issue?

@keithf4
Copy link

keithf4 commented Jun 17, 2023

Just curious if you mean this patch?

#38479

Unless I'm mistaken, this appears to just be ignoring the deadlock exception. If that's the case, while this may remove the deadlock error from the logs, it leaves another error and doesn't actually solve the problem as laid out by @obel1x. It also still leaves the errors in the database logs.

2023-06-16 22:39:17.362 EDT [192.168.1.201(21060)] [42816]: [l-2] user=owncloud,db=nextcloud,app=[unknown],e=40P01 DETAIL:  Process 42816 waits for ShareLock on transaction 7552200; blocked by process 42812.
        Process 42812 waits for ShareLock on transaction 7552196; blocked by process 42816.
        Process 42816: UPDATE "oc_filecache" SET "mtime" = GREATEST("mtime", $1), "etag" = $2, "size" = CASE WHEN "size" > '-1' THEN GREATEST("size" + $3, $4) ELSE "size" END WHERE ("storage" = $5) AND ("path_hash" IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', '69e3dbe03683dc4243000a1a15ce7396', '91b7f93e1c18dd3a7e009540fc8902d6', '59a0c074ac2079a35d6c8dd4533a5819', 'b42fcd8e462c5d41a3f27b007eccaa19', '3db43dcf520efef692085e18f3f3b4c4'))
        Process 42812: UPDATE "oc_filecache" SET "mtime" = GREATEST("mtime", $1), "etag" = $2 WHERE ("storage" = $3) AND ("path_hash" IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', '69e3dbe03683dc4243000a1a15ce7396', '91b7f93e1c18dd3a7e009540fc8902d6', '59a0c074ac2079a35d6c8dd4533a5819', 'b42fcd8e462c5d41a3f27b007eccaa19', '01bcf381cceb817c39e6a4ef06af1d36'))
2023-06-16 22:39:17.362 EDT [192.168.1.201(21060)] [42816]: [l-3] user=owncloud,db=nextcloud,app=[unknown],e=40P01 HINT:  See server log for query details.
2023-06-16 22:39:17.362 EDT [192.168.1.201(21060)] [42816]: [l-4] user=owncloud,db=nextcloud,app=[unknown],e=40P01 CONTEXT:  while rechecking updated tuple (797,28) in relation "oc_filecache"
2023-06-16 22:39:17.362 EDT [192.168.1.201(21060)] [42816]: [l-5] user=owncloud,db=nextcloud,app=[unknown],e=40P01 STATEMENT:  UPDATE "oc_filecache" SET "mtime" = GREATEST("mtime", $1), "etag" = $2, "size" = CASE WHEN "size" > '-1' THEN GREATEST("size" + $3, $4) ELSE "size" END WHERE ("storage" = $5) AND ("path_hash" IN ('d41d8cd98f00b204e9800998ecf8427e', '45b963397aa40d4a0063e0d85e4fe7a1', '69e3dbe03683dc4243000a1a15ce7396', '91b7f93e1c18dd3a7e009540fc8902d6', '59a0c074ac2079a35d6c8dd4533a5819', 'b42fcd8e462c5d41a3f27b007eccaa19', '3db43dcf520efef692085e18f3f3b4c4'))

@szaimen
Copy link
Contributor

szaimen commented Jun 21, 2023

See #6899 (comment)

@nextcloud-command
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

@nextcloud-command nextcloud-command added the stale Ticket or PR with no recent activity label Aug 3, 2023
@ghost
Copy link
Author

ghost commented Aug 18, 2023

Still an issue on NC 27.0.2

@akoskoronka
Copy link

Same problem on NC 28.0.2

@ThierryBZH
Copy link

Same on NC 29.0.2...

@ghost
Copy link
Author

ghost commented Jun 19, 2024

Can we please get this issue reopened as the issue still happens on 29.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 25-feedback bug needs info stale Ticket or PR with no recent activity
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Re-fix transaction isolation level nextcloud/server