Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore from trashbin triggers CacheQueryBuilder::whereFileId() Fatal error #1218

Closed
solracsf opened this issue Jan 6, 2021 · 7 comments
Closed
Labels
1. to develop Issues that are ready for development bug feature: caching Items related to caching feature: trashbin Items related to the trashbin feature integration s3

Comments

@solracsf
Copy link
Member

solracsf commented Jan 6, 2021

tl;dr

Reproduce case outside Group Folders on the same environment works.

Env

  • NC 20.0.4
  • GF 8.2.0
  • PHP 7.4
  • MySQL 8.0.22
  • Redis 5 as local, distributed and locking Memcache
  • S3 as primary storage

Reproduce

  • Delete a folder inside a Group Folder
  • Go to trashbin and restore it
  • A message says it has failed
Fatal webdav TypeError: Argument 1 passed to OC\Files\Cache\CacheQueryBuilder::whereFileId() must be of the type int, null given, called in /lib/private/Files/Cache/Cache.php on line 676
Error PHP Error: Trying to access array offset on value of type bool at /lib/private/Files/Cache/Cache.php#644
Error PHP Error: Trying to access array offset on value of type bool at /lib/private/Files/Cache/Cache.php#630
Log Messages ``` { "reqId": "0P3singiqHu88qAk7VRF", "level": 4, "time": "2021-01-06T15:01:05+01:00", "app": "webdav", "method": "MOVE", "url": "/remote.php/dav/trashbin/user/trash/2.%20Product%20Datasheets.d1609877680", "message": { "Exception": "TypeError", "Message": "Argument 1 passed to OC\\Files\\Cache\\CacheQueryBuilder::whereFileId() must be of the type int, null given, called in /lib/private/Files/Cache/Cache.php on line 676", "Code": 0, "Trace": [ { "file": "/lib/private/Files/Cache/Cache.php", "line": 676, "function": "whereFileId", "class": "OC\\Files\\Cache\\CacheQueryBuilder", "type": "->", "args": [ null ] }, { "file": "/apps/groupfolders/lib/Trash/TrashBackend.php", "line": 147, "function": "moveFromCache", "class": "OC\\Files\\Cache\\Cache", "type": "->", "args": [ { "__class__": "OC\\Files\\Cache\\Cache" }, "__groupfolders/trash/3/2. Product Datasheets.d1609941656", "__groupfolders/3/2. Product Datasheets" ] }, { "file": "/apps/files_trashbin/lib/Trash/TrashManager.php", "line": 65, "function": "restoreItem", "class": "OCA\\GroupFolders\\Trash\\TrashBackend", "type": "->", "args": [ { "__class__": "OCA\\GroupFolders\\Trash\\GroupTrashItem" } ] }, { "file": "/apps/files_trashbin/lib/Sabre/AbstractTrash.php", "line": 94, "function": "restoreItem", "class": "OCA\\Files_Trashbin\\Trash\\TrashManager", "type": "->", "args": [ { "__class__": "OCA\\GroupFolders\\Trash\\GroupTrashItem" } ] }, { "file": "/apps/files_trashbin/lib/Sabre/RestoreFolder.php", "line": 76, "function": "restore", "class": "OCA\\Files_Trashbin\\Sabre\\AbstractTrash", "type": "->", "args": [] }, { "file": "/3rdparty/sabre/dav/lib/DAV/Tree.php", "line": 164, "function": "moveInto", "class": "OCA\\Files_Trashbin\\Sabre\\RestoreFolder", "type": "->", "args": [ "2. Product Datasheets.d1609877680", "trashbin/user/trash/2. Product Datasheets.d1609877680", { "__class__": "OCA\\Files_Trashbin\\Sabre\\TrashFolder" } ] }, { "file": "/3rdparty/sabre/dav/lib/DAV/CorePlugin.php", "line": 641, "function": "move", "class": "Sabre\\DAV\\Tree", "type": "->", "args": [ "trashbin/user/trash/2. Product Datasheets.d1609877680", "trashbin/user/restore/2. Product Datasheets.d1609877680" ] }, { "file": "/3rdparty/sabre/event/lib/WildcardEmitterTrait.php", "line": 89, "function": "httpMove", "class": "Sabre\\DAV\\CorePlugin", "type": "->", "args": [ { "__class__": "Sabre\\HTTP\\Request" }, { "__class__": "Sabre\\HTTP\\Response" } ] }, { "file": "/3rdparty/sabre/dav/lib/DAV/Server.php", "line": 474, "function": "emit", "class": "Sabre\\DAV\\Server", "type": "->", "args": [ "method:MOVE", [ { "__class__": "Sabre\\HTTP\\Request" }, { "__class__": "Sabre\\HTTP\\Response" } ] ] }, { "file": "/3rdparty/sabre/dav/lib/DAV/Server.php", "line": 251, "function": "invokeMethod", "class": "Sabre\\DAV\\Server", "type": "->", "args": [ { "__class__": "Sabre\\HTTP\\Request" }, { "__class__": "Sabre\\HTTP\\Response" } ] }, { "file": "/3rdparty/sabre/dav/lib/DAV/Server.php", "line": 319, "function": "start", "class": "Sabre\\DAV\\Server", "type": "->", "args": [] }, { "file": "/apps/dav/lib/Server.php", "line": 332, "function": "exec", "class": "Sabre\\DAV\\Server", "type": "->", "args": [] }, { "file": "/apps/dav/appinfo/v2/remote.php", "line": 35, "function": "exec", "class": "OCA\\DAV\\Server", "type": "->", "args": [] }, { "file": "/remote.php", "line": 167, "args": [ "/apps/dav/appinfo/v2/remote.php" ], "function": "require_once" } ], "File": "/lib/private/Files/Cache/CacheQueryBuilder.php", "Line": 66, "CustomMessage": "--" }, "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Safari/537.36", "version": "20.0.4.0", "id": "5ff5c2b28a0e3" } ```
{
  "reqId": "0P3singiqHu88qAk7VRF",
  "level": 3,
  "time": "2021-01-06T15:01:05+01:00",,
  "user": "user",
  "app": "PHP",
  "method": "MOVE",
  "url": "/remote.php/dav/trashbin/user/trash/2.%20Product%20Datasheets.d1609877680",
  "message": {
    "Exception": "Error",
    "Message": "Trying to access array offset on value of type bool at /public/lib/private/Files/Cache/Cache.php#644",
    "Code": 0,
    "Trace": [
      {
        "file": "/public/lib/private/Files/Cache/Cache.php",
        "line": 644,
        "function": "onError",
        "class": "OC\\Log\\ErrorHandler",
        "type": "::",
        "args": [
          8,
          "Trying to access array offset on value of type bool",
          "/public/lib/private/Files/Cache/Cache.php",
          644,
          {
            "sourceCache": {
              "__class__": "OC\\Files\\Cache\\Cache"
            },
            "sourcePath": "__groupfolders/trash/3/2. Product Datasheets.d1609941656",
            "targetPath": "__groupfolders/3/2. Product Datasheets",
            "sourceData": false,
            "sourceId": null,
            "newParentId": 18468,
            "sourceStorageId": 1,
            "targetStorageId": 1
          }
        ]
      },
      {
        "file": "/public/apps/groupfolders/lib/Trash/TrashBackend.php",
        "line": 147,
        "function": "moveFromCache",
        "class": "OC\\Files\\Cache\\Cache",
        "type": "->",
        "args": [
          {
            "__class__": "OC\\Files\\Cache\\Cache"
          },
          "__groupfolders/trash/3/2. Product Datasheets.d1609941656",
          "__groupfolders/3/2. Product Datasheets"
        ]
      },
      {
        "file": "/public/apps/files_trashbin/lib/Trash/TrashManager.php",
        "line": 65,
        "function": "restoreItem",
        "class": "OCA\\GroupFolders\\Trash\\TrashBackend",
        "type": "->",
        "args": [
          {
            "__class__": "OCA\\GroupFolders\\Trash\\GroupTrashItem"
          }
        ]
      },
      {
        "file": "/public/apps/files_trashbin/lib/Sabre/AbstractTrash.php",
        "line": 94,
        "function": "restoreItem",
        "class": "OCA\\Files_Trashbin\\Trash\\TrashManager",
        "type": "->",
        "args": [
          {
            "__class__": "OCA\\GroupFolders\\Trash\\GroupTrashItem"
          }
        ]
      },
      {
        "file": "/public/apps/files_trashbin/lib/Sabre/RestoreFolder.php",
        "line": 76,
        "function": "restore",
        "class": "OCA\\Files_Trashbin\\Sabre\\AbstractTrash",
        "type": "->",
        "args": []
      },
      {
        "file": "/public/3rdparty/sabre/dav/lib/DAV/Tree.php",
        "line": 164,
        "function": "moveInto",
        "class": "OCA\\Files_Trashbin\\Sabre\\RestoreFolder",
        "type": "->",
        "args": [
          "2. Product Datasheets.d1609877680",
          "trashbin/user/trash/2. Product Datasheets.d1609877680",
          {
            "__class__": "OCA\\Files_Trashbin\\Sabre\\TrashFolder"
          }
        ]
      },
      {
        "file": "/public/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
        "line": 641,
        "function": "move",
        "class": "Sabre\\DAV\\Tree",
        "type": "->",
        "args": [
          "trashbin/user/trash/2. Product Datasheets.d1609877680",
          "trashbin/user/restore/2. Product Datasheets.d1609877680"
        ]
      },
      {
        "file": "/public/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
        "line": 89,
        "function": "httpMove",
        "class": "Sabre\\DAV\\CorePlugin",
        "type": "->",
        "args": [
          {
            "__class__": "Sabre\\HTTP\\Request"
          },
          {
            "__class__": "Sabre\\HTTP\\Response"
          }
        ]
      },
      {
        "file": "/public/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 474,
        "function": "emit",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          "method:MOVE",
          [
            {
              "__class__": "Sabre\\HTTP\\Request"
            },
            {
              "__class__": "Sabre\\HTTP\\Response"
            }
          ]
        ]
      },
      {
        "file": /3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 251,
        "function": "invokeMethod",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          {
            "__class__": "Sabre\\HTTP\\Request"
          },
          {
            "__class__": "Sabre\\HTTP\\Response"
          }
        ]
      },
      {
        "file": "/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 319,
        "function": "start",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/apps/dav/lib/Server.php",
        "line": 332,
        "function": "exec",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/apps/dav/appinfo/v2/remote.php",
        "line": 35,
        "function": "exec",
        "class": "OCA\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/remote.php",
        "line": 167,
        "args": [
          "/apps/dav/appinfo/v2/remote.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/lib/private/Log/ErrorHandler.php",
    "Line": 91,
    "CustomMessage": "--"
  },
  "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Safari/537.36",
  "version": "20.0.4.0",
  "id": "5ff5c2b28a197"
}
{
  "reqId": "0P3singiqHu88qAk7VRF",
  "level": 3,
  "time": "2021-01-06T15:01:05+01:00",
  "user": "user",
  "app": "PHP",
  "method": "MOVE",
  "url": "/remote.php/dav/trashbin/user/trash/2.%20Product%20Datasheets.d1609877680",
  "message": {
    "Exception": "Error",
    "Message": "Trying to access array offset on value of type bool at /lib/private/Files/Cache/Cache.php#630",
    "Code": 0,
    "Trace": [
      {
        "file": "/lib/private/Files/Cache/Cache.php",
        "line": 630,
        "function": "onError",
        "class": "OC\\Log\\ErrorHandler",
        "type": "::",
        "args": [
          8,
          "Trying to access array offset on value of type bool",
          "/lib/private/Files/Cache/Cache.php",
          630,
          {
            "sourceCache": {
              "__class__": "OC\\Files\\Cache\\Cache"
            },
            "sourcePath": "__groupfolders/trash/3/2. Product Datasheets.d1609941656",
            "targetPath": "__groupfolders/3/2. Product Datasheets",
            "sourceData": false
          }
        ]
      },
      {
        "file": "/apps/groupfolders/lib/Trash/TrashBackend.php",
        "line": 147,
        "function": "moveFromCache",
        "class": "OC\\Files\\Cache\\Cache",
        "type": "->",
        "args": [
          {
            "__class__": "OC\\Files\\Cache\\Cache"
          },
          "__groupfolders/trash/3/2. Product Datasheets.d1609941656",
          "__groupfolders/3/2. Product Datasheets"
        ]
      },
      {
        "file": "/apps/files_trashbin/lib/Trash/TrashManager.php",
        "line": 65,
        "function": "restoreItem",
        "class": "OCA\\GroupFolders\\Trash\\TrashBackend",
        "type": "->",
        "args": [
          {
            "__class__": "OCA\\GroupFolders\\Trash\\GroupTrashItem"
          }
        ]
      },
      {
        "file": "/apps/files_trashbin/lib/Sabre/AbstractTrash.php",
        "line": 94,
        "function": "restoreItem",
        "class": "OCA\\Files_Trashbin\\Trash\\TrashManager",
        "type": "->",
        "args": [
          {
            "__class__": "OCA\\GroupFolders\\Trash\\GroupTrashItem"
          }
        ]
      },
      {
        "file": "/apps/files_trashbin/lib/Sabre/RestoreFolder.php",
        "line": 76,
        "function": "restore",
        "class": "OCA\\Files_Trashbin\\Sabre\\AbstractTrash",
        "type": "->",
        "args": []
      },
      {
        "file": "/3rdparty/sabre/dav/lib/DAV/Tree.php",
        "line": 164,
        "function": "moveInto",
        "class": "OCA\\Files_Trashbin\\Sabre\\RestoreFolder",
        "type": "->",
        "args": [
          "2. Product Datasheets.d1609877680",
          "trashbin/user/trash/2. Product Datasheets.d1609877680",
          {
            "__class__": "OCA\\Files_Trashbin\\Sabre\\TrashFolder"
          }
        ]
      },
      {
        "file": "/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
        "line": 641,
        "function": "move",
        "class": "Sabre\\DAV\\Tree",
        "type": "->",
        "args": [
          "trashbin/user/trash/2. Product Datasheets.d1609877680",
          "trashbin/user/restore/2. Product Datasheets.d1609877680"
        ]
      },
      {
        "file": "/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
        "line": 89,
        "function": "httpMove",
        "class": "Sabre\\DAV\\CorePlugin",
        "type": "->",
        "args": [
          {
            "__class__": "Sabre\\HTTP\\Request"
          },
          {
            "__class__": "Sabre\\HTTP\\Response"
          }
        ]
      },
      {
        "file": "/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 474,
        "function": "emit",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          "method:MOVE",
          [
            {
              "__class__": "Sabre\\HTTP\\Request"
            },
            {
              "__class__": "Sabre\\HTTP\\Response"
            }
          ]
        ]
      },
      {
        "file": "/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 251,
        "function": "invokeMethod",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          {
            "__class__": "Sabre\\HTTP\\Request"
          },
          {
            "__class__": "Sabre\\HTTP\\Response"
          }
        ]
      },
      {
        "file": "/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 319,
        "function": "start",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/apps/dav/lib/Server.php",
        "line": 332,
        "function": "exec",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/apps/dav/appinfo/v2/remote.php",
        "line": 35,
        "function": "exec",
        "class": "OCA\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/remote.php",
        "line": 167,
        "args": [
          "/apps/dav/appinfo/v2/remote.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/lib/private/Log/ErrorHandler.php",
    "Line": 91,
    "CustomMessage": "--"
  },
  "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.101 Safari/537.36",
  "version": "20.0.4.0",
  "id": "5ff5c2b28a21e"
}
@solracsf solracsf added the bug label Jan 6, 2021
@solracsf solracsf changed the title [S3] Restore from trashbin triggers CacheQueryBuilder::whereFileId() Fatal error Restore from trashbin triggers CacheQueryBuilder::whereFileId() Fatal error Jan 6, 2021
@pierreozoux
Copy link
Member

@acsfer did you find a workaround? We just experienced the same :) the restore is a pain :)

@pierreozoux
Copy link
Member

@acsfer did you try without s3?

On one of our try, we could actually restore (there was the error message, but the restore did succeed).
But on antoher one (a real one :) ) it just didn't work..

@solracsf
Copy link
Member Author

I believe that this happens only with some implementations of S3 but not with AWS S3.

There is a recent issue about it nextcloud/server#25946

@pierreozoux pierreozoux added the s3 label Mar 10, 2021
@pierreozoux
Copy link
Member

The query to restore in the mean time:
(more a sql guy than a php one :/ )


COPY(
SELECT coalesce(fc5.fileid, fc4.fileid, fc3.fileid, fc2.fileid, fc1.fileid), concat(gft.original_location,'/',fc2.name,'/',fc3.name,'/',fc4.name,'/',fc5.name)
  FROM group_folders_trash gft 
  INNER JOIN filecache fc1 ON gft.file_id = fc1.fileid
  FULL OUTER JOIN filecache fc2 ON fc1.fileid = fc2.parent
  FULL OUTER JOIN filecache fc3 ON fc2.fileid = fc3.parent
  FULL OUTER JOIN filecache fc4 ON fc3.fileid = fc4.parent
  FULL OUTER JOIN filecache fc5 ON fc4.fileid = fc5.parent
  WHERE 
    deleted_time BETWEEN EXTRACT(EPOCH FROM TIMESTAMP WITH TIME ZONE '2021-03-08 18:00:00') and EXTRACT(EPOCH FROM TIMESTAMP WITH TIME ZONE '2021-03-08 19:00:00')

) to '/tmp/export.csv' DELIMITER ',' CSV HEADER;

cat /tmp/export.csv |  sed 's:/*$::' > /tmp/clean_export.csv

@pierreozoux
Copy link
Member

I believe that this happens only with some implementations of S3 but not with AWS S3.

There is a recent issue about it nextcloud/server#25946

I don't think it is related, but it is an interesting bug indeed that I'll follow closely.
fileid stays the same when you delete and when you restore the file, so it is not moved at all, just lignes in the db are changed. Hence I don't think it is related.

@pierreozoux pierreozoux added 1. to develop Issues that are ready for development feature: trashbin Items related to the trashbin feature labels Mar 11, 2021
@hrenard
Copy link

hrenard commented Mar 15, 2021

I took a look at the source code and did some testing and the issues seems to come from :

$targetFolder->getStorage()->moveFromStorage($trashStorage, $node->getInternalPath(), $targetLocation);
$targetFolder->getStorage()->getCache()->moveFromCache($trashStorage->getCache(), $node->getInternalPath(), $targetLocation);

My understanding is that the cache (aka the database) :

  • is not strictly consistent with the file system storage because it can be rebuilt from it
  • is strictly consistent with the s3 storage because it's only place where s3 ids are mapped to paths

The problem with s3 is that the cache has already been updated after the first operation and naturally the second operation can't find the old field id.

A quick fix would be to comment the second line when you're using s3 as primary storage.

To really fix this we could add some conditional logic, but it means keeping up to date a list of strictly consistent storage.
I personally think this logic should be abstracted from apps developers. But I don't fully comprehend the internals of Nextcloud, so I can't say if it is feasible.

@pierreozoux pierreozoux added feature: caching Items related to caching integration labels Mar 28, 2021
@pierreozoux
Copy link
Member

On my case, the restore actually works, but it prints an error to the user.
If the user has an accent in the username, it doesnt work at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. to develop Issues that are ready for development bug feature: caching Items related to caching feature: trashbin Items related to the trashbin feature integration s3
Projects
None yet
Development

No branches or pull requests

3 participants