-
Notifications
You must be signed in to change notification settings - Fork 87
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backends provided no user object / Duplicate entry for key 'gf_versions_uniq_index' #2559
Comments
Same here. |
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
To silence this error and get rid of the log spam, I commented out all of this in the function, because it will crash anyway. /**
* @param array{id: int, mount_point: string, groups: array<empty, empty>|array<array-key, int>, quota: int, size: int, acl: bool} $folder
*/
public function expireFolder(array $folder): void {
$view = new View('/__groupfolders/versions/' . $folder['id']);
$files = $this->versionsBackend->getAllVersionedFiles($folder);
$dummyUser = new User('', null, $this->dispatcher);
foreach ($files as $fileId => $file) {
if ($file instanceof FileInfo) {
//$versions = $this->versionsBackend->getVersionsForFile($dummyUser, $file);
//$expireVersions = $this->expireManager->getExpiredVersion($versions, $this->timeFactory->getTime(), false);
//foreach ($expireVersions as $version) {
// /** @var GroupVersion $version */
// $this->emit(self::class, 'deleteVersion', [$version]);
// $view->unlink('/' . $fileId . '/' . $version->getVersionFile()->getName());
//}
} else {
// source file no longer exists
$this->emit(self::class, 'deleteFile', [$fileId]);
$this->versionsBackend->deleteAllVersionsForFile($folder['id'], $fileId);
}
}
}
} |
You mean in the file |
That is an interesting question, as well as if there might be any interference with the files integrity check after updates so as the upcoming 27.1.4 in 2 or 3 days ... |
New release no luck. Still the same errors. (nextcloud 28 / groupfolders 16) |
There is a fix undergoing here : #2640 |
Any chance one of you could give it a try? |
@artonge Is this fix only for NC28? I can't update my instances currently because of missing app updates... |
This comment was marked as off-topic.
This comment was marked as off-topic.
I've found easier way: if you have to fix a lot of files in one folder, you may just rename that folder, expire and rename again. |
I don't feel that the help forum is appropriate at all, because this is not about users doing something wrong that can be corrected by other users, but this is a systemic issue with the Nextcloud project that plagues the groupfolders app more than others. The right place for discussing it seems to be #1215 |
I also experienced this error and after some searching I found a tricky workaround to get rid of the error (not recommended on production sites !). For me the problem/error starts when moving files from a groupfolder to a normal folder (with the standard move of copy dialog of the nextcloud functionality)
For me the errors are gone now, but only with risky manual deletions in the database and filesystem.
Running the occ command manually for checking if the error still exists: Some check queries if this is also the case in your nextcloud environment, if you get results from them then you got a problem :) //search for fileids in oc_files_versions and not existing in oc_file_cache // search fileids in oc_group_folder_versions and not existing in oc_file_cache // search for fileid's in non groupfolders but do exist in oc_group_folders_versions |
Ok this is happening on several of my instances. If i delete one file with the problem it moves on to the next. The files are unique in the database from what i can see. This prevents cron jobs from running properly. |
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '120589-1702287459' for key 'gf_versions_uniq_index' Stacktrace is similar to others in here. |
So is the error actually a unique constraint violation? Can anyone dig in and find if there is an actual duplicate key? |
An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '120589-1702287459' for key 'gf_versions_uniq_index' MariaDB [nextcloud]> select * from oc_group_folders_versions where file_id="120589" and timestamp="1702287459"; From the index it show that file_id and timestamp should be unique and i cant find a duplicate on that file it shows. |
And some of the files are even unlikely to have a version. Like recordings for example. |
How can the duplicity condition come true in the database? The 'gf_versions_uniq_index' constraint is there to avoid it... |
As I tried to explain in my earlier post the cron job script (call in the end VersionsBackend.php) is trying to insert a new row in oc_group_folders_versions, based in the file it finds on the filesystem. But the real question is why is the script VersionsBackend.php trying to do this insert again ? In my case (see my earlier comment) I moved a file from a groupfolder to a normal folder, and the versions in the database for this fileid where not removed from the dbtable oc_group_folders_versions and not remove from the groupfolder's filesystem. That causes inconsistent versions and the error. Hope this helps, I am not capable of making changes in the scipts, I am only the messenger... |
Well an expire is being run. So it should delete versions not add new ones. ill check the stacktrace again. |
Ok so lib/Versions/VersionsBackend.php public function function getVersionsForFile on line 62 runs on line 88 an insert for the current version. That one already has an entry and fails. |
{
"reqId":"cYDPBmuVlmSTbUScVQJA",
"level":3,
"time":"2024-01-15T10:55:02+00:00",
"remoteAddr":"",
"user":"--",
"app":"core",
"method":"",
"url":"--",
"message":"Error while running background job (class: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions, arguments: )",
"userAgent":"--",
"version":"28.0.1.1",
"exception":{
"Exception":"OC\\DB\\Exceptions\\DbalException",
"Message":"An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '117161-1701149347' for key 'gf_versions_uniq_index'",
"Code":1062,
"Trace":[
{
"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
"line":328,
"function":"wrap",
"class":"OC\\DB\\Exceptions\\DbalException",
"type":"::"
},
{
"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php",
"line":137,
"function":"executeStatement",
"class":"OC\\DB\\QueryBuilder\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend.php",
"line":88,
"function":"insert",
"class":"OCP\\AppFramework\\Db\\QBMapper",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":72,
"function":"getVersionsForFile",
"class":"OCA\\GroupFolders\\Versions\\VersionsBackend",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":59,
"function":"expireFolder",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/BackgroundJob/ExpireGroupVersions.php",
"line":43,
"function":"expireAll",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/Job.php",
"line":81,
"function":"run",
"class":"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":102,
"function":"start",
"class":"OCP\\BackgroundJob\\Job",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":92,
"function":"start",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
},
{
"file":"/var/www/nextcloud/cron.php",
"line":152,
"function":"execute",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
}
],
"File":"/var/www/nextcloud/lib/private/DB/Exceptions/DbalException.php",
"Line":71,
"Previous":{
"Exception":"Doctrine\\DBAL\\Exception\\UniqueConstraintViolationException",
"Message":"An exception occurred while executing a query: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '117161-1701149347' for key 'gf_versions_uniq_index'",
"Code":1062,
"Trace":[
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php",
"line":1938,
"function":"convert",
"class":"Doctrine\\DBAL\\Driver\\API\\MySQL\\ExceptionConverter",
"type":"->"
},
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php",
"line":1880,
"function":"handleDriverException",
"class":"Doctrine\\DBAL\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php",
"line":1208,
"function":"convertExceptionDuringQuery",
"class":"Doctrine\\DBAL\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/Connection.php",
"line":294,
"function":"executeStatement",
"class":"Doctrine\\DBAL\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php",
"line":386,
"function":"executeStatement",
"class":"OC\\DB\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
"line":280,
"function":"execute",
"class":"Doctrine\\DBAL\\Query\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
"line":326,
"function":"execute",
"class":"OC\\DB\\QueryBuilder\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php",
"line":137,
"function":"executeStatement",
"class":"OC\\DB\\QueryBuilder\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend.php",
"line":88,
"function":"insert",
"class":"OCP\\AppFramework\\Db\\QBMapper",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":72,
"function":"getVersionsForFile",
"class":"OCA\\GroupFolders\\Versions\\VersionsBackend",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":59,
"function":"expireFolder",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/BackgroundJob/ExpireGroupVersions.php",
"line":43,
"function":"expireAll",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/Job.php",
"line":81,
"function":"run",
"class":"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":102,
"function":"start",
"class":"OCP\\BackgroundJob\\Job",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":92,
"function":"start",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
},
{
"file":"/var/www/nextcloud/cron.php",
"line":152,
"function":"execute",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
}
],
"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/API/MySQL/ExceptionConverter.php",
"Line":62,
"Previous":{
"Exception":"Doctrine\\DBAL\\Driver\\PDO\\Exception",
"Message":"SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '117161-1701149347' for key 'gf_versions_uniq_index'",
"Code":1062,
"Trace":[
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php",
"line":132,
"function":"new",
"class":"Doctrine\\DBAL\\Driver\\PDO\\Exception",
"type":"::"
},
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php",
"line":1202,
"function":"execute",
"class":"Doctrine\\DBAL\\Driver\\PDO\\Statement",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/Connection.php",
"line":294,
"function":"executeStatement",
"class":"Doctrine\\DBAL\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php",
"line":386,
"function":"executeStatement",
"class":"OC\\DB\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
"line":280,
"function":"execute",
"class":"Doctrine\\DBAL\\Query\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
"line":326,
"function":"execute",
"class":"OC\\DB\\QueryBuilder\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php",
"line":137,
"function":"executeStatement",
"class":"OC\\DB\\QueryBuilder\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend.php",
"line":88,
"function":"insert",
"class":"OCP\\AppFramework\\Db\\QBMapper",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":72,
"function":"getVersionsForFile",
"class":"OCA\\GroupFolders\\Versions\\VersionsBackend",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":59,
"function":"expireFolder",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/BackgroundJob/ExpireGroupVersions.php",
"line":43,
"function":"expireAll",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/Job.php",
"line":81,
"function":"run",
"class":"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":102,
"function":"start",
"class":"OCP\\BackgroundJob\\Job",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":92,
"function":"start",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
},
{
"file":"/var/www/nextcloud/cron.php",
"line":152,
"function":"execute",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
}
],
"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Exception.php",
"Line":28,
"Previous":{
"Exception":"PDOException",
"Message":"SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '117161-1701149347' for key 'gf_versions_uniq_index'",
"Code":"23000",
"Trace":[
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php",
"line":130,
"function":"execute",
"class":"PDOStatement",
"type":"->"
},
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Connection.php",
"line":1202,
"function":"execute",
"class":"Doctrine\\DBAL\\Driver\\PDO\\Statement",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/Connection.php",
"line":294,
"function":"executeStatement",
"class":"Doctrine\\DBAL\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Query/QueryBuilder.php",
"line":386,
"function":"executeStatement",
"class":"OC\\DB\\Connection",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
"line":280,
"function":"execute",
"class":"Doctrine\\DBAL\\Query\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/private/DB/QueryBuilder/QueryBuilder.php",
"line":326,
"function":"execute",
"class":"OC\\DB\\QueryBuilder\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/AppFramework/Db/QBMapper.php",
"line":137,
"function":"executeStatement",
"class":"OC\\DB\\QueryBuilder\\QueryBuilder",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/VersionsBackend.php",
"line":88,
"function":"insert",
"class":"OCP\\AppFramework\\Db\\QBMapper",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":72,
"function":"getVersionsForFile",
"class":"OCA\\GroupFolders\\Versions\\VersionsBackend",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/Versions/GroupVersionsExpireManager.php",
"line":59,
"function":"expireFolder",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/apps/groupfolders/lib/BackgroundJob/ExpireGroupVersions.php",
"line":43,
"function":"expireAll",
"class":"OCA\\GroupFolders\\Versions\\GroupVersionsExpireManager",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/Job.php",
"line":81,
"function":"run",
"class":"OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":102,
"function":"start",
"class":"OCP\\BackgroundJob\\Job",
"type":"->"
},
{
"file":"/var/www/nextcloud/lib/public/BackgroundJob/TimedJob.php",
"line":92,
"function":"start",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
},
{
"file":"/var/www/nextcloud/cron.php",
"line":152,
"function":"execute",
"class":"OCP\\BackgroundJob\\TimedJob",
"type":"->"
}
],
"File":"/var/www/nextcloud/3rdparty/doctrine/dbal/src/Driver/PDO/Statement.php",
"Line":130
}
}
},
"message":"Error while running background job (class: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions, arguments: )",
"exception":{
},
"CustomMessage":"Error while running background job (class: OCA\\GroupFolders\\BackgroundJob\\ExpireGroupVersions, arguments: )"
}
} |
I think the fix for nextcloud/server#42805 applies here too. The read / write split can cause this issue. We could simply catch the unique constrain violation since the entry that is needed is there. |
Hey, could one of you test this PR: #2758 ? |
very nice! thank you!! |
There should already be releases including this patch. Other than that it's just basic usage of the patch command in linux or manually doing the changes made in the PR to the coresponding files. |
I have the latest version, the patch does not need to be executed. just run the command "./occ groupfolders:expire" ? |
The expiry should automatically run within the planned tasks executed by cron/webcron/ajax |
Unfortunately it looks like there is still a similar error but with Deleted files:
|
This looks like a different issue, can you open a new one? |
This is basically the old issue I've reported earlier #2466. |
Steps to reproduce
Error appears in the logs with cron jobs, or when using
docker run (...) occ groupfolders:expire -v
=> That was the only step to reproduce the "backends provided no user object" error (prior to update 15.3.2 - 16.0.1).
Regarding "duplicate entry for key", see 4 comments starting from #2559 (comment)
Actual behaviour
Server configuration
Operating system:
Debian 12 / Docker 24
Web server:
Nginx reverse proxy
Nextcloud version:
27.1
Group folders version:
15.3
Updated from an older Nextcloud/ownCloud or fresh install:
Updated from 27.0
Where did you install Nextcloud from:
Nextcloud AIO
Are you using external storage, if yes which one: local/s3/smb/sftp/...
No
Are you using encryption: yes/no
No
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/Saml/...
No
Logs (docker run)
Logs (cron errors in nextcloud admin panel)
The bug
It seems the error is due to this line where
getVersionsForFile
is called with a dummy user, which isn't implemented (yet ?) sincegetVersionsForFile
is clearly requiring a real user, because it will call$user->getUID()
which will return NULL and then cause the exception withOC\Files\Node\Root->getUserFolder("")
I couldn't find existing issues, but may be this is something already fixed in master / targeting nextcloud 28 ?
Will it disappear when AIO switch to NC 28 ?
The text was updated successfully, but these errors were encountered: