-
-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
S3 storage has folder size "pending" #7420
Comments
cc @icewind1991 |
S3 should be a lot smoother with 13. Could you test it again? |
After upgrading from 12.0.4 to 13beta3 the problem persisted. A installation with the web installer (12.0.3) fixes the problem, but when I upgrade to 12.0.4 I'm back to my old problem. Note, that I'm using the same database for all versions (not at the same time ofc). I don't have any issues with 13beta3 and a new database. Is there something wrong with the updater? Or did something get lost after all those small updates I did in the past? |
I am running into the same problem, running 13 b4 Error
|
After trying some stuff out, I noticed that when the root of the S3 bucket only contains folders, then the "Contents" attribute in the result paginator is not present. It should therefore be checked whether the "Contents" attribute is an array before iterating it with foreach similar to the CommonPrefixes. @TecJon @lesaff For a quick workaround, you can add a dummy file |
Sorry, I just noticed it still does not always show the correct size (it did at least once), but at least the logs don't overflow with Edit: it sometimes shows the correct size on the shared folder, but for folders inside it is still pending. |
cc @icewind1991 |
@MickVanDuijn thank you, unfortunately it didn't change for me. This issue might only be a side issue to a bigger problem with s3 that I am facing. #8299 |
Issue is still persist with nextCloud 13.0.1. |
Same problem here: NC 13.0.2, external storage is DigitalOcean's Spaces. All external folders showing "Pending" and log is flooded with error:
|
Same issue on 13.0.4 using snap installation on Ubuntu server. |
Same for 14.0.0.13 for an empty folder |
It seems with a GCP bucket with around 14000 objects, Nextcloud is stuck and can't access the files. Maybe a UTF8 problem? |
I applied the following patch to fix a bug, where root-level folders have not been identified in case they have been directly uploaded to S3 storage instead of upload via Nextcloud. This also fixed the "pending" size issue in Nextcloud 15.0.5. https://github.com/nextcloud/server/compare/bugfix/6954/scan-external-s3 |
cc @kesselb |
Thanks for your feedback @mfridge There are a few positive reports for this patch. I'll try to do some more testing and get it into 16. |
Is there some PR for that already, because the beta is planned for tomorrow ;) |
Hi, I just want to report the same issue of pending size for S3 external storage. |
How can I apply this patch? |
@mfridge provided the link. |
I am using Nextcloud on Hostio.cloud and added S3 as external storage and still see the Pending on the folders |
When adding S3 via external mapping, the pending state is constant with our version of NC 20 as well. |
I'm also affected by this annoying bug. |
along with folder size, even it doesn't show correct modified time also ,please suggest |
there were some issues with background scanning with S3 external storage recently. please try again with recent versions |
The same issue occurred at my end. And this was solved by setting up the Cron option from the Basic Settings section. The following are the steps:
If asked, press “1” to use the nano editor (which is super easy to use) and add the following line to your crontab file:
You have to replace the path /var/www/nextcloud/cron.php with the path to your current Nextcloud installation. You can verify if the cron job has been added and scheduled by executing:
Which returns:
This Cron method enables the execution of scheduled jobs without the inherent limitations the Web server might have. [https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/background_jobs_configuration.html] |
Steps to reproduce
Expected behaviour
The correct folder size should be displayed.
Actual behaviour
Folder size says "pending" on all levels in the file structure.
Server configuration
Operating system:
Ubuntu 16.04.3 64bit
Web server:
Apache/2.4.18
nginx/1.11.10
Database:
mysql Ver 14.14 Distrib 5.7.20
PHP version:
PHP-FPM 5.6.32
Nextcloud version: (see Nextcloud admin page)
12.0.4
Updated from an older Nextcloud/ownCloud or fresh install:
Updated to the newest release since ownCloud 9
Where did you install Nextcloud from:
php installation script
Signing status:
Signing status
List of activated apps:
App list
Nextcloud configuration:
Config report
Are you using external storage, if yes which one: local/smb/sftp/...
S3
Are you using encryption: yes/no
no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no
Client configuration
Browser:
any
Operating system:
any
Logs
Web server error log
Web server error log
Nextcloud log (data/nextcloud.log)
Nextcloud log
The text was updated successfully, but these errors were encountered: