Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting to also upload existing files #4788

Merged
merged 11 commits into from
Feb 18, 2020
Merged

Conversation

AndyScherzinger
Copy link
Member

@AndyScherzinger AndyScherzinger commented Nov 3, 2019

successor for #2873


Originally created by @koying

Please be kind, first time I put my nose into the code.

This adds a setting to the instant uploads to also upload files already present on the device at the time instant upload is setup.
Ref: https://help.nextcloud.com/t/why-doesnt-android-client-upload-existing-pictures/10365

95% of the PR is for handling of the setting itself.
It might be worth wondering whether this deserves a setting at all, really, ie whether not uploading existing files is worthwhile, as it's a bit counter-intuitive, imo

@AndyScherzinger AndyScherzinger added 2. developing needs info Waiting for info from user(s). Issues with this label will auto-stale. feature: auto upload labels Nov 3, 2019
@ashpieboop
Copy link
Contributor

In the original PR it was suggested that this should not be a setting but rather a one time action with a prompt when exiting the settings view.

However, this would mean that when enabling/disabling the synced folder through the button from the synced folder list (and not the full parameters view) would skip files created while disabled, and not resync them when reenabled.

Maybe this setting should be on by default ? This might also mean needing to warn the user that files with the same name at destination will be overridden while there is not conflict detection/management.

@AndyScherzinger
Copy link
Member Author

However, this would mean that when enabling/disabling the synced folder through the button from the synced folder list (and not the full parameters view) would skip files created while disabled, and not resync them when reenabled.

Maybe this setting should be on by default ? This might also mean needing to warn the user that files with the same name at destination will be overridden while there is not conflict detection/management.

Fully agree and so would probably @jancborchardt and @nextcloud/designers and @jospoortvliet. The user shouldn't have to think about it. It even say when a db entry is initially created the user could be asked if they want to upload existing files. So I'd rather see it an an initial opt-out thing. If initially the user decided to not upload existing files then they are left out, else the users simply want to always have them uploaded (which is likely the default: Backup anything and don't bother me).

@tobiasKaminsky
Copy link
Member

When enabling (don't matter how), this then takes all images from existing folder and uploads it?
How do we prevent cases where files already exists?
(sorry if this is again the same discussion).

We could do it this way:

  • show an info that this might lead to duplicated data
  • upload and do not overwrite any file with the same name
  • let user remove the possible duplicated files

@szaimen
Copy link
Contributor

szaimen commented Nov 7, 2019

@tobiasKaminsky In my opinion

upload and do not overwrite any file with the same name

Is the best option in combination with a warning that already existing files with the same name don't get overwritten. Then a user has the choice to choose a different upload folder (or delete files in this folder before uploading) if he/her wants to upload everything (and he/her isn't sure about whether there are already files with the same name in this upload folder). Best thing is, that this initial upload then can be triggered more then one time and only uploads files that are (based on the name) not already uploaded into this upload folder.

@dgtlmoon
Copy link

FWIW #2873 (comment) the APK works well for me

@cpw
Copy link

cpw commented Nov 12, 2019

Sadly, #4788 (comment) doesn't seem to work. I look forward to this feature.

@dgtlmoon
Copy link

dgtlmoon commented Nov 12, 2019

@cpw Sadly, please be more helpful/mindful, "doesnt seem to work" does really not help anyone to improve the issue, how exactly? please fill in a complete error report, device type, what you tried, messages you saw (paste the text as it is exactly), version number of client, android and nextcloud server, connection type etc

@cpw
Copy link

cpw commented Nov 12, 2019

Nevermind. I reinstalled this morning, and it seems to be trying to sync existing files this time. 🤷‍♀️

@ashpieboop
Copy link
Contributor

ashpieboop commented Nov 14, 2019

@Excel1 [edit: you referenced a comment from another PR while quoting the newest one]

Also keep in mind that this PR modifies the database which means in order to test in realistic conditions, you should perform a full manual uninstall of the previous test versions before installing this new APK.

Also this PR is still currently in discussion and development.

@Excel1
Copy link

Excel1 commented Nov 14, 2019

@ArisuOngaku thats absolutely right. I only wanna give feedback :) #4788 (comment) i recently tested this apk. Same issues: (Note 8 - Android 9) After clicking the cloud icon below the tab autoupload the app restart.

@ashpieboop
Copy link
Contributor

ashpieboop commented Nov 14, 2019

@Excel1 Thanks for that! I can't reproduce this crash. Would you mind sharing your logs?

(settings -> logs -> 3 dots up right corner -> send by email, and you can actually upload this as a file if you select the nextcloud app itself)

@Excel1
Copy link

Excel1 commented Nov 14, 2019

@ArisuOngaku sure. here it is. (i censored sensitive data)
logs.txt

@dgtlmoon
Copy link

@Excel1 did you censor this line? the URL looks really broken to me Creating SSL Socket with remote to.nextcloud.url:443

@ashpieboop
Copy link
Contributor

ashpieboop commented Nov 14, 2019

@Excel1 Thanks! I see no stack trace though if I'm not mistaken, this is the part where you experience the crash:

2019-11-14T10:44:32.137+0100;V;SyncedFolderProvider;Inserting /storage/0000-0000/DCIM/Camera with enabled=true
2019-11-14T10:44:33.397+0100;D;Debug;start logging

My best guess is that the local database was not wiped, did you fully uninstall the Nextcloud QA app before installing the new APK? If yes, then try that again but clean the app's cache before.

What indicates that the database was correctly wiped is that you need to login again when you start the app.

@Excel1
Copy link

Excel1 commented Nov 14, 2019

@ArisuOngaku Yes i did. And now i did it again (and deleted cache and app data). Should i also uninstall the original Nextcloud App ?
logs.txt

@ashpieboop
Copy link
Contributor

@Excel1 No, I see no reason the original Nextcloud app would interfere.
Though your logs are going back to 2019-10-13 which means it wasn't wiped. I'm sorry but we have to be sure: did you have to log back in when starting the app? If no, then the actual app cache is not getting properly wiped.

When I uninstall (manually) then install again the same APK (on emulator), my cache and logs get deleted. By any chance, are you correctly opening the Nextcloud QA app (not to be confused with Nextcloud)?

@Excel1
Copy link

Excel1 commented Nov 14, 2019

@ArisuOngaku Okay. I definitely uninstalled the right version (QA) and deleted the right cache (QA). Here are the log (definitely from the QA version) 🤔

logs.txt

@ashpieboop
Copy link
Contributor

@Excel1 Thanks a lot for your patience. I looked deeper in your logs and it looks like your cache actually gets properly deleted.

Are you familiar with android debug tools (adb) and the logcat utility? Recording the logs from there while reproducing the crash would give us a more precise and definitive answer to what's happening.

@Excel1
Copy link

Excel1 commented Nov 14, 2019

@ArisuOngaku No problem, i have to say thank you - for helping me.
I will try it and inform you about it as soon as I have done.

@ashpieboop
Copy link
Contributor

@AndyScherzinger I rebased this PR on the latest master to fix conflicts (and also removed the typo commit and integrated its changes in the first one by koying), am I allowed to force push and work on this PR?
(what I want to push master...ArisuOngaku:instantupload_all)

@AndyScherzinger
Copy link
Member Author

am I allowed to force push and work on this PR?

@ArisuOngaku absolutely yes! ❤️ 👍 Only when you label it 2. to review it shouldn't be rebased again since other might have checked out the branch for QA.

@ashpieboop ashpieboop force-pushed the instantupload_all branch 2 times, most recently from 44f75ec to 0b1c6e1 Compare November 19, 2019 16:48
@nextcloud nextcloud deleted a comment Nov 19, 2019
@nextcloud-android-bot
Copy link
Collaborator

Codacy

360

Lint

TypemasterPR
Warnings7777
Errors00

SpotBugs (new)

Warning TypeNumber
Bad practice Warnings26
Correctness Warnings69
Internationalization Warnings13
Malicious code vulnerability Warnings5
Multithreaded correctness Warnings9
Performance Warnings77
Security Warnings45
Dodgy code Warnings139
Total383

SpotBugs (master)

Warning TypeNumber
Bad practice Warnings26
Correctness Warnings71
Internationalization Warnings13
Malicious code vulnerability Warnings5
Multithreaded correctness Warnings9
Performance Warnings78
Security Warnings45
Dodgy code Warnings138
Total385

@nextcloud-android-bot
Copy link
Collaborator

APK file: https://www.kaminsky.me/nc-dev/android-artifacts/12728.apk

qrcode

To test this change/fix you can simply download above APK file and install and test it in parallel to your existing Nextcloud app.

@nextcloud-android-bot
Copy link
Collaborator

Codacy

360

Lint

TypemasterPR
Warnings7777
Errors00

SpotBugs (new)

Warning TypeNumber
Bad practice Warnings26
Correctness Warnings69
Internationalization Warnings13
Malicious code vulnerability Warnings5
Multithreaded correctness Warnings9
Performance Warnings77
Security Warnings45
Dodgy code Warnings139
Total383

SpotBugs (master)

Warning TypeNumber
Bad practice Warnings26
Correctness Warnings71
Internationalization Warnings13
Malicious code vulnerability Warnings5
Multithreaded correctness Warnings9
Performance Warnings78
Security Warnings45
Dodgy code Warnings138
Total385

@nextcloud-android-bot
Copy link
Collaborator

Screenshot test failed, but no output was generated. Maybe a preliminary stage failed.

@nextcloud-android-bot
Copy link
Collaborator

@ashpieboop
Copy link
Contributor

You're welcome! :)

@tobiasKaminsky tobiasKaminsky merged commit 5d428fe into master Feb 18, 2020
@delete-merged-branch delete-merged-branch bot deleted the instantupload_all branch February 18, 2020 07:45
@koying
Copy link
Contributor

koying commented Feb 18, 2020

@ArisuOngaku Congrats for pushing this to the end :)

@AndyScherzinger AndyScherzinger added this to the Nextcloud App 3.11.0 milestone Feb 18, 2020
tobiasKaminsky added a commit that referenced this pull request Feb 19, 2020
3f62afa Merge remote-tracking branch 'origin/master' into dev
5d428fe Merge pull request #4788 from nextcloud/instantupload_all
49f3a67 ConflictsResolveActivity: code style and readability improvements
6540e3e Upload list conflicts: replace trash icon with menu and add resolve action
a4ff1f1 Customize upload sync conflict notification strings
e77ffa0 Fix synced folder layout form control widgets alignment
91ef307 Fix database migration of field forceOverwrite to NameCollisionPolicy
52d089b FileUploader: fix codacy issues and SpotBugs
ec27f84 FileUploader: code cleanup
7713a28 FileUploader: require explicit NameCollisionPolicy and change default used value
6783e8a Make file uploads ask the user what to do when the file already exists on remote
213002f Make newly created synced folders auto upload existing files by default
1b7bd72 ADD: [instantupload] setting to also upload existing files
c243453 Merge pull request #5465 from nextcloud/fix/create-folder-crash-autosync
d178cde daily dev 20200218
@bastian-src
Copy link

I am using version 3.14 of the App and wonder why this feature is not included although it was already part of the 3.11 milestone?

@AndyScherzinger
Copy link
Member Author

@bastian-src just checked and the feature is there! Depending on your screen resolution you might have to scroll down in the auto upload config dialog (item is at the bottom) how to handle existing files in such a case and the 3rd checkbox from the top should say (upload existing).

So can you post a screenshot of the auto upload config dialog of your device?

@bastian-src
Copy link

bastian-src commented Nov 24, 2020

@AndyScherzinger sorry, my fault. The option is in the menu but I thought there would be another option/action necessary in order to trigger the upload of existing files. So, I think the appropriated behaviour would be that it uploads the existing files right after clicking on Save?
In my case, another dialogue pops up which gives me a warning about the power saving (which I excluded the app from) and afterwards nothing happens. The manual file upload works fine for me.

@AndyScherzinger
Copy link
Member Author

Hmmm, the upload should happen within 15 minutes

@bastian-src
Copy link

After 1h, the upload suddenly started. Thanks for your immediate response!

@AndyScherzinger
Copy link
Member Author

Sweet and glad to hear :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.