Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use correct path offset for sync_list exclusion matching #1269

Merged
merged 2 commits into from
Feb 13, 2021

Conversation

abraunegg
Copy link
Owner

  • When processing sync_list entries, by default, if the path starts with '/', and offset of 1 is used to ensure correct path matching can be done. However, if the path starts with '!/' or '-/' to exclude a certain path, an offset of 1 was still being used, thus, exclusion matching was not occurring correctly. Use the correct offset, based on the 'sync_list' entry.

* When processing sync_list entries, by default, if the path starts with '/', and offset of 1 is used to ensure correct path matching can be done. However, if the path starts with '!/' or '-/' to exclude a certain path, an offset of 1 was still being used, thus, exclusion matching was not occurring correctly. Use the correct offset, based on the 'sync_list' entry.
@abraunegg abraunegg added this to the v2.4.10 milestone Feb 12, 2021
@abraunegg
Copy link
Owner Author

abraunegg commented Feb 12, 2021

sync_list configuration:

#Exclude
/random_files/*
!/random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO
!/random_images
!/random_videos

Directory Tree:

.
├── Documents
│   ├── Images
│   └── Music
├── Downloads
│   ├── dL8OeDylk9HczSbecZrrNSyAHOaVtfib
│   └── Ep6B8ytdiA9t2op0DOCbhSE0eUO30HAO
├── Dropbox
├── file1.tmp
├── Images
│   └── image003.jpg
├── Music
│   ├── audio003.mp3
│   └── Downloaded\ Music
├── Public
│   ├── KRghvK9kq2trFTRQlnLGSJGvE7OFoAcl
│   └── RVGpx7p1X8s22CAqz9hiTH2fuporDf1T
├── random_files -> /home/alex/OneDrive/random_files
│   ├── 7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO
│   └── iWcWDyTgkANqqkquGVUCMGqqX67gPqR8
├── random_images -> /home/alex/OneDrive/random_images
│   ├── Q4NvqmP1DdO2syUQCOxWoC4CAGx27LH3
│   └── rxBrpjYZJOHoe5C87yCYUq4QekeOaOmE
├── random_videos -> /home/alex/OneDrive/random_videos
└── Remove\ Ubuntu\ Package.docx

20 directories, 4 files

Expectation is that the last 3 entries are excluded

Without PR:

./onedrive --confdir '~/.config/onedrive-personal/' --synchronize --verbose --resync
Using 'user' Config Dir: /home/alex/.config/onedrive-personal/
Using 'system' Config Dir: 
Configuration file successfully loaded
Deleting the saved status ...
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Opening the item database ...
All operations will be performed in: /home/alex/OneDrivePersonal
Application version: v2.4.9-19-g3ba8bb1
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5167850165
Fetching details for OneDrive Root
OneDrive Root does not exist in the database. We need to add it.
Added OneDrive Root to the local database
Initializing the Synchronization Engine ...
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Updated Remaining Free Space: 5167850165
Processing 96 OneDrive items to ensure consistent local state due to sync_list being used
Skipping item - excluded by sync_list config: random_videos
Skipping item - excluded by sync_list config: Dropbox
Skipping item - excluded by sync_list config: Downloads
Skipping item - excluded by sync_list config: Public
Skipping item - excluded by sync_list config: Images
Skipping item - excluded by sync_list config: Music
Skipping item - excluded by sync_list config: Documents
Skipping item - excluded by sync_list config: random_images
Number of items from OneDrive to process: 1
Uploading differences of /home/alex/OneDrivePersonal
Processing .
The directory has not changed
Processing Remove Ubuntu Package.docx
Processing random_files
The directory has not changed
Processing random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO
The directory has not changed
Processing random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO/file3.data
The file has not changed
Processing random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO/file0.data
The file has not changed
Processing random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO/file1.data
The file has not changed
Processing random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO/file2.data
The file has not changed
Processing random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO/file4.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8
The directory has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file0.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file1.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file2.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file3.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file4.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file5.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file6.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file7.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file8.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file9.data
The file has not changed
Uploading new items of /home/alex/OneDrivePersonal
Skipping item - excluded by sync_list config: ./random_images
Skipping item - excluded by sync_list config: ./random_videos
Skipping item - excluded by sync_list config: ./Dropbox
Skipping item - excluded by sync_list config: ./Documents
Skipping item - excluded by sync_list config: ./Images
Skipping item - excluded by sync_list config: ./Music
Skipping item - excluded by skip_file config: ./file1.tmp
Skipping item - excluded by sync_list config: ./Downloads
Skipping item - excluded by sync_list config: ./Public
Applying changes of Path ID: 66D53BE8A5056ECA!101
Updated Remaining Free Space: 5167850165
Processing 1 OneDrive items to ensure consistent local state due to a full scan being requested

The directory Processing random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO is incorrectly processed - should be excluded.

With PR:

./onedrive --confdir '~/.config/onedrive-personal/' --synchronize --verbose --resync
Using 'user' Config Dir: /home/alex/.config/onedrive-personal/
Using 'system' Config Dir: 
Configuration file successfully loaded
Deleting the saved status ...
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Opening the item database ...
All operations will be performed in: /home/alex/OneDrivePersonal
Application version: v2.4.9-18-gf37fdda
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 5167850165
Fetching details for OneDrive Root
OneDrive Root does not exist in the database. We need to add it.
Added OneDrive Root to the local database
Initializing the Synchronization Engine ...
Syncing changes from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Updated Remaining Free Space: 5167850165
Processing 96 OneDrive items to ensure consistent local state due to sync_list being used
Skipping item - excluded by sync_list config: random_videos
Skipping item - excluded by sync_list config: Dropbox
Skipping item - excluded by sync_list config: Downloads
Skipping item - excluded by sync_list config: Public
Skipping item - excluded by sync_list config: Images
Skipping item - excluded by sync_list config: Music
Skipping item - excluded by sync_list config: Documents
Skipping item - excluded by sync_list config: random_images
Skipping item - excluded by sync_list config: random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO
Number of items from OneDrive to process: 1
Uploading differences of ~/OneDrivePersonal
Processing .
The directory has not changed
Processing Remove Ubuntu Package.docx
Processing random_files
The directory has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8
The directory has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file0.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file1.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file2.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file3.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file4.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file5.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file6.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file7.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file8.data
The file has not changed
Processing random_files/iWcWDyTgkANqqkquGVUCMGqqX67gPqR8/file9.data
The file has not changed
Uploading new items of ~/OneDrivePersonal
Skipping item - excluded by sync_list config: ./random_files/7GLHbbPdz9UzBLQJxKHefZdyMSJmv5sO
Skipping item - excluded by sync_list config: ./random_images
Skipping item - excluded by sync_list config: ./random_videos
Skipping item - excluded by sync_list config: ./Dropbox
Skipping item - excluded by sync_list config: ./Documents
Skipping item - excluded by sync_list config: ./Images
Skipping item - excluded by sync_list config: ./Music
Skipping item - excluded by skip_file config: ./file1.tmp
Skipping item - excluded by sync_list config: ./Downloads
Skipping item - excluded by sync_list config: ./Public
Applying changes of Path ID: 66D53BE8A5056ECA!101
Updated Remaining Free Space: 5167850165
Processing 1 OneDrive items to ensure consistent local state due to a full scan being requested

The directory is now correctly excluded.

@abraunegg abraunegg merged commit cd7b7b0 into master Feb 13, 2021
@abraunegg abraunegg deleted the fix-synclist-offset-handling branch February 13, 2021 19:43
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked and limited conversation to collaborators May 27, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants