Commit graph

490 commits

Author SHA1 Message Date
abraunegg dcc0322cac
Update how './' is stripped from a path before check if excluded (Issue #901) (#902)
* Update how './' is stripped from a path before check if excluded
2020-05-02 04:46:44 +10:00
abraunegg bc9f2cf893
Update handling of fullScanRequired and syncListConfiguredFullScanOverride (Issue #896) (#899)
* Change 'syncListConfigured' to 'syncListConfiguredFullScanOverride' as this is what this variable
* Update handling of fullScanRequired and syncListConfiguredFullScanOverride
* When a 429 or 504 is generated when querying for 'changes' inside a changeset bundle, dont restart scanning changes from the beginning, retry original request after a delay
2020-05-02 04:05:06 +10:00
abraunegg a61d291340
Update #658 & #865 handling as when to setOneDriveFullScanTrigger (Issue #896) (#898)
* Change output for debugging
* Update setOneDriveFullScanTrigger to only be used when sync_list or skip_dir is used
* Unset oneDriveFullScanTrigger when it is currently set and no longer needed
2020-04-28 10:56:44 +10:00
abraunegg 73f71f6d52
Fix assertion when attempting to compute complex path (Issue #890) (#891)
* Before attempting to compute the complex path, check if the parent details are in the database
* Handle --dry-run and --no-remote-delete for shared folders
2020-04-27 20:59:35 +10:00
abraunegg c05243444f
Update progress bar to be more accurate when downloading large files (#888)
* Change from round to floor, so % bar increases when downloaded data is at X% not potentially under, thus leading to under reporting
* Add debug output when each % increase when downloading a file to assist with validating progress
* Start displaying ETA starting at 5% rather than 10%
2020-04-26 06:52:29 +10:00
abraunegg 628c58ad58
Catch unhandled curl exception on operationTimeout being exceeded (#886)
* Catch unhandled curl exception on operationTimeout being exceeded
2020-04-23 10:25:41 +10:00
abraunegg ccbeae4942
Implement resync in config option (Issue #874) (#875)
* Implement the ability to set --resync as a config option, default is false
2020-04-20 13:03:36 +10:00
abraunegg 2a7e48b823
Differenciate between OneNote objects and other unsupported objects (#870)
* Differenciate between OneNote objects and other unsupported objects
2020-04-15 14:54:21 +10:00
abraunegg d3aa0032bc
Perform full scan of OneDrive when required to ensure consistent local state (Issue #865) (#867)
* Use the configured 'fullScanRequired' every 10th loop to ensure that the local repository actually is in sync with OneDrive when sync_list is not used
* Use 'oneDriveFullScanTrigger' if it is set to trigger a full scan earlier than wait for next full sync if in monitor mode
2020-04-15 14:29:50 +10:00
abraunegg 7ca1ff95a8
Update handling of folders that cannot be read if parent should be skipped (#866)
* If a directory cannot be accessed, and is in a folder that should be skipped, do not print warning
2020-04-15 14:07:58 +10:00
abraunegg 7a18393ed0
Update skip_dotfiles handling of .folders when in monitor mode (#848)
* Prevent monitoring files inside .folders when using skip_dotfiles in monitor mode
* Handle '.folder' -> 'folder' move when skip_dotfiles is enabled
2020-04-06 20:05:06 +10:00
abraunegg 31629e697f
Update error logging output to be consistent (#860)
* Update error logging to be consistent when initialising fails
* Update error logging output to handle HTML error response reasoning
2020-04-06 19:40:42 +10:00
abraunegg c41adda7a0
Handle zero-byte modified files for OneDrive Business (#854)
* OneDrive does not support the uploading of zero-byte files when uploading via the OneDrive Web UI, however it is possible to upload then via the API as a 'new file'. If a local zero-byte file is modified, upload this as a 'new file' rather than attempt to upload as a normal modified file.
2020-04-02 19:25:22 +11:00
abraunegg 9ae6f8e808
Handle .nosync directive when downloading new files (Issue #827) (#842)
* Handle .nosync directive when downloading new files to an existing directory that is (was) in sync
* Handle the situation where a .nosync is created after some sort of partial download, so that the file, whilst now unwanted, is not downloaded
2020-04-02 07:39:02 +11:00
abraunegg 0170884bd1
Update --single-directory handling for --dry-run and --resync (#856)
* Update --single-directory handling when using --dry-run
* Update --single-directory handling when using --resync and items to sync are in a child folder rather than root, leading to parent database issues (foreign key constraint)
2020-04-02 06:56:50 +11:00
abraunegg 50fb2d8bfa
Update application output when using --dry-run and --resync and database is corrupt (Issue#844) (#845)
* Update application output when database is corrupt and how to resolve by using --resync
* Handle --dry-run and --resync correctly - do not copy the database, it may be corrupt
* Always check if items-dryrun.sqlite3 still exists, remove it as it maybe corrupt when --dry-run is used
2020-03-30 06:55:27 +11:00
TheOnlyTrueJonathanHeard 3708828806
Handle renaming dotfiles to another dotfile when skip_dotfile=true in monitor mode (#843)
* Handle renaming dotfiles to another dotfile when skip_dotfile=true in monitor mode

With thanks to @TheOnlyTrueJonathanHeard
2020-03-29 07:08:01 +11:00
GeorgeKalovyrnas caf51886cf
Escape parentheses from path or file names (#841)
* Escape parentheses from path or file names, when doing comparison with regex

With thanks to @GeorgeKalovyrnas
2020-03-28 07:49:33 +11:00
TheOnlyTrueJonathanHeard 7170498a95
Handle renaming files to a name starting with a dot when skip_dotfile=true (#838)
* Handle renaming files to a name starting with a dot when skip_dotfiles = true

With thanks to @TheOnlyTrueJonathanHeard
2020-03-28 07:20:00 +11:00
abraunegg feb3d96a7e
Release files for 2.4.0 (#833)
* Release files for 2.4.0
2020-03-23 09:18:06 +11:00
abraunegg 6f85d539da
Update client identifier and auth scopes (Issue #769) (#770)
* Update the client identifier to 'd50ca740-c83f-4d1b-b616-12c519384f0c'
* Update User Agent identifier to comply with OneDrive traffic decoration requirements
* Provide 'config' file option to modify / update client identifier to override application default
2020-03-23 07:01:51 +11:00
abraunegg 896a4676f5
Update regex parsing of response URI (#830)
* Update regex that extracts the response code from the response URI to avoid potentially generating a bad request to OneDrive, leading to a 'AADSTS9002313: Invalid request. Request is malformed or invalid.' response. With thanks to @zfil for fix.
2020-03-21 07:20:19 +11:00
abraunegg edd365d21b
Update error response handling for catching HTTP 429 responses - activityLimitReached (Issue #815) (#816)
* Overhaul OneDrive error response handling for 429 errors by utilising the HTTP response header Retry-After to configure the correct 'retry' window. If no retry window is set, defaults to 120 seconds.
* Update error response messaging when a 429 response is received
* Update how the original OneDrive query is retried when a 429 response is received
* Update the User Agent string to be more compliant with OneDrive decoration requirements to assist in avoiding 429 responses due to incorrect User Agent string being used. Updated to: ISV|abraunegg|OneDrive_Client_for_Linux/v%version_tag%
2020-03-20 06:12:47 +11:00
abraunegg 1e8395fb73
Handle moving files into a skipped .folder when skip_dotfiles = true (#828)
* If skip_dotfiles = true, moving files into a skipped .folder should not throw an error and should be removed from their previous location on OneDrive
* Add log message to indicate why delete operation is occurring on item which was moved
2020-03-20 05:35:21 +11:00
abraunegg 4cb0ebbfde
Update logging output (Issue #818) (#819)
* Tighten up the logging output to better clarify what is occurring when sync_list is being used
2020-03-15 06:58:57 +11:00
abraunegg b1fcb814c3
Update application output when just authorising application (Issue #820) (#821)
* Update application output to be clearer when just authorising the application and --synchronize or --monitor not passed in
* Update usage.md with updated authorize details and example
2020-03-15 06:29:44 +11:00
abraunegg 1bb4ed765c
Always log when a file is skipped due to the item being invalid (Issue #811)
* If an item is to be skipped due to being 'invalid' log this without the requirement of --verbose
2020-03-06 09:20:47 +11:00
abraunegg a4d72fc73c
Catch when OneDrive throws an error response on initialising (Issue #802) (#804)
* Catch when OneDrive throws an error response (40x, 50x) on initialising
2020-03-02 12:54:56 +11:00
abraunegg 76b80e953e
Update file handling when a failed download has occured due to CTRL-C (#800)
* If CTRL-C is used when downloading a file, remnants of this file may still exist when next sync occurs. If the file is not present in the local cache database, redownload the file from OneDrive. Does not impact however if --local-first directive is being used.
2020-02-15 10:02:29 +11:00
abraunegg fc6e2d9cfe
Catch 429 HTTPS return code when query for oneDriveRootDetails fails (#796)
* Catch 429 HTTPS return code when query for oneDriveRootDetails fails & provide error feedback that this was the reason why application initialization failed
2020-02-12 06:16:38 +11:00
abraunegg 2321518faa
Update how the 'username' is determined from the running process for logfile naming (Issue #786) (#787)
* Fix an issue where the 'username' for the running process is not correctly determined on some platforms when enabling logging via --enable-logging
2020-02-03 09:10:57 +11:00
abraunegg 14cd47b56f
Implement #699 - Perform skip_dir explicit match only (#768)
* Implement #699 - Perform skip_dir explicit match only
2020-01-29 16:37:50 +11:00
abraunegg f2fe718389
Implement #763 - Delete local files after sync (#767)
* Implement #763 - Delete local files after sync
2020-01-27 08:42:00 +11:00
abraunegg 5c2ad041bd
Implement warning on big deletes to safeguard data on OneDrive (#621)
* When testing changes to onedrive client configuration, the new configuration might be invalid (see #458 for example) and you remove all your data on OneDrive accidentally. This new feature attempts to protect your data on OneDrive when performing large deletes, so that a large delete is detected and asks for confirmation before actually processing this request. This feature does not impact `--monitor` mode of operation, only standalone mode of operation.
2020-01-03 07:46:58 +11:00
abraunegg 8549719df3
Fix --dry-run output to be consistent when using --upload-only & --no-remote-delete (Issue #758)
* Fix --dry-run output when using --upload-only & --no-remote-delete and deleting local files
2019-12-30 10:17:10 +11:00
abraunegg 217d72486d
Change synclist override flag to false as default when not using sync_list (#745)
* Set 'syncListConfiguredOverride' to false by default, only set to true at start if sync_list is true
* Remove logAndNotify as it is excessive for each change bundle to inform the desktop
* Add a log entry when a monitor sync loop with OneDrive starts & completes
2019-12-17 05:32:38 +11:00
abraunegg 8ecd1989b4
Fix where an item now out of scope should be flagged for local delete (#733)
* Fix where an item now out of scope should be flagged for local delete
2019-11-23 11:20:58 +11:00
Norbert Preining 133c45e545 Catch DNS resolution error (#730)
* Catch the DNS error the same way we catch the timeout error
2019-11-23 06:43:28 +11:00
abraunegg db5ceccffb
Check database for excluded sync_list items previously in scope (Issue #723) (#724)
* When using sync_list and files are moved out of sync scope online via OneDrive website, check if the item was previously in-sync in database. if true, flag for local delete.
* Rework fix for #568 so that if a directory is moved / created in sync_list scope, on the 'true-up' sync, any files that were moved with the folder are now downloaded
* Update when sync_list override is done
* Add debugging for each loop for sync_list scan handling
2019-11-21 19:58:34 +11:00
abraunegg 387bd53438
Add debug message to when a message is sent to dbus or notification daemon (#728)
* Add debug message when message is sent to notify daemon
* Update usage document to detail how to enable non-root systemd usage + notifications
2019-11-21 18:59:45 +11:00
abraunegg 0445ae03d3
Update Microsoft restriction and limitations (#725)
* Update Microsoft restriction and limitations about windows naming files to include '~' for folder names
2019-11-14 11:53:07 +11:00
abraunegg c876b9c575
Retry session upload fragment when transient errors occur to prevent silent upload failure (#721)
* When uploading data to OneDrive via a session, and OneDrive throws an exception response, retry the fragment upload instead of silently failing without any error message
* Add error message & error logging when uploading data via a session and OneDrive throws an exception
2019-11-11 20:27:34 +11:00
abraunegg 7b4ba2e5d2
Only monitor top level single directory + its children if using --single-directory (Issue #709)
* If --single-directory is being used, and we are using --monitor, only set inotify watches on the single directory
2019-11-02 10:41:18 +11:00
Tudor Brindus 7ba59e0b1d Fix --monitor when used with --download-only (#708)
* Fix --monitor when used with --download-only. This fixes the regression introduced in 12947d160f.
2019-10-31 14:59:44 +11:00
abraunegg df3fba1be0
Catch a 412 response when moving files (Issue #706) (#707)
* Catch a 412 response when moving files right after upload.
2019-10-31 12:58:09 +11:00
abraunegg 12947d160f
Fix unhandled monitor initialisation exception (Issue #704) (#705)
* Catch MonitorException when initialisation failure occurs, print error and exit ... cant enter monitor loop if we cant initialise correctly.
* Cleanup and add documentation update
2019-10-30 17:46:02 +11:00
abraunegg 9b3179540f Revert "Fix unhandled monitor initialisation exception"
This reverts commit e1be2b1e55.
2019-10-30 06:33:11 +11:00
abraunegg e1be2b1e55 Fix unhandled monitor initialisation exception
* Catch MonitorException when initialisation failure occurs, print error and exit ... cant enter monitor loop if we cant initialise correctly.
2019-10-30 06:32:17 +11:00
abraunegg fec892b038
Fix call from --monitor to observe --no-remote-delete (#703)
* Fix call from --monitor to observe --no-remote-delete
2019-10-30 05:09:40 +11:00
abraunegg e99872eaa2
Fix 'Key not found: name' when computing skip_dir path (#701)
* OneDrive API difference between Personal & Business where some Business accounts do not send 'name' key with parent reference. Check for existance of 'name' key before use.
2019-10-26 14:17:40 +11:00