Commit graph

634 commits

Author SHA1 Message Date
Yuan Liu 1e2827ad1c
Add Real-Time syncing of remote updates via webhooks (#1718)
* Add Real-Time syncing of remote updates via webhooks

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2021-11-24 06:54:28 +11:00
abraunegg 2f47beab60
Fix overwriting local files with older versions from OneDrive when items.sqlite3 does not exist and --resync is not used (#1717)
* Resolve issue where --resync is not used and items.sqlite3 is not available at application startup, local files, whilst potentially newer, will not be preserved thus leading to a potential data loss scenario
2021-11-24 06:44:45 +11:00
abraunegg ac2e07526a
Revert PR #1626 (#1716)
Revert PR #1626 due to deprecation function that needs to be resolved before PR can be merged again
2021-11-23 11:14:29 +11:00
abraunegg 2901534171
Add --disable-download-validation (#1686)
* Add --disable-download-validation to allow downloading files from OneDrive where SharePoint and the OneDrive API mis-represents the values in the API as compared to the actual HTTP server response sent to the client. In the API JSON response we get a "size" value of X, but the HTTP Server Content Length reports a size of Y. When this occurs, the download will report as failed. This was seen as part understanding the cause of https://github.com/abraunegg/onedrive/discussions/1667
2021-11-23 07:34:16 +11:00
Winston R. Milling 5d7e4532ef
Add --auth-response option and expose through entrypoint.sh (#1628)
* This allows for the use of the `--auth-files` switch in the docker environment.

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2021-11-23 07:08:04 +11:00
Yuan Liu bfeeae91de
Add Real-Time syncing of remote updates via webhooks (#1626)
* Add Real-Time syncing of remote updates via webhooks

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2021-11-23 07:06:13 +11:00
abraunegg 540c2369f3
Fix error 'Key not found: remaining' with Business Shared Folders (#1713)
* Fix an API regression where the 'remaining' value for some API responses is no longer present in the 'quota'  JSON response
2021-11-22 17:42:29 +11:00
abraunegg cceb3cccdd
Do not re-auth on --logout (#1705)
* When issuing a --logout, do not perform a reauthentication of the application
2021-11-17 06:05:11 +11:00
abraunegg 6febb13408
Catch unhandled API response errors when listing OneDrive Business Shared Folders (#1704)
* When listing OneDrive Business Shared Folders, try the API query and catch any API error response, rather than blindly attempting the API query
2021-11-17 05:51:30 +11:00
abraunegg 2f660ff7b0
Catch unhandled API response errors when querying OneDrive Business Shared Folders (#1703)
* When syncing OneDrive Business Shared Folders, try the API query and catch any API error response, rather than blindly attempting the API query
2021-11-16 19:48:44 +11:00
Kleckkinator 4d25e68c65
Added operation_timeout as a new configuration (#1685)
* Added operation_timeout as a new configuration
2021-11-10 07:46:41 +11:00
abraunegg f3f04be2dc
Always prompt for credentials on authentication (#1673)
* Ensure that each time the authentication URL is used, we prompt for credentials, rather than using any cached credentials in the browser
2021-11-02 06:58:12 +11:00
abraunegg 606ee52ad2
Fix edge case with OneDrive Personal Shared Folders (#1586)
* In a `--resync --upload-only --single-directory 'dir'` scenario, and where the root 'dir' for --single-directory is a 'shared folder' we will not have the 'tie' DB entry created because of using --upload-only because we do not download the folder structure from OneDrive. As a result, query of the folder will fail and file uploads will fail.

Simulate the 'tie' DB record only when --resync --upload-only --single-directory 'dir' is being used, and if that folder is 'remote' and if we are using a 'personal' account.

The 'impact' of this however is, because of `--resync --upload-only` being used, local files are not in the local DB cache anymore, thus are treated as *new* files, thus, will be attempted to be re-uploaded.
2021-07-22 08:17:02 +10:00
abraunegg 8a20275dd2
Fix getPathDetailsByDriveId query when using --dry-run and a nested path with --single-directory (#1583)
* Fix getPathDetailsByDriveId query when using --dry-run and a nested path with --single-directory
* Fix fake response generation to use generated values for all account types to avoid DB lookup failures when using --dry-run for Personal account types
2021-07-20 13:27:35 +10:00
abraunegg 6d92bc8210
Fix that Business Shared Folders with parentheses are ignored (#1560)
* Fix issue where matching a OneDrive Business Shared Folder that contains '()' is not matched correctly
2021-07-13 18:31:48 +10:00
abraunegg 1fbe1d34a7
Fix application crash due to a conversion overflow when calculating file offset for session uploads (#1558)
* Fix an unhandled application crash when calculating the required offset for a file fragment size when resuming a session upload
2021-07-12 05:59:32 +10:00
c-maia 7171b83f5b
Fix error message when deleting a large number of files (#1559)
"delete" word appears twice in the error message. This commit removes the extra "delete".
2021-07-11 07:37:57 +10:00
abraunegg 54a6575709
Fix application crash due to invalid UTF-8 sequence in the pathname for the application configuration (#1551)
* Catch a Invalid UTF-8 handling error when attempting to initialise the application. This is caused by the 'path' to the application configuration (typically ~/.config/onedrive) contains a bad UTF-8 character thus cannot be read / initialised
2021-07-07 06:26:52 +10:00
abraunegg fc5d7f9327
Fix application crash and incorrect handling of --single-directory when syncing a OneDrive Business Shared Folder due to using 'Add Shortcut to My Files' (#1542)
* When syncing OneDrive Business Shared Folders and using --single-directory, select correct driveId and itemId for the remote directory that needs to be synced
* Normally, the 'remoteItem' field will contain 'fileSystemInfo' however, if the user uses the 'Add Shortcut ..' option in OneDrive WebUI to create a 'link', this object, whilst remote, does not have 'fileSystemInfo' in the expected place, this leading to a application crash
2021-07-06 18:11:53 +10:00
abraunegg e236c7cf12
Handle OneDrive API Bad Request response when querying if file exists (#1535)
* Add exception handling for when the API returns a 400 error when attempting to query a path on OneDrive. If the path generates a 'bad request' response, this needs to be correctly handled.
2021-07-01 06:18:48 +10:00
abraunegg 761cf3eb87
Support DMD 2.097.0 as compiler (#1505)
* Support DMD 2.097.0 as compiler and resolve deprecation messages
2021-06-07 08:26:36 +10:00
abraunegg b0771f79e1
Fix application crash when SharePoint displayName is being withheld when using --get-O365-drive-id (#1445)
* Handle bad ShaprePoint data when the API does not return the expected data points when using those references to display what SharePoint sites are available when using --get-O365-drive-id
2021-05-10 08:04:14 +10:00
abraunegg c9ca00466a
Add file check when adding item to database if using --upload-only --remove-source-files (#1437)
* Add a file check when using --upload-only --remove-source-files so that parental paths are added to the database, to allow child objects to be uploaded in this scenario
2021-05-06 06:00:55 +10:00
abraunegg b7eedbd8cd
Fix that options --upload-only & --remove-source-files are ignored on an upload session restore (#1399)
* Fix that options --upload-only & --remove-source-files are ignored on an upload session restore
2021-04-12 06:16:23 +10:00
abraunegg 3d5957986a
Fix 'sync_list' handling of inclusions when name is included in another folders name (#1390)
* Fix 'sync_list' handling of inclusions when name is included in another folders name to avoid creating empty parental root folders. When a 'sync_list' entry ends with '/*' the common path matching causes a sub path match to occur. Before directly hitting as an exact match, perform an additional check of the path to sync against the sync_list entry without the '/*' being present.
2021-04-09 08:18:00 +10:00
abraunegg 615aa05b42
Fix an unhandled Error 412 when uploading modified files to OneDrive Business Accounts (#1394)
* Update error 412 handling when uploading modified files to OneDrive Business Accounts
2021-04-09 06:21:01 +10:00
abraunegg 94cc6ee3d6
Revert #1343 as OneDrive API is now fixed in relation to handling '%' within filenames (#1386)
* Revert #1343 as OneDrive API is now fixed in relation to handling '%' within filenames
2021-04-06 20:34:03 +10:00
abraunegg 1203aebeb7
Fix file kept in database when using --no-remote-delete & --remove-source-files (#1383)
* Fix file kept in database when using --no-remote-delete & --remove-source-files
2021-04-06 06:43:19 +10:00
abraunegg b1b814f10a
Fix uploading documents to Shared Business Folders when shared folder exists on a SharePoint site (#1352)
* Fix uploading documents to Shared Business Folders when shared folder exists on a SharePoint site due to Microsoft Sharepoint 'enrichment' of files

See: https://github.com/OneDrive/onedrive-api-docs/issues/935 for further details
2021-03-27 07:31:03 +11:00
abraunegg 5c8083001a
Update logging output to handle directory entries only (#1364)
* When in --monitor mode, avoid outputting misleading logging when the new or modified item is a file, not a directory
2021-03-27 07:29:08 +11:00
abraunegg fd04d632d6
Fix application crash when unable to rename folder structure due to unhandled file-system issue (#1366)
* Fix application crash when unable to rename folder structure due to unhandled file-system issue
2021-03-27 07:28:40 +11:00
abraunegg 6b20478635
Fix 'Item cannot be deleted from OneDrive because it was not found in the local database' (#1354)
* When in --monitor mode, and there are multiple driveId being used (shared folders), when a deletion event occurs, search the DB using all the known driveids rather than just the default one. When using just the default driveid, if the file to be deleted resides on another drive, the following error message is printed: 'Item cannot be deleted from OneDrive because it was not found in the local database' - which is not entirely accurate as the item is in the database, it is just not being searched for correctly.
2021-03-19 17:29:55 +11:00
abraunegg 0dcc76f9f1
Update 'Skipping shared folder due to existing name conflict' check for Business Shared Folders (#1351)
* Update 'Skipping shared folder due to existing name conflict' check for Business Shared Folders
2021-03-18 13:24:26 +11:00
abraunegg 037a6b43e1
Support multiple 'document libraries' within a single Shared Library Site (#1350)
* Support providing the 'drive_id' for multiple 'document libraries' within a single Shared Library Site
2021-03-17 18:23:01 +11:00
abraunegg 6bec0ddc64
Correctly handle files which contain '%' as invalid files (#1343)
* Fix checking filenames which contain '%' which should be classified as invalid based on the MS OneDrive API Invalid characters designation
2021-03-15 16:24:55 +11:00
abraunegg b8717fbc53
Use 'nextLink' value if present when searching for specific SharePoint site names (#1329)
* Use 'nextLink' value if present when searching for specific SharePoint site names
* Update error output to provide more details why an error occurred if a SharePoint site lacks the details we need to perform the match
2021-03-13 05:11:44 +11:00
abraunegg 7786a17915
Remove depreciated config option 'force_http_11' (#1323)
* Remove depreciated config option 'force_http_11' which was flagged as depreciated by PR #549 in v2.3.6 (June 2019).
2021-03-08 07:17:48 +11:00
abraunegg f107427480
Fix 'sync_list' path handling for sub item matching (#1317)
* Fix 'sync_list' path handling for sub item matching, so that items in parent are not implicitly matched when there is no wildcard present
2021-03-05 09:15:44 +11:00
abraunegg 81e40ba76d
Update 429 retry-after response handling (#1305)
* Always check the response headers when a 429 response is returned from OneDrive, so that the 'retry-after' value is correctly consumed and used
2021-03-03 11:12:27 +11:00
abraunegg 1078795d6f
Fix support for '/*' regardless of location within sync_list file (#1293)
* Fix support for '/*' regardless of location within sync_list file
2021-03-02 14:28:31 +11:00
abraunegg 42b7945d9c
Support sync_list matching full path root wildcard with exclusions to simplify sync_list configuration (#1273)
* Support sync_list matching full path root wildcard with exclusions to simplify sync_list configuration
2021-02-14 08:55:49 +11:00
abraunegg cd7b7b0b68
Use correct path offset for sync_list exclusion matching (#1269)
* When processing sync_list entries, by default, if the path starts with '/', and offset of 1 is used to ensure correct path matching can be done. However, if the path starts with '!/' or '-/' to exclude a certain path, an offset of 1 was still being used, thus, exclusion matching was not occurring correctly. Use the correct offset, based on the 'sync_list' entry.
2021-02-14 06:43:55 +11:00
abraunegg 3ea16b619f
Revert the config & logpath change and check correct input entry (#1270)
* Revert the config & logpath change introduced via #1267 as the logging path should not be the value checked, the input path is what should be checked
2021-02-13 17:37:22 +11:00
abraunegg f37fdda836
Fix logging output when handing downloaded files (#1267)
* Fix logging output when processing downloaded files to remove potential ambiguity as to what events are actually occurring with individual file events post download
2021-02-11 04:07:45 +11:00
abraunegg 1bfd995153 Revert "Fix logging output when handing downloaded new files (#1265)"
This reverts commit 99a0f4bfed.
2021-02-10 13:33:32 +11:00
abraunegg 99a0f4bfed
Fix logging output when handing downloaded new files (#1265)
* Fix logging output when processing files to remove potential ambiguity as to what events are actually occuring with indivudual file events
2021-02-10 13:26:10 +11:00
abraunegg 66a172d13a
Correctly handle '~' when present in 'log_dir' configuration option (#1258)
* Handle '~' when present in 'log_dir' and expand based on user and shell environment  variables to ensure correct expansion.
2021-02-06 20:46:56 +11:00
abraunegg 2c253362bf
Improve upload handling of files for SharePoint sites (#1250)
* After a file has been uploaded to SharePoint, use the response JSON and compare the quickXorHash to that of the local file before the upload. If there is a difference, Microsoft has modified the file post upload, thus we now need to download this 'updated' file. If there is no hash difference, we do not need to download the file.
2021-02-01 16:46:12 +11:00
abraunegg 3b87602a81
Use correct driveId value to query for changes when using --single-directory (#1235)
* Use the driveId value rather than defaultDriveId when looking for changes when using --single-directory
2021-01-22 14:38:11 +11:00
abraunegg 6b0b6a66f7
Search all distinct drive id's rather than just default drive id for --get-file-link (#1230)
* Add new DB query to return distinct driveId's from database
* Use all distinct drive ID's to query if path is in-sync with OneDrive before attempting to retreive the item file link from OneDrive
2021-01-20 19:46:56 +11:00