Commit graph

202 commits

Author SHA1 Message Date
abraunegg
ea7c3abd2d
Fix Bug #3355: Fix that long running big upload (250GB+) fails because of an expired access token (#3361)
* Revert back to v2.5.5 performSessionFileUpload() and apply minimal change for upload session offset handling to prevent desynchronisation on large files
* Add specific 403 handler for when the upload session URL itself expires
* Add 'file_fragment_size'
* Clean up debug logging output
* Add 'tempauth' to spelling words
* Update documentation URL's
* Ensure that on each fragment upload, whilst the application is using the 'tempauth' for session upload, the global OAuth2 token needs to be checked for validity and refreshed if required
* Add limit check for 'file_fragment_size' option
* Add to default 'config' file
* Update documentation for 'file_fragment_size'
* Add 'file_fragment_size' to --display-config output
* Add --file-fragment-size option to enable use via Docker option
* Add to manpage
* Update Docker entrypoint
* Update Docker | Podman documentation
* Update logging output to include connection method to URL
* Update Upload Session URL expiry update to include UTC and LocalTime values
* Update comment which was dropped / missed
* Clarify that this is the OAuth2 Access Token
* Clarify that the expiry timestamp is localTime
* Update PR with dynamic use of fragment size if fileSize > 100MiB
* Enforce multiple 320KiB for fragment size to align to Microsoft documentation
* Fix Docker entrypoint and confirm working for ONEDRIVE_FILE_FRAGMENT_SIZE
* Change 'defaultMaxFileFragmentSize' to 60
* Revise fragmentSize calculation to be as close to 60 MiB as possible without breaching Microsoft documented threshold
2025-07-03 17:21:16 +10:00
abraunegg
a640f2ff04
Catch unhandled OneDriveError exception due to libcurl failing to access the system CA certificate bundle (#3322)
* Catch unhandled OneDriveError exception when there is a SSL or disk space issue
* Add setDisableSSLVerifyPeer() and setEnableSSLVerifyPeer() functions, but ensure they are not used
* Correct warning messages from PR 3320
* Update allow.txt - add verifypeer
2025-06-08 13:36:18 +10:00
abraunegg
d8d695600a
Update OAuth2 Interactive Authorisation Flow prompts (#3323)
* Update OAuth2 Interactive Authorisation Flow prompts to remove any ambiguity on what actions a user needs to take
2025-06-08 13:20:57 +10:00
abraunegg
538ebe4be6 Revert "Update PR"
This reverts commit b020455534.
2025-06-08 07:03:02 +10:00
abraunegg
b020455534 Update PR
* Add functions, but ensure they are not used
* Correct warning messages from PR 3320
2025-06-08 07:00:54 +10:00
abraunegg
810197cc05
Implement FR #2693: Implement OAuth2 Device Authorisation Flow (#3313)
Adds support for Microsoft’s OAuth2 Device Authorisation Flow, enabling the client to authenticate using device and user codes on a second device. This is particularly useful for headless or limited-input environments where interactive browser login is not possible.

Includes:
- Initiating device code requests and displaying user instructions
- Polling token endpoint until user authorises the device or the code expires
- Error handling for pending, declined, and expired authorisation states
- Countdown display showing remaining authorisation window

This feature is enabled via the `use_device_auth` config option
2025-06-03 07:47:56 +10:00
abraunegg
61e5a1edb6
Implement FR #3209: Add native support for authentication via Intune dbus interface (#3274)
* Adds support for calling `acquireTokenInteractively` and `acquireTokenSilently` via D-Bus
* Parses and handles the full `brokerTokenResponse` object returned by the Intune broker
* Stores and reuses `account` data to enable silent token refresh without repeated interactive authentication
* Ensures the access token and its expiry time are properly calculated and stored for consistent token management
* Fallback to interactive authentication is triggered if silent authentication fails
2025-05-26 17:14:50 +10:00
abraunegg
9d329bcafc
Fix Bug #3167: Fix JSON exception on webhook subscription renewal due to 308 redirect (#3172)
* Fix JSON exception on subscription renewal when a 308 response is provided by the Microsoft Graph API
* Add support for 307 and 308 re-directions
* Update how create subscription is performed and what resourceItem path is used
2025-03-28 08:53:38 +11:00
abraunegg
fd74b19cae
Fix regression of Microsoft OneNote package folder being created in error (#3160)
* Fix regression of Microsoft OneNote package folder being created in error
2025-03-20 20:27:44 +11:00
abraunegg
e58896747f
Fix Bug #3115: Add further driveId validation when processing API response data for a OneDrive Personal Shared Folder (#3116)
* Remove 'sea8cc6beffdb43d7976fbc7da445c639' check
* Add further personal account driveId length checks when generating a /delta response for a Shared Folder
* Fix selectRemoteTypeByRemoteDriveId() function to also use the remote item id so that the correct DB record is responded with. This ensures that 'localPathExtension =' reflects the correct local path extension and not some other incorrect local path
* If the 'shared' JSON structure is missing, add a debug message
* Add further checks for invalid 15 character driveId when creating Shared Folder references
* Ensure that the 'driveId' value is correctly fetched and used due to 'UPPERCASE' and 'lowercase' values in API JSON response
* Increase DB version to ensure all old records in the DB are purged
* Sanitise JSON output for debug logging when enabled
* Simplify isValidUTF8Timestamp()
* Debug logging is not correct - missing all single --verbose entries which should be also included when performing a debug capture.
* Based on application debug logs, if /delta fails to send Shared Folders, and the client goes to create these online, we know that these then exist online and are a Shared Folder. Handle in a similar manner to OneDrive Business Shared Folders logic to create the required database tie records if the API fails to send that data originally.
* Remove double call to selectiveSync.isFileNameExcluded
* Align invalid UTF-8 message to be consistent with other UTF-8 failure messages

This PR specifically works around the Microsoft OneDrive change:
* Microsoft moving all OneDrive Personal Accounts to a new backend platform. This is the 'sea8cc6beffdb43d7976fbc7da445c639' change: https://github.com/OneDrive/onedrive-api-docs/issues/1890
* Microsoft failing to provide the Graph API data in the /delta call for accounts moved to the new backend platform: https://github.com/OneDrive/onedrive-api-docs/issues/1891
2025-03-10 06:03:50 +11:00
abraunegg
f8123a2b0a
Implement FR #2719: Add information about file creator/last editor as extended file attributes (#3113)
* Implement feature request to add information about file creator/last editor as extended file attributes
* On a --resync , if the data exists on disk, and 'write_xattr_data' has been enabled, write the xattr data out
2025-02-20 21:14:04 +11:00
abraunegg
1166f86366
Enhancement: Move curl debug logging to --debug-https (#3120)
* As all the curl engine reuse is now stable, move debug logging of curl engine stack to --debug-https
* Extract the DataCentre information from the http response header to use in application output
* Align 'Remaining Free Space' to accommodate Data Centre output
2025-02-14 15:34:22 +11:00
abraunegg
0216b63c61
Implement FR #3107: Add config option to not set file and directory permissions (#3112)
* Implement configuration option to not set directory and file permissions
2025-02-13 19:27:14 +11:00
abraunegg
6634284efa
Fix regression in handling uploading new and modified content to OneDrive Business and SharePoint (#3032)
* Fix regression from v2.4.x in handling uploading new and modified content to OneDrive Business and SharePoint to not create new versions of files post upload which adds to user quota
* Ad configuration option 'create_new_file_version' to create new versions if that is the desire
2024-12-09 14:24:33 +11:00
abraunegg
53c74c4a63
Update --dry-run authentication message (#3022)
* Update --dry-run authentication message based on question raised in #3021
2024-12-02 10:59:09 +11:00
abraunegg
228e7db188
Implement Feature Request: Support Permanent Delete on OneDrive (#2999)
* Implement Feature Request: Support Permanent Delete on OneDrive (#2803)
2024-11-26 19:30:41 +11:00
abraunegg
701c30bf0b
Cleanup debug logging output (#3007)
* Use debugHTTPSResponse as a global variable
* Only output HTTP response headers if using --debug-https
2024-11-25 06:43:39 +11:00
abraunegg
ae84b8cfef
Fix spelling problems as identified by GitHub Action check-spelling (#2946)
* Fix spelling problems as identified by GitHub Action check-spelling
2024-11-04 11:32:18 +11:00
abraunegg
2e93c1f4ac
Fix that curl generates a SIGPIPE that causes application to exit due to upstream device killing idle TCP connection (#2906)
* Add releaseTimestamp to track when a curlEngine was last used to determine if this engine should be reused or not
* Add forced HTTP/1.1 downgrade based on bad curl versions usually found in Ubuntu distributions
* Add note regarding 'Could not connect to server on handle'
* Make stale curl idle value a config option and update associated documentation
2024-10-21 10:13:20 +11:00
abraunegg
576b46ea70
Improve performance with reduced execution time and lower CPU/system resource usage (#2880)
* Use global variables for logging verbose|debug logging state
* Only perform debug logging steps including calculations and/or output if debugging is actually enabled
* Only perform verbose logging steps including calculations and/or output if verbose is actually enabled
* Optimise code execution and reduce resource usage by ~12% in real time, ~65% in CPU time, and ~87% in system time.
2024-10-08 05:47:50 +11:00
abraunegg
a3522fda30
OneDrive Client for Linux v2.5.0 (#2805)
OneDrive Client for Linux v2.5.0

---------

Signed-off-by: Thomas Staudinger <Staudi.Kaos@gmail.com>
Co-authored-by: JC-comp <147694781+JC-comp@users.noreply.github.com>
Co-authored-by: Dimitri Papadopoulos Orfanos <3234522+DimitriPapadopoulos@users.noreply.github.com>
Co-authored-by: Pierrick Caillon <megamisan@users.noreply.github.com>
Co-authored-by: Pierrick Caillon <pierrick.caillon@megami.fr>
Co-authored-by: Thomas Staudinger <Staudi.Kaos@gmail.com>
Co-authored-by: Yuan Liu <Lyncredible@users.noreply.github.com>
2024-09-16 11:14:46 +10:00
abraunegg
fb0a5f0796
Fix unhandled exception when authurl path is non-existent (#2456)
* Catch an unhandled exception when the user sets the authurl file to a location that cannot be accessed
2023-07-23 10:13:03 +10:00
abraunegg
dfd1d1aa4a
Implement #2414 to allow HTTP session timeout(s) tuning via config (#2425)
* Implement #2414 to allow HTTP session timeout(s) tuning via config (taken from v2.5.x)
2023-06-20 09:28:17 +10:00
abraunegg
6607ba537b
Fix webhook subscription creation for SharePoint Libraries (#2339)
* Fix webhook subscription creation for SharePoint Libraries
2023-03-18 10:03:49 +11:00
abraunegg
678add91f8
Update --get-O365-drive-id error handling (#2266)
* Ensure a 'Personal' account type is not being used
* If the /sites?search API cannot be found, display a more appropriate error message
2022-12-16 06:08:46 +11:00
abraunegg
d169dfc642
Improve application logging output for error messages (#2100)
* When enabling system logging to a log file, the actual ERROR line is forced to a new line in the application log. The reason for this is the \n prefix in the error message, which was in place so that when performing CLI logging or systemd logging, the error message would be displayed clearly. This change removes the \n from the actual error message, but inserts a newline before the error message is displayed (and also in some cases post error message) - thus keeping the application runtime look and feel, but improving the application log output.
2022-08-18 09:14:13 +10:00
abraunegg
d57a695632
Add option to request readonly access in oauth authorization step (#2065)
* Add option to request readonly access in oauth authorization step
* Add application-security.md to document application security
2022-08-06 08:24:51 +10:00
abraunegg
b2991fded9
Fix that the download progress bar doesn't always reach 100% when rate_limit is set (#2056)
* Fix that the download progress bar doesn't always reach 100% when rate_limit is set
2022-07-29 07:07:08 +10:00
abraunegg
c82e90a140
Ensure that any active http instance is shutdown before any application exit (#2035)
* When the client needs to exit due to an issue, ensure that the curl http instance is shutdown before the exit is performed. This also potentially solves some segmentation faults seen on Ubuntu|Debian platforms due to issues in the shared library libphobos2-ldc-shared.so.X
2022-07-07 09:37:45 +10:00
Sebastien Badia
0ab2955bd7
Various little spelling fixes (check with lintian during Debian packaging) (#2027)
* contrib: remove bash hashbang from completion

This file starts with the #! sequence that marks interpreted scripts, but
it is a bash completion script that is merely intended to be sourced.

* src: spelling error (Attemtping => Attempting)

* src: spelling error (reponse => response)

* src: spelling error (sucessfully => successfully)
2022-07-02 05:14:50 +10:00
abraunegg
0e0fdacf7c
Switch to using curl defaults for HTTP/2 operations (#2026)
* Switch to using curl defaults for HTTP/2 operations
2022-07-01 15:09:11 +10:00
abraunegg
0fffb8fbc0
Implement a check to validate local filesystem available space before attempting file download (#1971)
* Implement a check to validate local filesystem available space before attempting file download
* Implement 'space_reservation' config option with a default value of 50 MB
* Prevent the original run-away logging error message 'Failed initialization on handle XXXX' from occurring if the system is out of space
* Update documentation and man page
2022-06-01 05:57:05 +10:00
abraunegg
8b30c00c81
Update Internet connect disruption retry handling (#1909)
* Update Internet connect disruption retry handling and logging output
2022-04-21 08:03:22 +10:00
abraunegg
a5bd4e9133
Implement feature request to mark partially-downloaded files as .partial (#1868)
* Implement feature request to mark partially-downloaded files as .partial during download
2022-03-10 07:01:08 +11:00
abraunegg
3124b2dcf2
Implement feature request --modified-by (#1869)
* Implement --modified-by to display who last modified a file and when the modification was done
2022-03-10 07:00:07 +11:00
abraunegg
d68c27fb4f
Add --reauth to allow easy re-authentication of the client (#1867)
* Add --reauth to allow easy re-authentication of the client
2022-03-08 05:35:00 +11:00
abraunegg
76bd6e26a5
Catch errors from OneDrive when initialising subscriptions (#1846)
* Catch errors from OneDrive when initialising subscriptions
2022-03-06 10:02:14 +11:00
abraunegg
f72f6c9262
Explicitly set libcurl options (#1789)
* Explicitly set libcurl options to ignore SIGPIPE, use TCP NAGLE and ensure we are re-using connections. These should be 'enabled by default' by libcurl, but we need to ensure we are setting them to their supposed default

Potentially resolves the following issues: #494, #753, #792, #884, #1162, #1408, #1520, #1526 + others.
2022-01-12 09:14:04 +11:00
Kleckkinator
58d40c5174
Add --operation-timeout argument (#1759)
* This change is to create an argument version of the config operation_timeout.

There are many times where one may need to temporarily increase the timeout for an operation of a single large file that takes longer than the configured timeout. The command line argument is an excellent way to override the default or 'config' set option if required.
2021-12-30 11:19:55 +11:00
Yuan Liu
1e2827ad1c
Add Real-Time syncing of remote updates via webhooks (#1718)
* Add Real-Time syncing of remote updates via webhooks

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2021-11-24 06:54:28 +11:00
abraunegg
ac2e07526a
Revert PR #1626 (#1716)
Revert PR #1626 due to deprecation function that needs to be resolved before PR can be merged again
2021-11-23 11:14:29 +11:00
Winston R. Milling
5d7e4532ef
Add --auth-response option and expose through entrypoint.sh (#1628)
* This allows for the use of the `--auth-files` switch in the docker environment.

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2021-11-23 07:08:04 +11:00
Yuan Liu
bfeeae91de
Add Real-Time syncing of remote updates via webhooks (#1626)
* Add Real-Time syncing of remote updates via webhooks

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2021-11-23 07:06:13 +11:00
Kleckkinator
4d25e68c65
Added operation_timeout as a new configuration (#1685)
* Added operation_timeout as a new configuration
2021-11-10 07:46:41 +11:00
abraunegg
f3f04be2dc
Always prompt for credentials on authentication (#1673)
* Ensure that each time the authentication URL is used, we prompt for credentials, rather than using any cached credentials in the browser
2021-11-02 06:58:12 +11:00
abraunegg
1fbe1d34a7
Fix application crash due to a conversion overflow when calculating file offset for session uploads (#1558)
* Fix an unhandled application crash when calculating the required offset for a file fragment size when resuming a session upload
2021-07-12 05:59:32 +10:00
abraunegg
54a6575709
Fix application crash due to invalid UTF-8 sequence in the pathname for the application configuration (#1551)
* Catch a Invalid UTF-8 handling error when attempting to initialise the application. This is caused by the 'path' to the application configuration (typically ~/.config/onedrive) contains a bad UTF-8 character thus cannot be read / initialised
2021-07-07 06:26:52 +10:00
abraunegg
fc5d7f9327
Fix application crash and incorrect handling of --single-directory when syncing a OneDrive Business Shared Folder due to using 'Add Shortcut to My Files' (#1542)
* When syncing OneDrive Business Shared Folders and using --single-directory, select correct driveId and itemId for the remote directory that needs to be synced
* Normally, the 'remoteItem' field will contain 'fileSystemInfo' however, if the user uses the 'Add Shortcut ..' option in OneDrive WebUI to create a 'link', this object, whilst remote, does not have 'fileSystemInfo' in the expected place, this leading to a application crash
2021-07-06 18:11:53 +10:00
abraunegg
b8717fbc53
Use 'nextLink' value if present when searching for specific SharePoint site names (#1329)
* Use 'nextLink' value if present when searching for specific SharePoint site names
* Update error output to provide more details why an error occurred if a SharePoint site lacks the details we need to perform the match
2021-03-13 05:11:44 +11:00
abraunegg
81e40ba76d
Update 429 retry-after response handling (#1305)
* Always check the response headers when a 429 response is returned from OneDrive, so that the 'retry-after' value is correctly consumed and used
2021-03-03 11:12:27 +11:00