Commit graph

1,467 commits

Author SHA1 Message Date
abraunegg
ac8cf36046
Update Docker OS versions (#3150)
* Update Docker OS versions March 2025
  * Use Fedora 41
  * Use Alpine 3.21
2025-03-16 07:38:04 +11:00
abraunegg
5d660c3b00
Fix OneDrive Personal Shared Folder removal issue (#3147)
* When removing the a OneDrive Personal Shared Folder locally, ensure we are only removing our 'link' to the folder, not the entire folder from the remote user
* Ensure the 'root' object is removed for the Shared Folder if there are no other 'children' for that Shared Folder parent.
2025-03-10 17:34:30 +11:00
abraunegg
e58896747f
Fix Bug #3115: Add further driveId validation when processing API response data for a OneDrive Personal Shared Folder (#3116)
* Remove 'sea8cc6beffdb43d7976fbc7da445c639' check
* Add further personal account driveId length checks when generating a /delta response for a Shared Folder
* Fix selectRemoteTypeByRemoteDriveId() function to also use the remote item id so that the correct DB record is responded with. This ensures that 'localPathExtension =' reflects the correct local path extension and not some other incorrect local path
* If the 'shared' JSON structure is missing, add a debug message
* Add further checks for invalid 15 character driveId when creating Shared Folder references
* Ensure that the 'driveId' value is correctly fetched and used due to 'UPPERCASE' and 'lowercase' values in API JSON response
* Increase DB version to ensure all old records in the DB are purged
* Sanitise JSON output for debug logging when enabled
* Simplify isValidUTF8Timestamp()
* Debug logging is not correct - missing all single --verbose entries which should be also included when performing a debug capture.
* Based on application debug logs, if /delta fails to send Shared Folders, and the client goes to create these online, we know that these then exist online and are a Shared Folder. Handle in a similar manner to OneDrive Business Shared Folders logic to create the required database tie records if the API fails to send that data originally.
* Remove double call to selectiveSync.isFileNameExcluded
* Align invalid UTF-8 message to be consistent with other UTF-8 failure messages

This PR specifically works around the Microsoft OneDrive change:
* Microsoft moving all OneDrive Personal Accounts to a new backend platform. This is the 'sea8cc6beffdb43d7976fbc7da445c639' change: https://github.com/OneDrive/onedrive-api-docs/issues/1890
* Microsoft failing to provide the Graph API data in the /delta call for accounts moved to the new backend platform: https://github.com/OneDrive/onedrive-api-docs/issues/1891
2025-03-10 06:03:50 +11:00
abraunegg
9051d0d89a
Clarify troubleshooting steps (#3135)
* Clarify required troubleshooting steps
2025-02-24 20:13:33 +11:00
abraunegg
26602b84f1 Add API bug URL for 'sea8x' issue
* Add API bug URL for 'sea8x' issue
2025-02-24 06:38:02 +11:00
abraunegg
efecdf7e8f
Fix Bug #3126: Ignore 'sync_list' directory exclusion to scan for included data (#3130)
* When using 'sync_list' , we need to ignore specific exclusion to scan that path for new data, which may be actually included by an include rule, but the parent path is excluded
2025-02-23 07:13:57 +11:00
abraunegg
8ecdbab20f
Fix Bug #3122: Fix 'sync_list' globbing rule application (#3123)
* Add check for 'globbing' and 'wildcard' rules, that the number of segments before the first wildcard character need to match before the actual rule can be applied
* Only calculate ruleSegments and pathSegments once
* Use same calculation method to count path segments for applicability globbing rule as this is more consistent
2025-02-23 05:35:41 +11:00
abraunegg
a7c3dccb5c
Add account root id check for id account issue (#3129)
* Re-add 'sea8cc6beffdb43d7976fbc7da445c639' check which somehow was removed
2025-02-22 05:32:40 +11:00
abraunegg
f8123a2b0a
Implement FR #2719: Add information about file creator/last editor as extended file attributes (#3113)
* Implement feature request to add information about file creator/last editor as extended file attributes
* On a --resync , if the data exists on disk, and 'write_xattr_data' has been enabled, write the xattr data out
2025-02-20 21:14:04 +11:00
abraunegg
1166f86366
Enhancement: Move curl debug logging to --debug-https (#3120)
* As all the curl engine reuse is now stable, move debug logging of curl engine stack to --debug-https
* Extract the DataCentre information from the http response header to use in application output
* Align 'Remaining Free Space' to accommodate Data Centre output
2025-02-14 15:34:22 +11:00
abraunegg
0216b63c61
Implement FR #3107: Add config option to not set file and directory permissions (#3112)
* Implement configuration option to not set directory and file permissions
2025-02-13 19:27:14 +11:00
abraunegg
0e7d3f15f6
Implement FR #2544: Allow synchronizing smallest files first (#3114)
* Implement 'transfer_order' configuration option to allow the user to determine what order files are transferred in
2025-02-13 19:20:42 +11:00
abraunegg
68b61bef94 Calculate computedItemPath when required
* Calculate computedItemPath when required for Business Accounts and not before
2025-02-10 06:53:47 +11:00
abraunegg
768846a5dd
Fix path calculation for Client Side Filtering evaluations for Personal Accounts (#3111)
* Use original method for Personal Accounts, use new method for Business and SharePoint
2025-02-09 07:27:40 +11:00
abraunegg
96a3c2d7c9
Fix path calculation for Client Side Filtering evaluations (#3106)
* Correct the path calculation for 'skip_dir' evaluations to correctly cater for Shared Folders and the 'root' object
* Add debug logging if 'newItemPath' and requires calculation again before use
2025-02-08 05:32:35 +11:00
abraunegg
1c78078df7
Release files for 2.5.4 (#3098) v2.5.4
* Release files for 2.5.4
2025-02-03 17:55:39 +11:00
abraunegg
7f2d981683
Add './' sync_list rule check (#3097)
* Add './' sync_list rule check as this does not align to the documentation and these rules will not get matched correctly. The user needs to take corrective action.
2025-02-03 08:55:51 +11:00
abraunegg
ad22952167 Fix log output when generating a /delta response
* Fix log output when generating a /delta response
2025-02-03 08:14:03 +11:00
abraunegg
518b152c4c
Add code to support 'display_processing_time' use (#3095)
* Add code to support using 'display_processing_time' for functional performance
* Cleanup use of getFunctionName() so this is only called once and re-used
2025-02-02 08:18:15 +11:00
abraunegg
84a141b1d6
Calculate file transfer metrics in a consistent manner (#3091)
* Add back file transfer metrics which was available in v2.4.x (regression) 
* Calculate file transfer metrics in a consistent manner for all uploads and downloads
2025-01-29 06:38:21 +11:00
abraunegg
e392722b5e
Fix testInternetReachability function (#3087)
* Function should always return a boolean value and not throw an exception
2025-01-25 18:23:48 +11:00
abraunegg
defcd8bf2d
Update usage.md (#3084)
* Update usage.md
2025-01-24 20:15:31 +11:00
abraunegg
15656d31c8
Add OneDrive Personal Drive ID and Remote Drive ID length check (#3086)
* Add a check to ensure that OneDrive Personal Drive ID and Remote Drive ID values are 16 characters, padded by leading zeros if the provided JSON data has dropped these leading zeros
* If OneDrive Personal Account, validate all use of 'onedriveJSONItem["parentReference"]["driveId"].str' within application to ensure consistency in use and application
2025-01-24 19:59:09 +11:00
abraunegg
c60a889397
Fix --single-directory Shared Folder DB entry creation (#3082)
* Ensure that when using --single-directory that if the destination is a Shared Folder, the database records are created in a consistent manner
* Remove 'sea8cc6beffdb43d7976fbc7da445c639' due to 'non-alpha-in-dictionary' code scan result
2025-01-20 18:01:34 +11:00
abraunegg
4418d0d636
Support the moving of Shared Folder Links to other folders (#3052)
* Implement Feature Request #2824 to support the moving of Shared Folder Links to other folders
* Be consistent in log output for Personal Accounts, despite personal accounts not supporting relocatable Shared Folder links
* Update /delta generation message to include the driveId
* Code / Technical change was merged into https://github.com/abraunegg/onedrive/pull/3051 and committed to 'master' via 5a20154726
2025-01-20 10:48:09 +11:00
abraunegg
27a25bef4e
Fix Upload|Download Loop for AIP Protected File in Monitor Mode (#3071)
* Fix Upload|Download Loop for AIP Protected File in Monitor Mode
2025-01-18 07:46:56 +11:00
abraunegg
d0354fff94
Add account root id check for Microsoft OneDrive Personal account issue (#3077)
* Specifically add a check for the 'sea8cc6beffdb43d7976fbc7da445c639' string in the Microsoft OneDrive Personal Account Root ID response that denotes that the account cannot access Microsoft OneDrive at this point in time.
* Make sure the error gets flagged in the GUI so the user is aware of the problem
2025-01-17 12:17:18 +11:00
abraunegg
0247902d08 Update install.md
* Remove Ubuntu 23.04
* Add LTS designation for the Ubuntu Long Term Stuffed platforms
2025-01-17 09:53:14 +11:00
abraunegg
1d00a5228e Update install.md
* Add 'systemd-dev' due to Ubuntu 24.10 changes that now requires this package so that pkg-config can detect systemd
2025-01-15 07:00:15 +11:00
abraunegg
04a3ab32c0
Change dynamic linker to Linux only to correctly support FreeBSD and OpenBSD (#3069)
* Change dynamic linker to Linux only to correctly support FreeBSD and OpenBSD
* Add OpenBSD support to 'configure'
2025-01-13 19:37:38 +11:00
abraunegg
a402d1ea1d
Fix Alpine Docker build post FreeBSD support (#3068)
* Fix Alpine Docker build post FreeBSD support
2025-01-13 16:08:58 +11:00
abraunegg
260be9496a
Support FreeBSD - Code and documentation changes to support FreeBSD (#3048)
* Code and documentation changes to support FreeBSD
2025-01-13 15:32:18 +11:00
abraunegg
c7037f83dc
Fix issue #3057 where timestamp is not set when using --disable-download-validation (#3064)
* When --disable-download-validation is used, we still need to set the file timestamp correctly to avoid integrity checking issues
* Implement setFileTimestamp() as a common function so that when setting a file timestamp this is done in a consistent manner
* Update debug logging to be more agnostic to support either file or directory use
* Use same function for files and directories to ensure consistency
* Update function to detail what is being set, when it is set, and when it is successful
2025-01-12 20:11:27 +11:00
abraunegg
5a20154726
Support adding SharePoint Libraries as Shared Folder Links (#3051)
* Support adding SharePoint Libraries as Shared Folder Links
* Remove section of code that is no longer used
* Be consistent in log output for Personal Accounts, despite personal accounts not supporting relocatable Shared Folder links
* Update /delta generation message to include the driveId
* Ensure 'skip_dir' rules are correctly applied to remote shared folder links
2025-01-10 16:46:22 +11:00
abraunegg
71a71da1e0
Fix that the root Personal Shared Folder is not handled due to missing API data (#3047)
* Handle the 'root' folder of a OneDrive Personal Shared Folder in a more direct manner due to OneDrive API inconsistency in European Data Centres
2024-12-17 04:11:29 +11:00
abraunegg
c7af2186bd
Fix that item deleted after events of IN_MOVED_FROM & IN_CLOSE_WRITE (#3045)
* Update inotify logging output for deletion events to include the path that is being attempted to be deleted, to allow for easier event analysis of what is going on
* Fix path got deleted in handling of move & close_write event when using 'vim'. Refer to https://github.com/abraunegg/onedrive/issues/2586 and 2afbc17c12 (diff-6e533f0067)
2024-12-13 07:05:52 +11:00
abraunegg
77416841d4
Fix moving online files inside directories when using 'sync_list' (#3042)
* Fix issue that when using 'sync_list' if a file is moved to a newly created online folder, whilst the folder is created database wise, ensure this folder exists on local disk  to support included files from 'sync_list' entries
2024-12-13 05:17:30 +11:00
abraunegg
ea7ed66b21
Fix skip_dir logic when reverse traversing folder structure (#3034)
* Fix skip_dir logic when reverse traversing folder structure
* Update what path is used for logging output if skipped
2024-12-11 05:48:27 +11:00
abraunegg
6634284efa
Fix regression in handling uploading new and modified content to OneDrive Business and SharePoint (#3032)
* Fix regression from v2.4.x in handling uploading new and modified content to OneDrive Business and SharePoint to not create new versions of files post upload which adds to user quota
* Ad configuration option 'create_new_file_version' to create new versions if that is the desire
2024-12-09 14:24:33 +11:00
abraunegg
279057962c
Update Arch Linux instructions (#3036)
* Update Arch Linux instructions
* Re-order list
2024-12-08 14:32:40 +11:00
abraunegg
39377515a6
Fix regression of --display-config use (#3035)
* Fix regression of --display-config use after fast failing if --sync or --monitor has not been used
2024-12-07 17:42:29 +11:00
Ben
eade671215
correct minor typo (#3033)
I noticed this small typo when looking at a PR, hope you don't mind!
2024-12-07 09:57:17 +11:00
abraunegg
900ffa814f
Update docker.md (#3028)
* Fix details on docker compose process to clearly articulate that the client must be manually authenticated before running using docker compose file
2024-12-05 06:15:12 +11:00
abraunegg
53c74c4a63
Update --dry-run authentication message (#3022)
* Update --dry-run authentication message based on question raised in #3021
2024-12-02 10:59:09 +11:00
abraunegg
2ef6d4739d Update .gitignore
Update .gitignore to exclude all the spelling items
2024-11-30 09:11:53 +11:00
abraunegg
f7af328229
Update isValidUTF8 function with further validation (#3016)
* Update isValidUTF8 function to use 'validate' rather than individual character checking. The isValidUTF8 is only used to check a UTC timestamp in isValidUTCDateTime
* Enhance isValidUTF8 to check for empty strings, length constraints of UTC timestamps
* Add further logging in the event UTC validation fails
* Update allow.txt - add FFFD
2024-11-30 08:21:01 +11:00
Pieter Scheffers
c7f1b7f407
feat: optionally use standalone mode for syncing using Docker container (#3018)
* feat: optionally use standalone mode for syncing using Docker container

* Update podman.md
* Update podman.md

---------

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2024-11-29 06:29:30 +11:00
abraunegg
44d639cbe1
Fix download error of an updated file leads to online file deletion (#3017)
* Fix issue when downloading a file, and this fails due to an API error (400, 401, 5xx) - if the file existed in the database, the file will be deleted locally due to the download failure. Ensure that post this failure, we delete the relevant database record as per other download failures to avoid cascading an online deletion that should not occur.
2024-11-29 05:11:52 +11:00
abraunegg
b06b6238d8
Check for no --sync | --monitor earlier to fail fast if not used (#3011)
* Check for no --sync | --monitor earlier to fail fast here to avoid setting up all the other components, database, initialising the API as this is all pointless if we just fail out later
2024-11-27 07:50:28 +11:00
abraunegg
228e7db188
Implement Feature Request: Support Permanent Delete on OneDrive (#2999)
* Implement Feature Request: Support Permanent Delete on OneDrive (#2803)
2024-11-26 19:30:41 +11:00