Commit graph

422 commits

Author SHA1 Message Date
abraunegg e2f1b68b1a Update sync.d
* Whenever the function is being returned, ensure that API instance is shutdown and destroyed
2024-01-22 08:25:51 +11:00
abraunegg e92947ea32 Update path validation
* Update check for ASCII control codes and specific non-ASCII control characters
* Add UTF-16 path validation check
2024-01-21 16:15:56 +11:00
abraunegg f2414f4a56 Update sync.d
* Update logging output for added clarity
2024-01-21 07:11:58 +11:00
abraunegg c8d29c8231 Add libcurl timeout on handle messaging
* Add libcurl timeout on handle messaging to use 'ip_protocol_version' to set what IP protocol to use. Testing has shown that when this is triggered, it is libcurl having an issue with IPv6 DNS resolution.
2024-01-20 07:41:53 +11:00
abraunegg 814d364c13 Update libcurl socket reuse handling
* Clarify CURLOPT_CONNECTTIMEOUT in documentation
* Clarify CURLOPT_TIMEOUT in documentation
* Add 'threads' as a config file option to control the number of threads used, default of 8, max of 16
* Change CURLOPT_CONNECTTIMEOUT value to v2.4.x value default of 10 seconds
* Configure libcurl to reuse connections
2024-01-20 06:15:33 +11:00
abraunegg 4e93777d7a Fix edge case file upload bugs when using --local-first
* Add PR 2527v2 to 'alpha-5': Fix edge case file upload bugs when using --local-first
2024-01-20 05:09:17 +11:00
abraunegg e470b21818 Update sync.d
* Add UTC note when detailing that there is a time difference
2024-01-19 09:08:50 +11:00
abraunegg a1f10cc5eb Update sync.d
* Add further timestamp qualifications
2024-01-18 18:56:22 +11:00
abraunegg e8406b719e Reinstate safeRename for online item moves
Reinstate safeRename for online item moves
2024-01-17 09:46:36 +11:00
abraunegg c328b029c5 Fix debug output
* Fix debug output
2024-01-17 09:04:42 +11:00
abraunegg b1a07a25f5 Update sync.d
* If timestamp needs to be corrected, return false so that the correct actions can be taken
* If in a --download-only scenario, do not update the timestamp online
2024-01-17 08:53:50 +11:00
abraunegg c5f2ec4622 Update sync.d
Fix reported bug where the online date is being changed in download-only mode
2024-01-16 05:19:25 +11:00
abraunegg 635102e8af Perform DB scan earlier and reuse result
* Remove potentially redundant applicability check of a path if this is already in the database. This is a manual merge of #2525
2024-01-14 07:17:25 +11:00
abraunegg 0ce088f0e2 Update 'root:' removal for 'skip_dir' path checking
* Update 'root:' removal for 'skip_dir' path checking
2024-01-13 16:03:43 +11:00
abraunegg d20b62da57 Fix 'config' file parsing and option handling
Fix that 'config' does not load when a deprecated option is at the beginning of the config file
2024-01-13 07:46:47 +11:00
abraunegg e4a6d62072 Update sync.d
* Cleanup array memory before we start adding files
2024-01-11 20:52:41 +11:00
abraunegg 34407b451f Update sync.d
* Just sync the triggered file, no scan of parent directory
2024-01-11 12:26:38 +11:00
abraunegg ca8c2d1bdc Fix zero byte file handling
* Fix zero byte file handling
2024-01-11 10:51:18 +11:00
abraunegg 48a803aa46 Initial commit of onedrive-v2.5.0-alpha-5
* Initial commit of onedrive-v2.5.0-alpha-5
2024-01-09 09:13:17 +11:00
abraunegg 43b0bed4cb
Fix debugging log output when reporting file size and hash mismatch (#2473)
* Fix debug logging output with correct hash values as these were switched
* Only calculate local file size post download once and reuse value
2023-08-24 18:08:21 +10:00
abraunegg 268dcf9120
Fix that zero-byte files do not have a hash as reported by the OneDrive API thus should not generate an error message (#2436)
* The OneDrive API does not present a hash for all files, most notably those that are zero byte in size (this may be fixed by the OneDrive API in the future). Add a wrapper to the existing makeItem function to test the file size before working out if this is a going to be a problem and if it is, then print out an error message if the file size is greater than 0 including either the full filename path or the items id.
2023-06-21 13:21:05 +10:00
abraunegg 06420c9a0a
Remove sha1 use and cleanup defunct remaining crc32 use (#2424)
* Remove sha1 from being used by the client as this is being depreciated by Microsoft in July 2023 - https://devblogs.microsoft.com/microsoft365dev/deprecation-of-sha1hash-on-onedrive-personal/
* Complete the removal of crc32 as this is also no longer present for a long time, but some code elements still existed
* Only compute quickXorHash, not quickXorHash and sha256Hash as computing sha256Hash is CPU expensive
* Update cache database stored items to only store quickXorHash and sha256Hash values (remove crc32 and sha1)
2023-06-20 06:55:00 +10:00
abraunegg 00ce7eed0e
Fix that folders are renamed despite using --dry-run (#2343)
* Fix that folders are renamed despite using --dry-run
2023-03-27 17:22:48 +11:00
abraunegg cfb4933778
Fix that a HTTP 504 - Gateway Timeout causes local files to be deleted when using --download-only & --cleanup-local-files mode (#2342)
* Fix that a HTTP 504 - Gateway Timeout causes local files to be deleted when using --download-only & --cleanup-local-files mode
2023-03-19 09:54:58 +11:00
abraunegg 678add91f8
Update --get-O365-drive-id error handling (#2266)
* Ensure a 'Personal' account type is not being used
* If the /sites?search API cannot be found, display a more appropriate error message
2022-12-16 06:08:46 +11:00
abraunegg 0b89a1fea0
Change order of Microsoft OneDrive restrictions and limitations check and skip_file|skip_dir (#2238)
* Perform check against Microsoft OneDrive restriction and limitations after user configured items to skip has been performed
2022-11-26 05:42:38 +11:00
abraunegg 5a7da61cbb
Fix issue of silent exit when unable to delete local files when using --cleanup-local-files (#2227)
* Add try blocks for performing local deletes to catch in function permission issues preventing file|folder local deletion.
2022-11-19 06:18:56 +11:00
abraunegg 0ec1c95e4a
Only call isPathExcludedViaSyncList if 'sync_list' is enabled where possible (#2213)
* Only check 'sync_list' if this has been enabled and configured, otherwise do not undertake a check as it is computationally redundant where possible
2022-11-10 16:58:54 +11:00
abraunegg 941e1e215d
Ensure path does not start with '/' before sync_list check (#2215)
* Ensure path does not start with '/' before sync_list check due to changes from 2211 and 2212
2022-11-10 14:28:20 +11:00
abraunegg 361c3cf0a1
Update Fix 2211 and 2212 (#2214)
* Update fixes for 2211 and 2212 due to undesirable path issue and output when application processing is actually correct
2022-11-10 13:43:36 +11:00
abraunegg 4a6c78a7a6
Only calculate path with computeItemPath if path is empty (#2212)
* Only calculate path with computeItemPath if path is empty, avoiding a double calculation and DB operations
2022-11-10 07:03:58 +11:00
abraunegg c73c0aa608
Fix application crash when local file is changed to a symbolic link with non-existent target (#2211)
* Add a try & catch block for testing if the file exists locally to catch any filesystem error that may be generated
* Test path to be valid if a symbolic link
2022-11-10 06:50:37 +11:00
abraunegg 7daf8797f8
Add performance output for key functions (#2180)
* Add developer option 'display_processing_time' to control if performance timing is outputted or not
* If option is enabled, print performance data around how long key functions are taking to process data to assist with understanding any performance related questions
2022-10-12 13:34:42 +11:00
abraunegg 514df625b9
Add option to create a read-write shareable link (#2154)
* Implement Feature Request 2110 to add option to create a read-write shareable link
2022-09-26 17:56:42 +10:00
abraunegg e2ed2d2b58
Update notification handling for specific skipped scenarios (#2153)
* Update skip notification handling for the following scenarios:
  * Invalid Name (Microsoft Naming Convention)
  * Invalid Name (Contains an invalid whitespace item)
  * Invalid Name (Contains HTML ASCII Code)
  * Invalid Item (Invalid symbolic link)
2022-09-26 07:40:22 +10:00
abraunegg 738be2d150
Add option to cleanup local files regardless of sync state when using --download-only (#2113)
* Add option to cleanup local files regardless of sync state when using --download-only
2022-08-31 06:41:52 +10:00
abraunegg d169dfc642
Improve application logging output for error messages (#2100)
* When enabling system logging to a log file, the actual ERROR line is forced to a new line in the application log. The reason for this is the \n prefix in the error message, which was in place so that when performing CLI logging or systemd logging, the error message would be displayed clearly. This change removes the \n from the actual error message, but inserts a newline before the error message is displayed (and also in some cases post error message) - thus keeping the application runtime look and feel, but improving the application log output.
2022-08-18 09:14:13 +10:00
abraunegg 9b8a25f34c
Fix that logging output for skipped uploaded files is missing (#2094)
* Add missing logfile output when enabling logging so that when uploaded new & modified files are skipped, this is correctly reflected in logfile output
2022-08-17 05:48:11 +10:00
abraunegg c82e90a140
Ensure that any active http instance is shutdown before any application exit (#2035)
* When the client needs to exit due to an issue, ensure that the curl http instance is shutdown before the exit is performed. This also potentially solves some segmentation faults seen on Ubuntu|Debian platforms due to issues in the shared library libphobos2-ldc-shared.so.X
2022-07-07 09:37:45 +10:00
Sebastien Badia 0ab2955bd7
Various little spelling fixes (check with lintian during Debian packaging) (#2027)
* contrib: remove bash hashbang from completion

This file starts with the #! sequence that marks interpreted scripts, but
it is a bash completion script that is merely intended to be sourced.

* src: spelling error (Attemtping => Attempting)

* src: spelling error (reponse => response)

* src: spelling error (sucessfully => successfully)
2022-07-02 05:14:50 +10:00
abraunegg ca984eba70
Update OneDrive API response handling for National Cloud Deployments (#2023)
* Update OneDrive API response handling for National Cloud Deployments
* Add developer option to allow easy switch between /children and /delta to query OneDrive for changes
2022-06-30 07:08:29 +10:00
abraunegg 042949f1c1
Fix 'foreign key constraint failed' when using OneDrive Business Shared Folders (#2017)
In some OneDrive Business scenarios, the shared folder /delta response lacks the 'root' drive details. When this occurs, this creates the following error: A database statement execution error occurred: foreign key constraint failed. Ensure we query independently the root details for this shared folder and ensure that it is added before we process the /delta response
2022-06-24 14:54:32 +10:00
abraunegg 1935def140
Update when DB is updated by OneDrive API data and update when file hash is required to be generated (#2004)
* Only update the database if the item modified time is different, otherwise we are needlessly updating the database with data that is the same
* Generating a file hash, during the file integrity check is computationally expensive. Only generate a file hash if the modified time of the file is different, otherwise it is pointless to generate the file hash during each integrity check
* Only flush SHM and WAL post integrity check
2022-06-15 09:16:06 +10:00
abraunegg 4e1deec933
Update Business Shared Folders to use a /delta query (#1989)
* Change Business Shared Folders to use a /delta query now that the OneDrive API appears to support such query
2022-06-05 14:52:24 +10:00
abraunegg 0fffb8fbc0
Implement a check to validate local filesystem available space before attempting file download (#1971)
* Implement a check to validate local filesystem available space before attempting file download
* Implement 'space_reservation' config option with a default value of 50 MB
* Prevent the original run-away logging error message 'Failed initialization on handle XXXX' from occurring if the system is out of space
* Update documentation and man page
2022-06-01 05:57:05 +10:00
Philipp 446be3743c
Fix logging output when download failed (#1894)
* Fix logging output when download failed
2022-04-29 16:54:55 +10:00
abraunegg 37bce6f82d
Update local folder creation timestamp with timestamp from OneDrive (#1934)
* When a new folder is created locally, use the lastModifiedDateTime as provided by OneDrive for the folders timestamp instead of the time when the client created the directory

Note: This only will impact new folder that are created, will not touch or modify existing folders
2022-04-26 08:23:18 +10:00
abraunegg 29b0ef0b98
Fix crash when trying to sync broken symbolic link (#1917)
* Check the response for directory checks as to if it generates a file exception error
2022-04-08 05:50:47 +10:00
abraunegg 3124b2dcf2
Implement feature request --modified-by (#1869)
* Implement --modified-by to display who last modified a file and when the modification was done
2022-03-10 07:00:07 +11:00
abraunegg d68c27fb4f
Add --reauth to allow easy re-authentication of the client (#1867)
* Add --reauth to allow easy re-authentication of the client
2022-03-08 05:35:00 +11:00