Commit graph

332 commits

Author SHA1 Message Date
abraunegg 3b18638a63
Remove duplicate code for error output functions and enhance error logging output (#1170)
* Remove duplicate code for error output functions and enhance error logging output
2020-12-09 14:18:16 +11:00
abraunegg 335eab326d
Handle case where API provided deltaLink generates a further API error (#1175)
* Handle case where API provided deltaLink generates a further API error. Refer to: https://github.com/OneDrive/onedrive-api-docs/issues/1408
2020-12-09 14:03:49 +11:00
abraunegg 88f78c163f
Fix application crash in --monitor mode due to 'Failed to stat file' when setgid is used on a directory (#1157)
* Add debugging output when adding paths and items to an inotify watch
* Only if a path is a directory, perform a path walk
* Catch Exception as error when permissions cause access exception
2020-11-26 06:35:20 +11:00
abraunegg 8c23efc6c5
Fix changing permissions on pre-existing local directories (#1152)
* When attempting to create local directories, test to determine if they exist locally first before creating & setting file system permissions
2020-11-20 05:36:20 +11:00
abraunegg aa533ef5a4 Update sync.d
* tweak skip message output
2020-11-13 09:52:48 +11:00
abraunegg b59596523e Update sync.d
* Full fix
2020-11-13 08:27:09 +11:00
abraunegg b57bb3fde5 Update sync.d
* Remove this
2020-11-13 08:02:28 +11:00
abraunegg dd989b73d1 Update sync.d
* Flag that this download failed
2020-11-13 07:48:14 +11:00
abraunegg ac02b768d3 catch folder creation errors due to file system permissions error
* catch folder creation errors due to file system permissions error
2020-11-13 07:34:52 +11:00
abraunegg cc2b8085f5 Update error catching and handling
* Update error catching and handling
2020-11-13 05:42:24 +11:00
mp1994 7a4abfff40
Check if localModifiedTime >= item.mtime: avoid re-upload (#1131)
* Check if localModifiedTime is >= than item.mtime: avoid re-upload for equal modified time

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2020-11-08 08:44:57 +11:00
abraunegg d2193c97f5
Fix uploading data to shared business folders when using --upload-only (#1127)
* Handle uploading new data to a OneDrive Shared Business Folder when using --upload-only and if the shared folder has never been synced before
2020-11-05 05:19:39 +11:00
abraunegg 363f64df36
Check for the presence of .nosync when processing DB entries (#1124)
* Add check for .nosync if enabled for directory database objects as these might also be relevant mount points which have 'gone' away
2020-11-04 07:28:40 +11:00
abraunegg 106e114929
Update --list-shared-folders to only show folders (#1114)
* Update --list-shared-folders to only show folders
2020-11-01 10:06:08 +11:00
abraunegg cf4ee5017f
Configure specific File and Folder Permissions (#1108)
* Configure specific File and Folder Permissions, defaults for new items downloaded from OneDrive:
Directories: 700
Files: 600
2020-10-30 08:00:26 +11:00
abraunegg fe86dd1809
Fix Business Shared Folder handling to process only Business Shared Folders (#1104)
* Update Business Shared Folder handling to process only Shared Folders, not individually shared files
2020-10-26 17:23:24 +11:00
abraunegg 9f60b95eed
Fix logging output for modification comparison source (#1103)
* Fix logging output for modification comparison source
2020-10-23 06:36:14 +11:00
abraunegg b4328db2ef
Update debugging output for /delta changes available queries (#1089)
* Update debugging output for /delta changes available queries
2020-10-07 09:33:00 +11:00
themoox 2f35a33f4f
Allow config to specify a tenant id for non-multi-tenant applications (#1085)
* Allow config to specify a tenant id for non-multi-tenant applications
* Add instructions on how to find your O365 tenant identifier for National Cloud deployments.

Co-authored-by: Ryan Brown <ryan.brown@avarint.com>
Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2020-10-06 08:06:04 +11:00
abraunegg e4d7821026
Handle deletion events for directories when inotify events are missing (#1082)
* Handle the missing inotify events for deleted items as a deletion event when running in --monitor
2020-10-03 09:21:56 +10:00
abraunegg 7684c74600
Handle deletion events when inotify events are missing (#1073)
* Handle the missing inotify events for deleted items as a deletion event when running in --monitor
2020-09-25 08:22:37 +10:00
abraunegg 3604dc6984
Add notification to advise users when the client needs to be reauthenticated (#1046)
* Utilise errorAndNotify function to advise users when the client needs to be reauthenticated
2020-09-25 06:42:44 +10:00
abraunegg 0c770efa96
Implement Feature: Create shareable link (#1061)
* Implement feature request to create a shareable file link. Default is to create a read-only anonymous link

Co-authored-by: Norbert Preining <norbert@preining.info>
2020-09-14 17:49:50 +10:00
abraunegg 05fc734440
Handle removing child items on OneDrive when parent item responds with access denied (Issue #1041) (#1045)
* Handle removing child items on OneDrive when parent item responds with access denied due to parent folder permissions issue.
2020-09-02 06:23:41 +10:00
abraunegg 2099ec2994
Flag when remaining free space is being restricted (#1029)
* In certain scenarios quota details are restricted, thus, remaining free space tracking is not possible. Flag this is the case rather than report a zero value.
2020-08-20 10:52:46 +10:00
abraunegg 880631982c
Update --single-directory handling of parent path matching (#1014)
* In certain scenarios when using --single-directory (either very first run or with --resync) the parent path for the selected path will not be in the database, thus, all items that should be synced are not. Add another special case handler to handle this scenario.
2020-08-09 15:48:27 +10:00
abraunegg 5907d9eb0a
Update how available free space is tracked when uploading files to OneDrive (#1011)
* Update how available free space is tracked when uploading files to OneDrive and Shared Folders
2020-08-08 08:56:00 +10:00
abraunegg 786446f79b
Fix 'Skipping uploading this new file as parent path is not in the database' when uploading to a Personal Shared Folder (#1009)
* Fix 'Skipping uploading this new file as parent path is not in the database' when uploading to a Personal Shared Folder
* When syncing OneDrive Personal Shared Folders, add the driveId to the driveIdArray
* Rename function to be more representative of what function does
2020-08-06 17:45:10 +10:00
abraunegg 2fe0574bcf
Update Office365 / SharePoint site search query (#1006)
* Update Office 365 / SharePoint site search query and response if query return zero match
2020-08-04 11:03:52 +10:00
abraunegg 928ad9512f
Check shared folder DB entries for matching skip_dir items (#1003)
* Check shared folder DB entries for matching skip_dir items
2020-07-28 07:00:34 +10:00
Mateusz P-K f6002311cd
Improve --single-directory sync performance (#992)
* Remove erroneous 'return' statement which could prematurely end processing all changes returned from OneDrive
* Improve --single-directory sync performance

This commit modifies code inside applyDifferences function,
the part responsible for deleting changes/moving them out
of the selected sync directories. This change makes an HTTP
request responsible for checking whether a changed item still
exists on OneDrive only be sent only if it may in fact influence
whether an item will be deleted from the synced folder or not
or just discarded.

This makes an enormous performance boost, because it limits
redundant HTTP requests that ask about changed items that will
be discarded or not.

Signed-off-by: Mateusz P-K <mateusz.kaplon@gmail.com>
2020-07-21 15:19:41 +10:00
abraunegg 25fd9a1b00
Update the 'Processing XXXX' output to display the full path (#987)
* Update the 'Processing XXXX' output to display the full path, rather than just the item name from the database. In cases where you have the same file or directory name, having the full path where it is present is more informative
2020-07-07 19:51:05 +10:00
abraunegg 810c47fcfe
Increase maximum file size to 100GB (#989)
* In July 2020, Microsoft increased the maximum file size for uploads from 15GB / 20GB for Personal / Business Accounts to 100GB per file for all account types
2020-07-07 19:19:50 +10:00
abraunegg 8c0d43183d
Resolve client 'hang' when attempting to sync a Unix pipe file (#985)
* Resolve client 'hang' when attempting to sync a unix pipe file
2020-07-07 18:05:36 +10:00
abraunegg e23e06305a
Update 'skip_dir' and 'skip_file' pattern matching and handling (Issue #982) (#983)
* Update 'skip_dir' and 'skip_file' pattern matching and handling
2020-07-07 05:17:03 +10:00
abraunegg 41d92ddab5
Process OneDrive Personal Shared Folders only if account type is 'personal' (#972)
* Update code to ensure that OneDrive Shared Folders are processed, only if account type is 'personal'
2020-06-28 10:57:21 +10:00
abraunegg 9cc72c2396
Implement OneDrive Business Shared Folders Support (Issue #459) (#473)
* Implement OneDrive Business Shared Folders Support
2020-06-27 19:10:37 +10:00
abraunegg 1c10effb9b
Implement support for National cloud deployments (Issue #937) (#938)
* Implement support for National cloud deployments as per https://docs.microsoft.com/en-us/graph/deployments
2020-06-17 07:57:14 +10:00
Christian Ponte-Fernández 3b85d574e0
Implement recursive deletion when Retention Policy is enabled (#955)
* Implement recursive deletion when Retention Policy is enabled
2020-06-17 06:49:43 +10:00
abraunegg e321c373fd
Handle a 'race' condition to process inotify events (#948)
* Handle a 'race' condition to process inotify events generated whilst performing DB or filesystem walk
2020-06-12 06:46:59 +10:00
abraunegg df361b261b
Update message output for performFullItemScan or oneDriveFullScanTrigger (#945)
* Update message output depending on if performFullItemScan or oneDriveFullScanTrigger was set
2020-06-05 06:45:19 +10:00
abraunegg 8376a360cc
Check if symbolic link is relative to location path (#942)
* If a symbolic link is 'relative', it is relative to the path where the symbolic link exists, not the current working directory. Test if the failed symbolic link read is due to the link being relative, and advise accordingly.
2020-06-02 13:46:57 +10:00
abraunegg f0c5ca95c6
Update logging output of skipped files (#932)
* Normalise output when specifying a particular file which is being skipped
2020-05-26 19:53:35 +10:00
abraunegg aee2052f1e
Strengthen data preservation checks & update how bypass preservation is set (#931)
* Strengthen bypass data preservation variable checks
* If a file is modified locally, and '--resync' is used, there is no way to tell if this file is a failed download or 'valid' thus potentially lead to a data loss scenario. In this case, and ONLY if --resync was issued, rename the local file for data preservation
2020-05-26 17:04:22 +10:00
abraunegg 0096e7efce
Add file validation for moved / renamed files (#929)
* When renaming a file locally, ensure that the target filename is valid before attempting to upload to OneDrive
2020-05-25 10:51:57 +10:00
abraunegg fc90ccb425
Implement 'bypass_data_preservation' enhancement (Issue #824) (#831)
* On some Linux distributions, the file system search tool locally modifies certain files after indexing. Even though the file contents has not changed, the file itself has, as the local modified timestamp has been updated. This then causes timestamp checks to be invalid. To ignore this in cases where this is occurring, configure 'bypass_data_preservation' to 'true' in the config file and local data protection rules will be ignored.
* Add warning to application startup if 'bypass_data_preservation' has been enabled
2020-05-21 06:48:02 +10:00
abraunegg 206ab8f516
Resolve issues identified by Valgrind (#910)
* Resolve issues identified by 'valgrind' where possible
2020-05-20 11:37:11 +10:00
abraunegg 2529b99265
Update enforce statement for parent id database query (Issue #912) (#913)
* Update database enforce check with conditional check on parent drive ID. If the parent is a shared folder, the parent ID will never be in the database as we are never provided that parent ID.
2020-05-10 05:42:46 +10:00
abraunegg cb06395a46
Catch unhandled MonitorException when inotify throws an error (Issue #903) (#905)
* Catch uphandled MonitorException when inotify throws an error
* Change monitor loop init full scan to false at start as fullScanOverride now correctly handled, negating need to true-up at application start
* Handle '100 Continue' response during upload
* Update handling of delta link being expired
* Update progress bar handling for uploads as #888 changed bar dynamics
2020-05-06 07:20:13 +10:00
abraunegg eda3d3c4bb
Better handle out-of-space message (#907)
* Better handle out-of-space message when OneDrive is out of space
2020-05-05 15:04:59 +10:00