Commit graph

573 commits

Author SHA1 Message Date
abraunegg 88f78c163f
Fix application crash in --monitor mode due to 'Failed to stat file' when setgid is used on a directory (#1157)
* Add debugging output when adding paths and items to an inotify watch
* Only if a path is a directory, perform a path walk
* Catch Exception as error when permissions cause access exception
2020-11-26 06:35:20 +11:00
abraunegg 48fc64a247
Check application_id before setting redirect URL when using specific Azure endpoints (#1165)
* If using specific AD endpoints, but using the default application ID, the redirect URL does not match. Check if the application_id before setting the redirect URL.
2020-11-26 05:51:29 +11:00
abraunegg a4a5a5f267
Fix logging output when authentication authorisation fails (#1156)
* If OneDrive generates an auth error, dont log that the auth was sucessful
2020-11-21 06:46:05 +11:00
abraunegg ac4da2e2ba
Update #1142 logging output handling (#1153)
* Update logging output to debug rather than standard log
2020-11-20 06:33:47 +11:00
abraunegg 8c23efc6c5
Fix changing permissions on pre-existing local directories (#1152)
* When attempting to create local directories, test to determine if they exist locally first before creating & setting file system permissions
2020-11-20 05:36:20 +11:00
abraunegg 17d5f97cb0
Merge branch 'master' into fix-issue-1137 2020-11-14 06:21:03 +11:00
abraunegg 550ab4b6dd
Use config set option for 'remove_source_files' and 'skip_dir_strict_match' rather than ignore (#1142)
* Remove items from direct config string update, as they are configurable via config file, thus set option via config file is not used
* Add debugging and application config output for used options
2020-11-14 06:17:53 +11:00
abraunegg aa533ef5a4 Update sync.d
* tweak skip message output
2020-11-13 09:52:48 +11:00
abraunegg b59596523e Update sync.d
* Full fix
2020-11-13 08:27:09 +11:00
abraunegg b57bb3fde5 Update sync.d
* Remove this
2020-11-13 08:02:28 +11:00
abraunegg dd989b73d1 Update sync.d
* Flag that this download failed
2020-11-13 07:48:14 +11:00
abraunegg ac02b768d3 catch folder creation errors due to file system permissions error
* catch folder creation errors due to file system permissions error
2020-11-13 07:34:52 +11:00
abraunegg cc2b8085f5 Update error catching and handling
* Update error catching and handling
2020-11-13 05:42:24 +11:00
abraunegg 68cc51ab4d Update onedrive.d
* Remove try block - incompatible with auto designation needed to open file ...
2020-11-12 21:51:07 +11:00
abraunegg aa47f1119f Fix download failure due to incorrect filesystem permissions
* Add try block for opening new file for writing when downloading a new file to catch any read only file systems
* Move set file attributes to master function, incase there is a download failure, exit scope cannot set attributes on a file that is non existent
2020-11-12 21:35:06 +11:00
abraunegg 2c80033c89
Correctly set config directory permissions at first start (#1132)
* When ~/.config/onedrive/ gets created for the first time, directory permissions are not set, thus, effective permissions of 'd---------' get applied. This then causes issues attempting to update the 'refresh_token' as permission is denied.
* When permission is denied, the file exception error is not correctly handled
* Set refresh_token file permissions
2020-11-09 08:06:48 +11:00
mp1994 7a4abfff40
Check if localModifiedTime >= item.mtime: avoid re-upload (#1131)
* Check if localModifiedTime is >= than item.mtime: avoid re-upload for equal modified time

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2020-11-08 08:44:57 +11:00
abraunegg 57d6753f80
Merge contents of SQLite WAL file into main database file on sync completion (#1128)
* When using WAL and SHM files, certain thresholds are used to automatically determine when WAL and SHM data is committed to the main database file. If these thresholds are not met, the data is not written. Before terminating the program, commit the data, and at the end of each monitor loop, commit the data
* Only try and write data to DB file and destroy variable if it was set to begin with
* Only try and destroy 'onedrive' variable if set, rather than set to null
2020-11-06 10:28:15 +11:00
abraunegg d2193c97f5
Fix uploading data to shared business folders when using --upload-only (#1127)
* Handle uploading new data to a OneDrive Shared Business Folder when using --upload-only and if the shared folder has never been synced before
2020-11-05 05:19:39 +11:00
abraunegg b4c3da6e10
Check the given local path in reverse order for a skip_dir match on --resync (#1125)
* When matching skip_dir entries, if there is no direct first match, perform a reverse directory walk to test each path block for a match
2020-11-05 04:51:35 +11:00
abraunegg 363f64df36
Check for the presence of .nosync when processing DB entries (#1124)
* Add check for .nosync if enabled for directory database objects as these might also be relevant mount points which have 'gone' away
2020-11-04 07:28:40 +11:00
abraunegg 106e114929
Update --list-shared-folders to only show folders (#1114)
* Update --list-shared-folders to only show folders
2020-11-01 10:06:08 +11:00
abraunegg b365ec6322
Cleanup dryrun shm and wal files if they exist (#1121)
* Cleanup dryrun shm and wal files if they exist
2020-11-01 06:55:17 +11:00
abraunegg cf4ee5017f
Configure specific File and Folder Permissions (#1108)
* Configure specific File and Folder Permissions, defaults for new items downloaded from OneDrive:
Directories: 700
Files: 600
2020-10-30 08:00:26 +11:00
abraunegg fe86dd1809
Fix Business Shared Folder handling to process only Business Shared Folders (#1104)
* Update Business Shared Folder handling to process only Shared Folders, not individually shared files
2020-10-26 17:23:24 +11:00
abraunegg 9f60b95eed
Fix logging output for modification comparison source (#1103)
* Fix logging output for modification comparison source
2020-10-23 06:36:14 +11:00
sebastiaopf a3c4ba1b3b
Adding support for authentication w/ single tenant custom applications (#1094)
* Adding support for authentication with single tenant custom applications in OneDrive Business/Enterprise accounts.

When using a custom single tenant application the /common/ part on the authentication URLs needs to be replaced
by the tenant name. This can be configured on the azure_tenant_id setting, but this entry was being ignored unless
a custom azure_ad_endpoint was also set.

* Updated documentation to mention that, when configuring for Single Tenant Applications,
both the Tenant Name and Application ID must be included on the config file.

Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2020-10-10 15:39:04 +11:00
abraunegg b4328db2ef
Update debugging output for /delta changes available queries (#1089)
* Update debugging output for /delta changes available queries
2020-10-07 09:33:00 +11:00
themoox 2f35a33f4f
Allow config to specify a tenant id for non-multi-tenant applications (#1085)
* Allow config to specify a tenant id for non-multi-tenant applications
* Add instructions on how to find your O365 tenant identifier for National Cloud deployments.

Co-authored-by: Ryan Brown <ryan.brown@avarint.com>
Co-authored-by: abraunegg <alex.braunegg@gmail.com>
2020-10-06 08:06:04 +11:00
abraunegg e4d7821026
Handle deletion events for directories when inotify events are missing (#1082)
* Handle the missing inotify events for deleted items as a deletion event when running in --monitor
2020-10-03 09:21:56 +10:00
abraunegg e44e6b4233
Fix uninitialised value error as reported by valgrind (#1075)
* Fix uninitialised value error as reported by valgrind
2020-09-25 12:55:21 +10:00
abraunegg 7684c74600
Handle deletion events when inotify events are missing (#1073)
* Handle the missing inotify events for deleted items as a deletion event when running in --monitor
2020-09-25 08:22:37 +10:00
abraunegg 3604dc6984
Add notification to advise users when the client needs to be reauthenticated (#1046)
* Utilise errorAndNotify function to advise users when the client needs to be reauthenticated
2020-09-25 06:42:44 +10:00
abraunegg 19cf656c19
Implement Feature: Multiple skip_dir & skip_file configuration rules (#1068)
* Implement Feature: Multiple skip_dir & skip_file configuration rules
2020-09-19 07:51:49 +10:00
abraunegg 4809f91655
Implement Feature: Support negative patterns in sync_list for fine grained exclusions (#1065)
* Support negative patterns in sync_list where a new line is matched in a negative manner to be excluded is prepended with a '!' or '-' character to exclude an otherwise included path
2020-09-19 06:59:19 +10:00
abraunegg 1364f36aba
Support wildcard within sync_list entries (#1063)
* Support wildcard within sync_list entries
* Update usage documentation
2020-09-16 17:03:00 +10:00
abraunegg 0c770efa96
Implement Feature: Create shareable link (#1061)
* Implement feature request to create a shareable file link. Default is to create a read-only anonymous link

Co-authored-by: Norbert Preining <norbert@preining.info>
2020-09-14 17:49:50 +10:00
abraunegg 05fc734440
Handle removing child items on OneDrive when parent item responds with access denied (Issue #1041) (#1045)
* Handle removing child items on OneDrive when parent item responds with access denied due to parent folder permissions issue.
2020-09-02 06:23:41 +10:00
abraunegg fa3eb8bc1a
Remove 'Icon' check as no longer valid (#1038)
* Remove 'Icon' check as no longer listed as an invalid file or folder names in OneDrive API documentation
2020-08-28 06:18:56 +10:00
abraunegg 431678753d
Update --single-directory path handling when path does not exist locally (#1031)
* If the requested path for --single-directory does not exist locally, create it rather than display error and exit
2020-08-21 07:06:56 +10:00
abraunegg 2099ec2994
Flag when remaining free space is being restricted (#1029)
* In certain scenarios quota details are restricted, thus, remaining free space tracking is not possible. Flag this is the case rather than report a zero value.
2020-08-20 10:52:46 +10:00
abraunegg 880631982c
Update --single-directory handling of parent path matching (#1014)
* In certain scenarios when using --single-directory (either very first run or with --resync) the parent path for the selected path will not be in the database, thus, all items that should be synced are not. Add another special case handler to handle this scenario.
2020-08-09 15:48:27 +10:00
abraunegg 5907d9eb0a
Update how available free space is tracked when uploading files to OneDrive (#1011)
* Update how available free space is tracked when uploading files to OneDrive and Shared Folders
2020-08-08 08:56:00 +10:00
abraunegg 786446f79b
Fix 'Skipping uploading this new file as parent path is not in the database' when uploading to a Personal Shared Folder (#1009)
* Fix 'Skipping uploading this new file as parent path is not in the database' when uploading to a Personal Shared Folder
* When syncing OneDrive Personal Shared Folders, add the driveId to the driveIdArray
* Rename function to be more representative of what function does
2020-08-06 17:45:10 +10:00
abraunegg 2fe0574bcf
Update Office365 / SharePoint site search query (#1006)
* Update Office 365 / SharePoint site search query and response if query return zero match
2020-08-04 11:03:52 +10:00
abraunegg 928ad9512f
Check shared folder DB entries for matching skip_dir items (#1003)
* Check shared folder DB entries for matching skip_dir items
2020-07-28 07:00:34 +10:00
abraunegg 76ce1036a4
Update when 'Sync with OneDrive is ..' is outputted when in --monitor mode (#999)
* The original intention here was that 'Sync with OneDrive is ...' is outputted when the sync starts / ends in 'verbose' mode, and not added to when not using any 'verbose'. What is occurring is, this is being added when MONITOR_LOG_QUIET threshold is met, which was not the intention. This now fixes when this message is outputted so that it is more reflective of when a sync is being done, when in --monitor mode
2020-07-22 15:56:20 +10:00
abraunegg beb737e903
Fix segfault when moving file when using --monitor (#997)
* Fix segfault when attempting to perform a comparison on an inotify event when determining if event path is directory or file
2020-07-22 06:54:35 +10:00
Mateusz P-K f6002311cd
Improve --single-directory sync performance (#992)
* Remove erroneous 'return' statement which could prematurely end processing all changes returned from OneDrive
* Improve --single-directory sync performance

This commit modifies code inside applyDifferences function,
the part responsible for deleting changes/moving them out
of the selected sync directories. This change makes an HTTP
request responsible for checking whether a changed item still
exists on OneDrive only be sent only if it may in fact influence
whether an item will be deleted from the synced folder or not
or just discarded.

This makes an enormous performance boost, because it limits
redundant HTTP requests that ask about changed items that will
be discarded or not.

Signed-off-by: Mateusz P-K <mateusz.kaplon@gmail.com>
2020-07-21 15:19:41 +10:00
abraunegg 25fd9a1b00
Update the 'Processing XXXX' output to display the full path (#987)
* Update the 'Processing XXXX' output to display the full path, rather than just the item name from the database. In cases where you have the same file or directory name, having the full path where it is present is more informative
2020-07-07 19:51:05 +10:00