Commit graph

260 commits

Author SHA1 Message Date
abraunegg ccd4dfbaf7 Resolve #341 '--single-directory' edge case
* Resolve where '--single-directory' is used to sync a single directory,
but a file / folder is 'moved' on OneDrive to outside the scope of the
focus of the '--single-directory' path. Before this change, the file
would remain in the local path (original location) whilst it would
reside in the new OneDrive path. This patch looks for this mis match and
deletes the local file / folder to reflect it is no longer in the same
OneDrive path.
2018-04-29 12:39:27 +10:00
abraunegg 6fe06d03ac Resolve 'Directory not empty' issues on bulk deletes
* In some cases OneDrive does not send all delete changes or changes are
out of order. When deleting local files, this can lead to a 'Directory
not empty' error. This change adds to check for any remaining children
of a directory that is to be deleted and delete them before
2018-04-26 09:45:18 +10:00
abraunegg 65a1181fcf Resolve std.exception.ErrnoException when user not part of users group
* Resolve: std.exception.ErrnoException@std/stdio.d(423): Cannot open
file `/var/log/onedrive/xxxxx.onedrive.log' in mode `a' (Permission
denied). This issue occurs if the user running 'onedrive' is not part of
the standard 'users' group (GID 100)
2018-04-25 12:11:34 +10:00
abraunegg bc15d7f77f Display informative message when not using --synchronize
* Prior to all the changes in #314, the client would sync regardless of
options. #314 introduced a flag that must be included if you want to
sync, however there was no feedback if '--synchronize' was not included
and the sync was not occuring. This patch provides that notification &
feedback.
2018-04-24 12:14:36 +10:00
abraunegg 034eccfaec Resolve #132 - Couldn't resolve host name on handle
* By default the DNS timeout on HTTP.method is too short. If DNS
resolution takes too long, a resolution error is thrown. Increase DNS
timeout to 5 seconds.
* Cleanup unittest - double declaration
2018-04-24 12:10:27 +10:00
abraunegg dff245d29b Resolve 'view.delta can only be called on the root'
* Resolve the error 'view.delta can only be called on the root' when
using OneDrive Business Account & --single-directory switch
* Change checkDatabaseForOneDriveRoot so that it is performed at init()
and not needed to be called a number of other times throughout the code
- leads to less HTTPS calls being made
* Cleanup comments that are no longer relevant due to
checkDatabaseForOneDriveRoot being run at init()
2018-04-23 10:58:47 +10:00
abraunegg 63f991a64e Reduce HTTPS calls for rootId & driveId by using init() set values
* Rather than make repeated HTTPS calls for the OneDrive rootId &
driveId, reuse the rootId & driveId as set via init() where possible
2018-04-21 16:30:35 +10:00
abraunegg 7a394a065f Resolve issue #73 - OneDrive Error 507 Insufficient Storage Exception
* Resolve https://github.com/skilion/onedrive/issues/73 by by tracking
the change in available free disk space when uploading a new file to
OneDrive
2018-04-21 14:54:50 +10:00
abraunegg 82ef982240 Configure at sync.init() global variables for driveId, rootId, accountType & space available
* Configure at sync.init() global variables for driveId, rootId,
accountType & space available
2018-04-21 14:32:39 +10:00
abraunegg 134c14e459 Resolve 'Key not found: lastModifiedDateTime' (OneDrive API change)
* Original patch (77c0cdbe24) whilst it
resolved the issue of a OneDrive API change, when a file was deleted on
OneDrive, it was not actually deleted on the local file system. This
patch re-implements the fix for 'Key not found: lastModifiedDateTime' by
taking into account the deleted state of the item and correctly removes
the local file that is now in the deleted state.
2018-04-21 14:03:02 +10:00
abraunegg 751d790631 Revert "Resolve 'Key not found: lastModifiedDateTime'"
This reverts commit 77c0cdbe24.
2018-04-21 07:32:31 +10:00
abraunegg 076b180e63 Revert "Manually revert lastModifiedDateTime"
This reverts commit 4a8456fa96.
2018-04-21 07:32:18 +10:00
abraunegg 4a8456fa96 Manually revert lastModifiedDateTime
* Manually revert lastModifiedDateTime
2018-04-21 07:24:19 +10:00
abraunegg 8a059dc0e1 Revert "Switch to using list_children for an item id's children"
This reverts commit fedaedacd7.
2018-04-21 07:19:04 +10:00
abraunegg 3d224d0595 Revert "Resolve issue #73 - 507 Insufficiente Storage Exception"
This reverts commit 648c1ef094.
2018-04-21 07:19:00 +10:00
abraunegg 894bdcde08 Revert "Fix itemdb.d assertion"
This reverts commit 66d00c5ec9.
2018-04-21 07:18:53 +10:00
abraunegg 095aa64cd6 Revert "Revert 77c0cdbe24 temporarily"
This reverts commit be03fa48df.
2018-04-21 07:18:46 +10:00
abraunegg be03fa48df Revert 77c0cdbe24 temporarily
* The fix in 77c0cdbe24 whilst resolves
'Key not found: lastModifiedDateTime' introduces a bug where files
deleted by onedrive, are not deleted locally. Reverting the change to
implement a fuller fix for 'Key not found: lastModifiedDateTime'.
2018-04-21 07:09:18 +10:00
abraunegg 66d00c5ec9 Fix itemdb.d assertion
* Fix core.exception.AssertError@src/itemdb.d(293): Assertion failure
introduced after onedrive.viewChildrenById(driveId, id) change
(fedaedacd7)
2018-04-20 18:41:49 +10:00
abraunegg 648c1ef094 Resolve issue #73 - 507 Insufficiente Storage Exception
* Resove https://github.com/skilion/onedrive/issues/73 by implementing a
check, initialised on init() to track the space used by uploads vs space
available
2018-04-19 16:14:42 +10:00
abraunegg fedaedacd7 Switch to using list_children for an item id's children
* Change how onedrive get's the children from a particular item id.
Previously view.delta was used to return the children, however
view.delta is not implemented on children when using OneDrive Business
accounts. By using list_children, we can get the children from any id
correctly when using either a 'personal' or 'business' account
2018-04-19 14:31:35 +10:00
abraunegg 77c0cdbe24 Resolve 'Key not found: lastModifiedDateTime'
* Fix 'Key not found: lastModifiedDateTime'
* Handling of Tombstoned items in the database
* Add for debugging purposes the rootID, driveID and Account Type
* Code cleanup - remove old commented out debugging items that are not
needed
2018-04-18 06:28:08 +10:00
abraunegg 835806f0eb Update "hidden" directory & skip_file handling
* Partial rollback of 570d42269e to
original logic behind isNameExcluded()
* Removed .* from default skip_file configuration
2018-04-16 21:39:45 +10:00
abraunegg 5cb92c01b9 Expand HTTPS debug to include GET JSON responses
* Change debug flag from --debug-http to --debug-https as OneDrive only
uses HTTPS, not HTTP
* Expand --debug-https to print the JSON responses to GET operations.
This is to assist with debugging #334 where a change in the MS API
(https://github.com/OneDrive/onedrive-api-docs/issues/834) no longer
sends 'lastModifiedDateTime' with responses to some GET's. This issue
seems to manefest itself with OneDrive Business accounts currently.
2018-04-15 20:02:39 +10:00
abraunegg 0febd67bd3 Add Microsoft naming conventions and namespace validation for items that will be uploaded
* Add Pull #104 (@JunpeiAnzai) to ensure that uploaded files conform to
Windows naming conventions and namespaces. Refer to
https://msdn.microsoft.com/en-us/library/aa365247 for further details
regarding Windows naming conventions and namespaces.
2018-04-13 11:20:38 +10:00
abraunegg 21f4d8e858 Add init.d service file & modify how configDirName is set
* Add init.d service file & helper script to start service
* Change how configDirName is set as XDG_CONFIG_HOME does not exist on
systems where X11 is not present. When using init scripts and using
XDG_CONFIG_HOME, ~ is not expanded thus existing config cannot be found.
Using ~ in --confdir also does not work when running under init.d
2018-04-13 09:33:16 +10:00
abraunegg 091e675d85 Update README.md based on current changes made
* Update readme
* Tweak logging output based on sync options selected
2018-04-12 10:18:18 +10:00
abraunegg 7eb11f0b44 Log why we are skipping a large file for upload
Log the reason why a file is not being uploaded (missed this in fec7654)
2018-04-12 08:44:52 +10:00
abraunegg fec7654118 Resolve Issues #121, #294, #329
Resolve onedrive crash when uploading files that exceed the allowed
OneDrive individual file limit
2018-04-12 07:31:42 +10:00
abraunegg 570d42269e Fix onedrive not syncing "hidden" directories
* Fix the handling of hidden filenames & directories (.hello, .git etc)
so that these are uploaded / downloaded without issue.
* Add verbose logging for uploaded files as to why they were skipped
2018-04-11 13:02:06 +10:00
abraunegg 5e0e763317 Change logging of upload, download and deletion activities
Prior to this change logging of upload, download and deletion activities
were only written to the log file when using the --verbose flag, however
they would be written to console / syslog. This corrects that behaviour.
2018-04-10 20:03:28 +10:00
abraunegg f1aa190a91 Add uploadOnly flag and add username to logfile name
* Add specific uploadOnly flag so that the onedrive client only uploads
data from the local directory and does not download changes from
OneDrive
* Change the logfile name to include the username running the onedrive
client
2018-04-07 17:06:57 +10:00
abraunegg 8463e60acc Update onedrive.d to remove '&'
Change '&error' to 'error' on line 37 of onedrive.d

Issue uncovered where prior code with '&' would compile without issue on
DMD reference compiler v2.078.3 and below, but using '2.079.0' this
would throw an error if '&' was present.

The original patch added the '&' to get the code to compile, this simple
reverts this change.

Note: For future reference compile this code with DMD v2.079.0 or later.
2018-03-30 08:14:00 +11:00
abraunegg dd73ae3c4b OneDrive Client Changes
* Fix 4xx errors including (412 pre-condition)
* Add Logging - log to a file (/var/log/onedrive/onedrive.log)
* Add http(s) debugging as a flag
* Add dont sync when just blindly running the application
* Add individual folder sync - ie ~/OneDrive/blah/ vs. syncing
everything in ~/OneDrive/
* Add sync from local directory first rather than download first then
upload
* Add upload long path check (430 character limitation)
2018-03-14 15:43:40 +11:00
skilion c231b13ad4 Disabled buffering on stdout 2018-02-18 18:24:46 +01:00
skilion 008a5756b3 Fix crash when the delta link is expired 2018-02-18 18:02:38 +01:00
skilion d9a5d1b3b9 fix regex for parsing authentication uri 2018-01-20 21:13:01 +01:00
Mikael Göransson a1e326f9b4 invalid_grant, error 70000
when trying to redeem an code, this error always occured, no matter
which parts of the response uri/url that was used.

the old regexp for parsing the code contained 3 groups, where as the
actual code always was the last group, and the second group was either ?
or &, and the first group would've been everything up until "code=".

changed from a matching group to character class so there would only be
two matching groups, so calling popFront() would actually leave the
authorization code in the front.
2018-01-20 20:51:33 +01:00
skilion 867c8ed26c improved regex 2018-01-19 18:56:14 +01:00
skilion 03f746a650 Grammar fix 2018-01-19 17:56:36 +01:00
skilion c7e09304a6 fix for https://github.com/skilion/onedrive/issues/269 2018-01-06 21:59:05 +01:00
skilion abad626b3d ensures that the parentId of the root is null 2018-01-06 19:27:27 +01:00
skilion bb3121e10d fix for https://github.com/skilion/onedrive/issues/269 2018-01-06 17:50:36 +01:00
skilion 9f7d9249bc restore prev redirect url 2018-01-02 16:32:16 +01:00
skilion 96c364c5fc removed tmp instruction 2018-01-02 15:06:20 +01:00
skilion d7f6674f60 removed redundant key parentDriveId 2018-01-02 15:05:32 +01:00
skilion 4ebc4a8544 handle move and delete of remote folders in monitor mode 2018-01-02 13:41:56 +01:00
skilion 9bedec0d7c handle moving items between drives 2018-01-01 18:38:08 +01:00
skilion c54d1927e9 do not throw in makeItem 2018-01-01 16:20:28 +01:00
skilion 764975e224 download only option 2017-12-31 17:11:17 +01:00
skilion 80cfdf62cf handle the case of deleting a remote folder locally 2017-12-31 16:56:56 +01:00
skilion fba3ed999e handle large uploads with parent id 2017-12-31 16:11:02 +01:00
skilion c8d5e03be8 upload files new with parent id 2017-12-31 13:47:18 +01:00
skilion 6907daa5e8 create dir with parent id 2017-12-31 13:18:11 +01:00
skilion 95c952fe62 handle deleting remote folders 2017-12-31 12:44:41 +01:00
skilion b7adc4d0cc WIP on remote folders 2017-12-31 02:30:31 +01:00
skilion b250214577 fix path handling of direct children of the root 2017-12-29 16:51:08 +01:00
skilion dd3e1b2861 fix wrong parameters in call 2017-12-29 12:24:26 +01:00
skilion 48559c592d fix handling root of remote item 2017-12-28 19:58:31 +01:00
skilion 90ab54cfc6 restore foreign keys and indices 2017-12-28 19:57:51 +01:00
skilion 8e91f57cb3 fix select children 2017-12-28 15:42:33 +01:00
skilion e3f011b724 write logs on stdout instead of stderr 2017-12-28 15:21:41 +01:00
skilion 35ce743b39 removed exception chaining 2017-12-28 15:03:15 +01:00
skilion 789ec85e0c partial remote items support 2017-12-27 15:13:28 +01:00
skilion d9c9915bc3 update API to use full item addressing 2017-12-27 15:12:38 +01:00
skilion 22db459ac4 fixed bug where remotely deleted and recreated directories would make the application crash 2017-08-01 19:20:02 +02:00
skilion 9765b27902 new parameter syncdir 2017-08-01 19:11:50 +02:00
skilion 7fae9c1bef versioning 2017-07-14 11:40:57 +02:00
skilion 849dd36276 check if the item is in the db before executing operations on it 2017-07-02 15:38:21 +02:00
lanhin 8f5f54f6a1 Merge branch 'master' of github.com:skilion/onedrive 2017-06-22 22:25:04 +08:00
skilion a61abc4e15 fix issue #202 2017-06-20 22:01:57 +02:00
skilion 1581c6989a fix error in QuickXorHash implementation 2017-06-16 11:40:07 +02:00
skilion 974be0884b fixed bug where an item is incorrectly assumed to be a file 2017-06-15 15:17:47 +02:00
skilion 126f2719b8 hack for onedrive biz 2017-06-15 14:19:23 +02:00
skilion 3529a4f78b make makeItem() more resiliant 2017-06-15 13:45:14 +02:00
skilion 9e2123249a do not check token status during initialization 2017-06-15 13:02:04 +02:00
skilion b703a824c7 compute path before performing selective sync 2017-06-15 12:59:33 +02:00
skilion 65bb14ad37 WIP for remote items support 2017-06-14 22:49:00 +02:00
skilion 7de0d97e6e fix error message 2017-06-14 22:30:29 +02:00
skilion 35f81b2115 itemdb support for remote items 2017-06-14 15:50:02 +02:00
skilion 74c931cdba added test for NULL values 2017-06-14 15:48:15 +02:00
skilion c5a9b8b480 fix bug where remote items are wrongly identified as deleted 2017-06-12 17:54:33 +02:00
skilion 4f542800a7 fix bug when using personalized config dir 2017-06-12 17:54:33 +02:00
skilion c2815d6498 fix indentation 2017-05-29 01:14:22 +02:00
skilion ad5441ccdc fix indentation 2017-05-29 00:23:55 +02:00
skilion b5cbaa13cc cleaned comments 2017-05-28 23:15:03 +02:00
skilion 8b6c99b0d9 notify deletion in verbose mode 2017-05-28 23:14:37 +02:00
skilion 305242d8a1 fix typo 2017-05-28 23:01:58 +02:00
skilion 809ccde562 workaround for old OneDrive files 2017-05-28 22:45:09 +02:00
skilion 8360d6508d align message 2017-05-28 22:17:21 +02:00
skilion 44fc36fd8d support for SHA1 and QuickXor hash algorithms 2017-05-28 22:13:19 +02:00
skilion 691862b18f implement the recommended way to enumerate changes 2017-05-28 20:54:57 +02:00
skilion 3d8daa086d added --print-token 2017-05-28 20:14:50 +02:00
skilion 80c60beef9 added newline at the end 2017-05-28 19:55:00 +02:00
skilion 7fec22bef1 fix variable type in for loop 2017-05-28 19:51:02 +02:00
skilion 9030a107b7 fix third person in comments 2017-05-28 19:49:55 +02:00
skilion f19b86e4c7 QuickXor utility function 2017-05-28 19:37:51 +02:00
skilion a2298be257 QuickXor implementation 2017-05-28 19:28:10 +02:00
lanhin 79139600ac Merge branch 'master' of github.com:skilion/onedrive
This may fix some up load problems.
2017-04-24 13:45:52 +08:00
skilion 97a9d53914 refactored selective sync code
fixed bug in selective sync
2017-03-24 22:31:56 +01:00