Commit graph

105 commits

Author SHA1 Message Date
Norbert Preining aa2e2245f0 Support file based authorization (Issue: #524) (#525)
* support file based authorization
* remove generated files after authentication
2019-06-11 17:12:36 +10:00
abraunegg 20eda60fdf
Catch a curl timeout exception (#499)
* Catch a curl timeout exception
* If we hit a curl exception, log that we hit this, as the handling of the exception in 'some' instances may actually not log an error as to 'why'
2019-05-13 09:10:19 -04:00
Norbert Preining ea22d8fef5 config options for command line switches (#449)
* config options for command line switches to allow for better config handling in docker containers
2019-04-11 12:26:20 +10:00
abraunegg 7c2b3e853d
Implement more meaningful 5xx error responses (Issue #445) (#451)
* Issue #445 and https://github.com/OneDrive/onedrive-api-docs/issues/1035 highlight that the current 5xx error messages are too generic and need to be more specific. This patch makes those error messages more specific to the response code from OneDrive API
2019-04-04 17:13:15 +11:00
abraunegg 6a6b8f128f
Issue #432 and #441 combined (#447)
* Re-add original #432 and #441 fixes to master
2019-04-02 05:21:02 +11:00
abraunegg abb82868e4
Roll back to v2.3.1 (#446)
* Roll back to v2.3.1 after bad re-base merge with PR #389
2019-04-02 04:51:25 +11:00
abraunegg f38b13dd00
Resolve file creation loop when working directly in the synced folder with libreoffice (#442)
* Add debug line for flagging to disable upload validation checks
* Handle Microsoft 'enrichment' of certain documents when stored on Sharepoint
* Handle skip_dir checks when nothing to check against (false positive)
* Update default 'skip_file' to include tmp and lock files generated by LibreOffice
* Update code comments & logging output
* Update readme & default config file
* Update database version due to changing defaults of 'skip_file' which will force a rebuild and use of new skip_file default regex
2019-04-02 04:00:22 +11:00
Norbert Preining 362bf35688 Merge branch 'master' into norbert/config-options 2019-03-17 12:25:27 +09:00
abraunegg 4c3b959bf6
Resolve unhandled application crash when invalid auth response is used (Issue #399) (#410)
* Add a http 400 response error handler
* If the response uri generates a 400 error, the JSON response will not contain the access_token. Request to re-authenticate
2019-03-12 11:50:21 +11:00
abraunegg 98624267c5
Implement --dry-run (#337)
* Implement new feature --dry-run
2019-03-11 17:57:47 +11:00
Norbert Preining a635cab0ad config options: working version 2019-02-22 16:15:10 +09:00
abraunegg 1fa7e5f20c
Add extra debugging & http protocol downgrade (Issue #314) (#327)
* Add --force-http-1.1 flag to downgrade any HTTP/2 curl operations to HTTP 1.1 protocol
* Explicitly set all bool items to false at initialisation
* Update --display-config to display sync_list if configured
* Add debug handling to display when sync_list is loaded
* Add debug handling to output the handling of OneDrive changes
2019-01-06 05:43:44 +11:00
abraunegg 46ef8ed376 Revert "Update handling of HTTP 412 - Precondition Failed errors (#issue 325) (#326)"
This reverts commit 388cc1ded5.
2019-01-05 17:18:28 +11:00
abraunegg 388cc1ded5
Update handling of HTTP 412 - Precondition Failed errors (#issue 325) (#326)
* Update HTTP 412 - Precondition Failed messaging
* Add debug messaging when 5xx errors occur
* Update 5xx message to include
2019-01-05 07:57:10 +11:00
abraunegg 2553366a89
Implement Feature Request: Add status command or switch (Issue #112) (#307)
* Implement Feature Request: Add status command or switch
2018-12-28 12:26:03 +11:00
Norbert Preining cc6cbf5ac7 Notification support via libnotify (#270)
* add external sources for (d)notify with README and license statements
* add new logAndNotify function, make building a compile time option
* use logAndNotify and main.d, make timeout message silent
* update documentation for notification support
* add command line switch --disable-notifications
* add build-deps for libnotify in README.md
2018-12-06 06:19:00 +11:00
abraunegg 29808c905c
Implement --get-O365-drive-id to get correct SharePoint Shared Library (Issue #248) (#260)
* Implement --get-O365-drive-id to return the correct drive_id of a SharePoint Shared Library to use for syncing that repository
2018-12-04 10:59:23 +11:00
abraunegg cdb11326b9
Fix unreachable statements (#268)
* Add warning flag to compiler
* Fix unreachable statements

**Credit:** Zamir SUN <sztsian@gmail.com>
2018-12-04 06:29:18 +11:00
Norbert Preining 9817104ca1 continue in monitor mode when sync timed out (#265)
* update handling of connection timeouts in monitor mode & allow application continue rather than exit
2018-12-03 13:41:31 +11:00
Norbert Preining c24e4b0b1e adjust timeout values for libcurl (issue #256) (#261)
This change adjusts the two core timeout parameters of libcurl operation
- dataTimeout: changed from 3600sec to 300sec
  this timeout controls the max time when there is no data incoming
  (actually below CURLOPT_LOW_SPEED_LIMIT)
- operationTimeout: added 3600sec
  this timeout controls the maximally allowed connection timeout
2018-12-02 11:30:50 +11:00
abraunegg a26a69ce9f
Change logging to a separate logfile is no longer the default (#239)
* Change '--download' to '--download-only' to align with '--upload-only'
* Enable logging to a separate file only if the '--enable-logging' flag is passed through at client run time
* Implement configuration option for logfile location, if logging is enabled
2018-11-24 07:13:16 +11:00
abraunegg c5ee62efd8
Sync with shared folders/drives for onedrive business (#206)
* Refactor PR104 for latest code base to enable client use with SharePoint libraries
* Implement '--disable-upload-validation' as SharePoint modifies files when uploaded
* Enable '--disable-upload-validation' by default if account type is documentLibrary - as most likely this is a SharePoint repository
2018-11-24 06:26:30 +11:00
abraunegg eaedfac34a
Update onedrive.d for additional debug logging (#238)
* In case the user does not use the --verbose tag, if we are doing --https-debug we want this printed to the console
2018-11-18 07:05:01 +11:00
abraunegg e70722862c
update files for 2.1.6 release (#235)
* update files for 2.1.6 release
2018-11-15 06:43:58 +11:00
abraunegg f785396643
Update HTTP/2 handling for session uploads (#233)
* Remove HTTP 1.1 downgrade for session uploads
* Handle HTTP/2 0 (zero) response code for successful session data upload
* Add debugging for actual server response, not curl interpreted response when using --debug-https
2018-11-15 06:08:55 +11:00
abraunegg 8a98bf3e30
Add handling for HTTP Code 302 generated by HTTP/2 calls (#229)
* Update initial Curl 7.62.0 fix with a better solution which handles the redirect's given by HTTP/2 connections (credit @Popa21)
* Use HTTP/1.1 for session uploads otherwise the response when using HTTP/2 generates a 'JSONValue is not an object' error
2018-11-12 06:01:57 +11:00
abraunegg 11e477f045
Resolve files will not download when using curl 7.62.0 (http2 issue) (#225)
* Force connections to use HTTP 1.1 as Curl 7.62.0 defaults to using http2 if h2 support is built in which causes issues
2018-11-11 06:38:41 +11:00
abraunegg 4919a58246
Resolve File download fails if the file is marked as malware in OneDrive (Issue #153) (#155)
* Gracefully handle the situation where OneDrive has marked a file as malware and will not provide the file for download
2018-09-13 08:41:46 +10:00
abraunegg b9faa5cd1f
Resolve HTTP request returned status code 429 (Too Many Requests) (Issue #133) (#138)
* Adding a retry method for downloading files when a HTTP request returned status code 429 (Too Many Requests) response is returned from OneDrive
2018-08-27 10:45:26 +10:00
abraunegg 8031f9491a
Resolve Gateway Timeout - JSONValue is not an object (Issue #127) (#129)
* Resolve Gateway Timeout - JSONValue is not an object (Issue #127)
2018-08-27 10:35:15 +10:00
abraunegg bed2b6c75f
Implement Feature Request: Add a progress bar for large uploads & downloads (Issue #12) (#99)
* Implement feature request to add a progress bar for large file uploads & downloads
2018-08-10 07:46:27 +10:00
abraunegg 91d8b7ab93
Gracefully handle a timeout when accessing the Microsoft OneDrive Service (Issue #82) (#86)
* Gracefully handle a timeout when accessing the Microsoft OneDrive Service
* Update HTTP connection handling for long running operations based on https://forum.dlang.org/post/k2vbk5$re7$1@digitalmars.com & https://github.com/dlang/phobos/pull/797
2018-08-02 08:25:37 +10:00
abraunegg d0209d8dbc
Gracefully handle OneDrive account password change (#73)
* Gracefully handle OneDrive account password change rather than crashing if authorization token is now invalid
2018-07-16 09:58:36 +10:00
abraunegg a4e055fdc3
Update 'HTTP 412 - Precondition Failed' error handling (#67)
* Update 'HTTP 412 - Precondition Failed' error handling based on local testing to ensure a http 412 response is gracefully handled & the modified file is subsequently uploaded to OneDrive successfully.
2018-07-16 06:36:31 +10:00
abraunegg 83c28ba3b6
Fix: Multiple versions of file shown on website after single upload (Issue #2) (#40)
* Resolve multiple versions of file shown on website after single upload - however Issue #23 & OneDrive API bug (https://github.com/OneDrive/onedrive-api-docs/issues/877) will still create 2 file versions on OneDrive Business. The work around is to disable file versions until #877 is resolved.
2018-07-03 07:24:57 +10:00
abraunegg 17e5138b5a
Resolve sync issues with Shared Folders & Personal OneDrive Accounts (#26)
* Resolve Shared Folders not syncing (OneDrive Personal)
* Resolve 'Key not found: fileSystemInfo' when then item is a remote item (OneDrive Personal)
* Resolve where file names are being truncated by 5 characters with each sync (OneDrive Business)
* remove extra // not required
* Remove commented out code to avoid confusion
2018-06-17 12:02:58 +10:00
abraunegg d6b4930d0a Revert "Resolve: Key not found: fileSystemInfo when syncing shared folders"
This reverts commit e7267e597b.
2018-06-13 16:35:52 +10:00
abraunegg e7267e597b Resolve: Key not found: fileSystemInfo when syncing shared folders
* Resolve #11 where shared folders were unable to be sync'd due to fileSystemInfo data being within the remoteItem object
* Initial work on resolving #2, but fix not validated or complete
2018-06-09 21:43:39 +10:00
abraunegg 41976ed216 Update long path handling
* Update long path handling to account for differences between OneDrive Personal and OneDrive Business Accounts
* Update logging to be cleaner on fragment uploads
2018-05-09 06:47:03 +10:00
abraunegg 0b5dc66507 Rework OneDrive generating a 412 'Precondition Failed' Error
* Rework the 412 fix so that the OneDrive client gracefully handles the
error & retries the metadata update without the eTag/cTag causing the
issue.
* Fix the double logging to console of 'upload' and 'download' items
2018-05-03 16:21:53 +10:00
abraunegg 034eccfaec Resolve #132 - Couldn't resolve host name on handle
* By default the DNS timeout on HTTP.method is too short. If DNS
resolution takes too long, a resolution error is thrown. Increase DNS
timeout to 5 seconds.
* Cleanup unittest - double declaration
2018-04-24 12:10:27 +10:00
abraunegg dff245d29b Resolve 'view.delta can only be called on the root'
* Resolve the error 'view.delta can only be called on the root' when
using OneDrive Business Account & --single-directory switch
* Change checkDatabaseForOneDriveRoot so that it is performed at init()
and not needed to be called a number of other times throughout the code
- leads to less HTTPS calls being made
* Cleanup comments that are no longer relevant due to
checkDatabaseForOneDriveRoot being run at init()
2018-04-23 10:58:47 +10:00
abraunegg 8a059dc0e1 Revert "Switch to using list_children for an item id's children"
This reverts commit fedaedacd7.
2018-04-21 07:19:04 +10:00
abraunegg fedaedacd7 Switch to using list_children for an item id's children
* Change how onedrive get's the children from a particular item id.
Previously view.delta was used to return the children, however
view.delta is not implemented on children when using OneDrive Business
accounts. By using list_children, we can get the children from any id
correctly when using either a 'personal' or 'business' account
2018-04-19 14:31:35 +10:00
abraunegg 5cb92c01b9 Expand HTTPS debug to include GET JSON responses
* Change debug flag from --debug-http to --debug-https as OneDrive only
uses HTTPS, not HTTP
* Expand --debug-https to print the JSON responses to GET operations.
This is to assist with debugging #334 where a change in the MS API
(https://github.com/OneDrive/onedrive-api-docs/issues/834) no longer
sends 'lastModifiedDateTime' with responses to some GET's. This issue
seems to manefest itself with OneDrive Business accounts currently.
2018-04-15 20:02:39 +10:00
abraunegg 8463e60acc Update onedrive.d to remove '&'
Change '&error' to 'error' on line 37 of onedrive.d

Issue uncovered where prior code with '&' would compile without issue on
DMD reference compiler v2.078.3 and below, but using '2.079.0' this
would throw an error if '&' was present.

The original patch added the '&' to get the code to compile, this simple
reverts this change.

Note: For future reference compile this code with DMD v2.079.0 or later.
2018-03-30 08:14:00 +11:00
abraunegg dd73ae3c4b OneDrive Client Changes
* Fix 4xx errors including (412 pre-condition)
* Add Logging - log to a file (/var/log/onedrive/onedrive.log)
* Add http(s) debugging as a flag
* Add dont sync when just blindly running the application
* Add individual folder sync - ie ~/OneDrive/blah/ vs. syncing
everything in ~/OneDrive/
* Add sync from local directory first rather than download first then
upload
* Add upload long path check (430 character limitation)
2018-03-14 15:43:40 +11:00
skilion d9a5d1b3b9 fix regex for parsing authentication uri 2018-01-20 21:13:01 +01:00
Mikael Göransson a1e326f9b4 invalid_grant, error 70000
when trying to redeem an code, this error always occured, no matter
which parts of the response uri/url that was used.

the old regexp for parsing the code contained 3 groups, where as the
actual code always was the last group, and the second group was either ?
or &, and the first group would've been everything up until "code=".

changed from a matching group to character class so there would only be
two matching groups, so calling popFront() would actually leave the
authorization code in the front.
2018-01-20 20:51:33 +01:00
skilion 867c8ed26c improved regex 2018-01-19 18:56:14 +01:00