Commit graph

322 commits

Author SHA1 Message Date
abraunegg c91dcc3e49
Add handling for JSON response error when OneDrive API returns a 404 due to API regression (#340)
* Update sync.d to handle OneDrive API Regression & provide details on workaround
2019-01-16 20:14:55 +11:00
abraunegg 1d902a5beb
Handle Item not found exception when unable to query OneDrive 'root' for changes (#336)
* Add a 404 error handler when OneDrive cannot query what changes are available for a particular 'root' folder id
* Add additional verbose debug logging to assist in the future
* Change to not throw an exception when OneDrive cannot query what changes are available for a particular 'root' folder - rather display some meaningful details instead of crashing the application
* As we know the actual 'items.sqlite3' location, display that as the file to remove rather than a 'generic' message as this is more helpful
2019-01-11 05:31:10 +11:00
abraunegg bf43ecda9e
Ensure that remote deletes are handled correctly (#333)
* Fix remote deletes the fix for #323, #324 & #331 introduced a bug where an item when remote deleted, would actually be re-inserted to the database & eventually cause a database assertion
2019-01-07 04:22:09 +11:00
abraunegg b75163245f
Update handling of 'remote' folder designation as 'root' items (Issue #323, #324, #331) (#328)
* Add extra verbose debugging
* Update handling of 'remote' folder designation as 'root' items
2019-01-07 03:02:08 +11:00
abraunegg 1fa7e5f20c
Add extra debugging & http protocol downgrade (Issue #314) (#327)
* Add --force-http-1.1 flag to downgrade any HTTP/2 curl operations to HTTP 1.1 protocol
* Explicitly set all bool items to false at initialisation
* Update --display-config to display sync_list if configured
* Add debug handling to display when sync_list is loaded
* Add debug handling to output the handling of OneDrive changes
2019-01-06 05:43:44 +11:00
abraunegg ddc5d602da
Update sync.d (#329)
* Update HTTP 412 error notifications to better identify where 412 error is occuring
* Add a check for 'id' key on metadata update to prevent 'std.json.JSONException@std/json.d(494): Key not found: id' (Issue #325)
2019-01-06 05:35:55 +11:00
abraunegg 46ef8ed376 Revert "Update handling of HTTP 412 - Precondition Failed errors (#issue 325) (#326)"
This reverts commit 388cc1ded5.
2019-01-05 17:18:28 +11:00
abraunegg 388cc1ded5
Update handling of HTTP 412 - Precondition Failed errors (#issue 325) (#326)
* Update HTTP 412 - Precondition Failed messaging
* Add debug messaging when 5xx errors occur
* Update 5xx message to include
2019-01-05 07:57:10 +11:00
Norbert Preining b9890ae2e5
better help output formatting (Issue #298) 2018-12-28 18:01:50 +09:00
Norbert Preining fcf1ef2e1f
signal handling and database shutdown (#315)
Add a signal handler and shutdown database connection on SIGINT and SIGTERM
to make sure that .waf files are properly incorportated into the sqlite database.
2018-12-28 11:19:20 +09:00
abraunegg 2553366a89
Implement Feature Request: Add status command or switch (Issue #112) (#307)
* Implement Feature Request: Add status command or switch
2018-12-28 12:26:03 +11:00
abraunegg 7a6b507367
Update sync.d (#316)
* Update logic handling of --single-directory to prevent inadvertent local data loss
2018-12-28 11:57:32 +11:00
Norbert Preining 2ec6aa3df6
make sure sqlite checkpointing works by properly finalizing statements (#310)
If an sqlite statement is prepared and reused by using reset again and again, the connection
keeps a lock on the database, and checkpointing cannot ensure that transaction from the
wal file are carried over into the main database.

By preparing the statement every time it is used, the destructor calls finalize and thus
the lock is released.

Possible impacts during high frequency changes via monitor mode etc might arise.
2018-12-26 16:11:06 +09:00
abraunegg 5d388581b1
Resolve 'sync_dir' not read from 'config' file when run in Docker container #306 (#308)
* Fix logic for sync_dir handling on headless systems
* Fix logic where potentially a 'default' ~/OneDrive sync_dir could be set despite 'config' file configured for an alternate
* Add debug handling for sync_dir operations
* Add debug handling for homePath calculation
* Add debug handling for configDirBase calculation
* Reorder main() so that log.vdebug works where required
* Rework configDirName as this had the same issue as syncDir
* Update configDirName & syncDir '~' handling & replacement to be more robust and only replace if '~' is first char in string
* Add additional debug logging if syncDir is created
2018-12-23 11:15:10 +11:00
abraunegg 1060d85118
Resolve JSONException when supplying --get-O365-drive-id option with a string containing spaces (#305)
* JSONException when supplying --get-O365-drive-id option with a string containing spaces
2018-12-21 11:28:16 +11:00
abraunegg 594bfcec54
Fix syncdir option is ignored (#302)
* Explicitly set value for syncdir if the option is passed in via CLI rather than 'config' file
2018-12-20 17:27:34 +11:00
abraunegg dfd3045ce4
Fix exception when missing parameter from usage option (#299)
* Handle the exception generated when an option is used which is expecting an input parameter but none is given
2018-12-20 11:05:32 +11:00
Norbert Preining 025a3b2878
check config file keys for validity (#296)
* check config file keys for validity, use setValue instead of direct access
* Update config.d
  Add 'drive_id' to be initialised, set to an empty string
* exit application if there is a configuration file error
* Issue #293 was caused by a spelling error in the configuration file. 
  If the configuration file has errors, we should not load it or run
  using the application defaults as this may have undesirable consequences
  for users data
* missed returning false on key issue
* Missed this edit of the file
2018-12-20 08:51:21 +09:00
abraunegg 7d522583ef
Implement option --display-config (Issue #100) (#292)
* Implement --display-config to show the application configuration without actually performing a sync or application init
2018-12-20 05:42:28 +11:00
abraunegg 53df30810e check displayName rather than description (#290)
* In testing, 'description' and 'displayName' were the same, but in reality we are checking against the 'displayName' value as this is what we are asking the user to input, thus we need to check against displayName
2018-12-18 13:20:17 +09:00
Norbert Preining 91aaf91aa6 Reinitialize sync engine after three failed trials (#283)
* After extended suspend, or bad connection, the connection to OD might
go stale - thus no network connection to Microsoft OneDrive Service, skipping sync
and this continues ad infinitum. Change the last action to re-initialize the sync engine.
2018-12-13 07:08:18 +11:00
Norbert Preining 6a62406db7 Deal with deleted move targets (Issue: 273) (#278)
if the move target has been deleted before the monitor action completes,
the monitor action throws an exception when stat-ing the target for
mtime. Do not do anything in case the move target has disappeared
2018-12-08 05:02:47 +11:00
Norbert Preining 3608dcc4c2 Add very verbose (debug) mode by double -v -v (#279)
* Add very verbose (debug) mode by double -v -v - currently only used for debugging the inotify events
2018-12-08 05:01:22 +11:00
Norbert Preining 87cd18379d Deal with items that disappear during upload (Issue: #258) (#276)
* Deal with disappearing items during upload (Issue: #258)
2018-12-06 20:28:03 +11:00
Norbert Preining 06aa42f6d1 better log messages, less noise on temporary timeouts (#275)
* better log messages, less noise on temporary timeouts
2018-12-06 18:05:52 +11:00
abraunegg 0562c52492 Handle short lived files (Issue #273) (PR #274)
* Handle short lived files (Issue #273)
* Update README.md
* Add --disable-notifications to readme
2018-12-06 08:50:46 +09:00
Norbert Preining cc6cbf5ac7 Notification support via libnotify (#270)
* add external sources for (d)notify with README and license statements
* add new logAndNotify function, make building a compile time option
* use logAndNotify and main.d, make timeout message silent
* update documentation for notification support
* add command line switch --disable-notifications
* add build-deps for libnotify in README.md
2018-12-06 06:19:00 +11:00
Norbert Preining b10e641fe9 allow starting offline in monitor mode (#266)
* SyncEngine: allow for multiple calls to init by saving the state
* Factor out syncengine initialization code so that it can be reused
* Don't exit in monitor when started offline
* Always call initSyncEngine during regular syncs, this will call the SyncEngine init routine which caches the state, thus a quasi no-op.
* Add forgotten http.shutdown
2018-12-04 11:15:44 +11:00
abraunegg 29808c905c
Implement --get-O365-drive-id to get correct SharePoint Shared Library (Issue #248) (#260)
* Implement --get-O365-drive-id to return the correct drive_id of a SharePoint Shared Library to use for syncing that repository
2018-12-04 10:59:23 +11:00
abraunegg cdb11326b9
Fix unreachable statements (#268)
* Add warning flag to compiler
* Fix unreachable statements

**Credit:** Zamir SUN <sztsian@gmail.com>
2018-12-04 06:29:18 +11:00
Norbert Preining 9817104ca1 continue in monitor mode when sync timed out (#265)
* update handling of connection timeouts in monitor mode & allow application continue rather than exit
2018-12-03 13:41:31 +11:00
Norbert Preining c24e4b0b1e adjust timeout values for libcurl (issue #256) (#261)
This change adjusts the two core timeout parameters of libcurl operation
- dataTimeout: changed from 3600sec to 300sec
  this timeout controls the max time when there is no data incoming
  (actually below CURLOPT_LOW_SPEED_LIMIT)
- operationTimeout: added 3600sec
  this timeout controls the maximally allowed connection timeout
2018-12-02 11:30:50 +11:00
Norbert Preining 0d05274f32 Fix renaming of files sync issues (#252)
* issue #249 do not safeRename the target if it is in sync
* issue #249 only remove actual path if the associated ids agree
2018-11-29 21:31:44 +11:00
Norbert Preining 11138602ed gracefully handle connection errors in monitor mode (#253)
* gracefully handle connection errors, resolve issue #234
2018-11-29 20:48:24 +11:00
abraunegg a26a69ce9f
Change logging to a separate logfile is no longer the default (#239)
* Change '--download' to '--download-only' to align with '--upload-only'
* Enable logging to a separate file only if the '--enable-logging' flag is passed through at client run time
* Implement configuration option for logfile location, if logging is enabled
2018-11-24 07:13:16 +11:00
abraunegg c5ee62efd8
Sync with shared folders/drives for onedrive business (#206)
* Refactor PR104 for latest code base to enable client use with SharePoint libraries
* Implement '--disable-upload-validation' as SharePoint modifies files when uploaded
* Enable '--disable-upload-validation' by default if account type is documentLibrary - as most likely this is a SharePoint repository
2018-11-24 06:26:30 +11:00
abraunegg 23e79d39e1 Update sync.d
* Update exit codes on sync.init() to -1
2018-11-22 07:11:14 +11:00
abraunegg cbd3ecbf71
Resolve unable to create folders on shared OneDrive accounts (Issue #217) (#244)
* Update logic to use remote item details for folder creation
* Backport sync.d changes from PR #246 to this PR before merge
2018-11-22 07:03:04 +11:00
abraunegg fc162b94e2
Resolve database assertion failure due to authentication HTTP/1.1 400 Bad Request (#246)
* Enhance initialisation error handling to catch 4xx errors
2018-11-21 18:54:08 +11:00
abraunegg eaedfac34a
Update onedrive.d for additional debug logging (#238)
* In case the user does not use the --verbose tag, if we are doing --https-debug we want this printed to the console
2018-11-18 07:05:01 +11:00
abraunegg e70722862c
update files for 2.1.6 release (#235)
* update files for 2.1.6 release
2018-11-15 06:43:58 +11:00
abraunegg f785396643
Update HTTP/2 handling for session uploads (#233)
* Remove HTTP 1.1 downgrade for session uploads
* Handle HTTP/2 0 (zero) response code for successful session data upload
* Add debugging for actual server response, not curl interpreted response when using --debug-https
2018-11-15 06:08:55 +11:00
abraunegg 8a98bf3e30
Add handling for HTTP Code 302 generated by HTTP/2 calls (#229)
* Update initial Curl 7.62.0 fix with a better solution which handles the redirect's given by HTTP/2 connections (credit @Popa21)
* Use HTTP/1.1 for session uploads otherwise the response when using HTTP/2 generates a 'JSONValue is not an object' error
2018-11-12 06:01:57 +11:00
abraunegg 86ea576144
Handle HTTP request returned status code 412 (Precondition Failed) for session uploads (#227)
* Handle HTTP request returned status code 412 (Precondition Failed) for session uploads to OneDrive Personal Accounts
* Fix Failed to remove file /root/.config/onedrive/resume_upload: No such file or directory if there is a session upload error and the resume file does not get created
* Handle response codes when using 2 different systems using --upload-only but the same OneDrive account and uploading the same filename to the same location
2018-11-11 11:15:48 +11:00
abraunegg 11e477f045
Resolve files will not download when using curl 7.62.0 (http2 issue) (#225)
* Force connections to use HTTP 1.1 as Curl 7.62.0 defaults to using http2 if h2 support is built in which causes issues
2018-11-11 06:38:41 +11:00
abraunegg eb2cbfd0a8
Resolve 'Key not found: path' when syncing from some shared folders (#211)
* Add a check to validate if the parentReference object has a path object
* Fix spelling error - trough -> through
* Only upload changes on remote folder if the item is in the database - dont assert if false
2018-10-30 06:20:52 +11:00
abraunegg 53b72e0d9d
Resolve syncing of Shared Folders due to OneDrive API change - Issue #144 (#191)
* Update handling of shared folders and items which was previously broken to a OneDrive API change
* Extract syncFolderName from either local or remote folder - not just local folder.
2018-10-10 19:31:21 +11:00
abraunegg 74b9163b06
Resolve Key not found: expirationDateTime on session resume (Issue #174) (#176)
* Handle an invalid response on session resume when a 4xx / 5xx response is generated from the OneDrive service
2018-10-04 09:38:23 +10:00
abraunegg 14b2de8f4c
Resolve crash with strange filename (Issue #151) (#178)
* Update file handling to look for HTML ASCII codes which will cause uploads to fail
2018-10-04 09:33:39 +10:00
abraunegg 31fc6a4d4f
Validate filename length before download to conform with Linux FS limits #142 (#180)
* Validate filename length before download to conform with Linux FS limits
2018-10-04 09:31:28 +10:00