From 3502e0cee4b8def75c4ac027e7f1fb1bd2664237 Mon Sep 17 00:00:00 2001 From: abraunegg Date: Fri, 22 Sep 2023 05:26:32 +1000 Subject: [PATCH] readd documentation * re-add documentation --- changelog.md | 1041 ++++++++++++++++++++ config | 57 ++ docs/advanced-usage.md | 302 ++++++ docs/application-security.md | 97 ++ docs/build-rpm-howto.md | 379 +++++++ docs/business-shared-folders.md | 192 ++++ docs/docker.md | 320 ++++++ docs/install.md | 277 ++++++ docs/known-issues.md | 54 + docs/national-cloud-deployments.md | 145 +++ docs/podman.md | 289 ++++++ docs/privacy-policy.md | 65 ++ docs/sharepoint-libraries.md | 228 +++++ docs/terms-of-service.md | 54 + docs/ubuntu-package-install.md | 383 ++++++++ docs/usage.md | 1468 ++++++++++++++++++++++++++++ readme.md | 81 ++ 17 files changed, 5432 insertions(+) create mode 100644 changelog.md create mode 100644 config create mode 100644 docs/advanced-usage.md create mode 100644 docs/application-security.md create mode 100644 docs/build-rpm-howto.md create mode 100644 docs/business-shared-folders.md create mode 100644 docs/docker.md create mode 100644 docs/install.md create mode 100644 docs/known-issues.md create mode 100644 docs/national-cloud-deployments.md create mode 100644 docs/podman.md create mode 100644 docs/privacy-policy.md create mode 100644 docs/sharepoint-libraries.md create mode 100644 docs/terms-of-service.md create mode 100644 docs/ubuntu-package-install.md create mode 100644 docs/usage.md create mode 100644 readme.md diff --git a/changelog.md b/changelog.md new file mode 100644 index 00000000..8f7f357a --- /dev/null +++ b/changelog.md @@ -0,0 +1,1041 @@ +# Changelog +The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) +and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). + +## 2.5.0 - TBA + + +### Changed +* Renamed various documentation files to align with document content + + +## 2.4.25 - 2023-06-21 +### Fixed +* Fixed that the application was reporting as v2.2.24 when in fact it was v2.4.24 (release tagging issue) +* Fixed that the running version obsolete flag (due to above issue) was causing a false flag as being obsolete +* Fixed that zero-byte files do not have a hash as reported by the OneDrive API thus should not generate an error message + +### Updated +* Update to Debian Docker file to resolve Docker image Operating System reported vulnerabilities +* Update to Alpine Docker file to resolve Docker image Operating System reported vulnerabilities +* Update to Fedora Docker file to resolve Docker image Operating System reported vulnerabilities +* Updated documentation (various) + +## 2.4.24 - 2023-06-20 +### Fixed +* Fix for extra encoded quotation marks surrounding Docker environment variables +* Fix webhook subscription creation for SharePoint Libraries +* Fix that a HTTP 504 - Gateway Timeout causes local files to be deleted when using --download-only & --cleanup-local-files mode +* Fix that folders are renamed despite using --dry-run +* Fix deprecation warnings with dmd 2.103.0 +* Fix error that the application is unable to perform a database vacuum: out of memory when exiting + +### Removed +* Remove sha1 from being used by the client as this is being depreciated by Microsoft in July 2023 +* Complete the removal of crc32 elements + +### Added +* Added ONEDRIVE_SINGLE_DIRECTORY configuration capability to Docker +* Added --get-file-link shell completion +* Added configuration to allow HTTP session timeout(s) tuning via config (taken from v2.5.x) + +### Updated +* Update to Debian Docker file to resolve Docker image Operating System reported vulnerabilities +* Update to Alpine Docker file to resolve Docker image Operating System reported vulnerabilities +* Update to Fedora Docker file to resolve Docker image Operating System reported vulnerabilities +* Updated cgi.d to commit 680003a - last upstream change before requiring `core.d` dependency requirement +* Updated documentation (various) + +## 2.4.23 - 2023-01-06 +### Fixed +* Fixed RHEL7, RHEL8 and RHEL9 Makefile and SPEC file compatibility + +### Removed +* Disable systemd 'PrivateUsers' due to issues with systemd running processes when option is enabled, causes local file deletes on RHEL based systems + +### Updated +* Update --get-O365-drive-id error handling to display a more a more appropriate error message if the API cannot be found +* Update the GitHub version check to utilise the date a release was done, to allow 1 month grace period before generating obsolete version message +* Update Alpine Dockerfile to use Alpine 3.17 and Golang 1.19 +* Update handling of --source-directory and --destination-directory if one is empty or missing and if used with --synchronize or --monitor +* Updated documentation (various) + +## 2.4.22 - 2022-12-06 +### Fixed +* Fix application crash when local file is changed to a symbolic link with non-existent target +* Fix build error with dmd-2.101.0 +* Fix build error with LDC 1.28.1 on Alpine +* Fix issue of silent exit when unable to delete local files when using --cleanup-local-files +* Fix application crash due to access permissions on configured path for sync_dir +* Fix potential application crash when exiting due to failure state and unable to cleanly shutdown the database +* Fix creation of parent empty directories when parent is excluded by sync_list + +### Added +* Added performance output details for key functions + +### Changed +* Switch Docker 'latest' to point at Debian builds rather than Fedora due to ongoing Fedora build failures +* Align application logging events to actual application defaults for --monitor operations +* Performance Improvement: Avoid duplicate costly path calculations and DB operations if not required +* Disable non-working remaining sandboxing options within systemd service files +* Performance Improvement: Only check 'sync_list' if this has been enabled and configured +* Display 'Sync with OneDrive is complete' when using --synchronize +* Change the order of processing between Microsoft OneDrive restrictions and limitations check and skip_file|skip_dir check + +### Removed +* Remove building Fedora ARMv7 builds due to ongoing build failures + +### Updated +* Update config change detection handling +* Updated documentation (various) + +## 2.4.21 - 2022-09-27 +### Fixed +* Fix that the download progress bar doesn't always reach 100% when rate_limit is set +* Fix --resync handling of database file removal +* Fix Makefile to be consistent with permissions that are being used +* Fix that logging output for skipped uploaded files is missing +* Fix to allow non-sync tasks while sync is running +* Fix where --resync is enforced for non-sync operations +* Fix to resolve segfault when running 'onedrive --display-sync-status' when run as 2nd process +* Fix DMD 2.100.2 depreciation warning + +### Added +* Add GitHub Action Test Build Workflow (replacing Travis CI) +* Add option --display-running-config to display the running configuration as used at application startup +* Add 'config' option to request readonly access in oauth authorization step +* Add option --cleanup-local-files to cleanup local files regardless of sync state when using --download-only +* Add option --with-editing-perms to create a read-write shareable link when used with --create-share-link + +### Changed +* Change the exit code of the application to 126 when a --resync is required + +### Updated +* Updated --get-O365-drive-id implementation for data access +* Update what application options require an argument +* Update application logging output for error messages to remove certain \n prefix when logging to a file +* Update onedrive.spec.in to fix error building RPM +* Update GUI notification handling for specific skipped scenarios +* Updated documentation (various) + +## 2.4.20 - 2022-07-20 +### Fixed +* Fix 'foreign key constraint failed' when using OneDrive Business Shared Folders due to change to using /delta query +* Fix various little spelling fixes (check with lintian during Debian packaging) +* Fix handling of a custom configuration directory when using --confdir +* Fix to ensure that any active http instance is shutdown before any application exit +* Fix to enforce that --confdir must be a directory + +### Added +* Added 'force_http_11' configuration option to allow forcing HTTP/1.1 operations + +### Changed +* Increased thread sleep for better process I/O wait handling +* Removed 'force_http_2' configuration option + +### Updated +* Update OneDrive API response handling for National Cloud Deployments +* Updated to switch to using curl defaults for HTTP/2 operations +* Updated documentation (various) + +## 2.4.19 - 2022-06-15 +### Fixed +* Update Business Shared Folders to use a /delta query +* Update when DB is updated by OneDrive API data and update when file hash is required to be generated + +### Added +* Added ONEDRIVE_UPLOADONLY flag for Docker + +### Updated +* Updated GitHub workflows +* Updated documentation (various) + +## 2.4.18 - 2022-06-02 +### Fixed +* Fixed various database related access issues steming from running multiple instances of the application at the same time using the same configuration data +* Fixed --display-config being impacted by --resync flag +* Fixed installation permissions for onedrive man-pages file +* Fixed that in some situations that users try --upload-only and --download-only together which is not possible +* Fixed application crash if unable to read required hash files + +### Added +* Added Feature Request to add an override for skip_dir|skip_file through flag to force sync +* Added a check to validate local filesystem available space before attempting file download +* Added GitHub Actions to build Docker containers and push to DockerHub + +### Updated +* Updated all Docker build files to current distributions, using updated distribution LDC version +* Updated logging output to logfiles when an actual sync process is occuring +* Updated output of --display-config to be more relevant +* Updated manpage to align with application configuration +* Updated documentation and Docker files based on minimum compiler versions to dmd-2.088.0 and ldc-1.18.0 +* Updated documentation (various) + +## 2.4.17 - 2022-04-30 +### Fixed +* Fix docker build, by add missing git package for Fedora builds +* Fix application crash when attempting to sync a broken symbolic link +* Fix Internet connect disruption retry handling and logging output +* Fix local folder creation timestamp with timestamp from OneDrive +* Fix logging output when download failed + +### Added +* Add additional logging specifically for delete event to denote in log output the source of a deletion event when running in --monitor mode + +### Changed +* Improve when the local database integrity check is performed and on what frequency the database integrity check is performed + +### Updated +* Remove application output ambiguity on how to access 'help' for the client +* Update logging output when running in --monitor --verbose mode in regards to the inotify events +* Updated documentation (various) + +## 2.4.16 - 2022-03-10 +### Fixed +* Update application file logging error handling +* Explicitly set libcurl options +* Fix that when a sync_list exclusion is matched, the item needs to be excluded when using --resync +* Fix so that application can be compiled correctly on Android hosts +* Fix the handling of 429 and 5xx responses when they are generated by OneDrive in a self-referencing circular pattern +* Fix applying permissions to volume directories when running in rootless podman +* Fix unhandled errors from OneDrive when initialising subscriptions fail + +### Added +* Enable GitHub Sponsors +* Implement --resync-auth to enable CLI passing in of --rsync approval +* Add function to check client version vs latest GitHub release +* Add --reauth to allow easy re-authentication of the client +* Implement --modified-by to display who last modified a file and when the modification was done +* Implement feature request to mark partially-downloaded files as .partial during download +* Add documentation for Podman support + +### Changed +* Document risk regarding using --resync and force user acceptance of usage risk to proceed +* Use YAML for Bug Reports and Feature Requests +* Update Dockerfiles to use more modern base Linux distribution + +### Updated +* Updated documentation (various) + +## 2.4.15 - 2021-12-31 +### Fixed +* Fix unable to upload to OneDrive Business Shared Folders due to OneDrive API restricting quota information +* Update fixing edge case with OneDrive Personal Shared Folders and --resync --upload-only + +### Added +* Add SystemD hardening +* Add --operation-timeout argument + +### Changed +* Updated minimum compiler versions to dmd-2.087.0 and ldc-1.17.0 + +### Updated +* Updated Dockerfile-alpine to use Apline 3.14 +* Updated documentation (various) + +## 2.4.14 - 2021-11-24 +### Fixed +* Support DMD 2.097.0 as compiler for Docker Builds +* Fix getPathDetailsByDriveId query when using --dry-run and a nested path with --single-directory +* Fix edge case when syncing OneDrive Personal Shared Folders +* Catch unhandled API response errors when querying OneDrive Business Shared Folders +* Catch unhandled API response errors when listing OneDrive Business Shared Folders +* Fix error 'Key not found: remaining' with Business Shared Folders (OneDrive API change) +* Fix overwriting local files with older versions from OneDrive when items.sqlite3 does not exist and --resync is not used + +### Added +* Added operation_timeout as a new configuration to assist in cases where operations take longer that 1h to complete +* Add Real-Time syncing of remote updates via webhooks +* Add --auth-response option and expose through entrypoint.sh for Docker +* Add --disable-download-validation + +### Changed +* Always prompt for credentials for authentication rather than re-using cached browser details +* Do not re-auth on --logout + +### Updated +* Updated documentation (various) + +## 2.4.13 - 2021-7-14 +### Fixed +* Support DMD 2.097.0 as compiler +* Fix to handle OneDrive API Bad Request response when querying if file exists +* Fix application crash and incorrect handling of --single-directory when syncing a OneDrive Business Shared Folder due to using 'Add Shortcut to My Files' +* Fix application crash due to invalid UTF-8 sequence in the pathname for the application configuration +* Fix error message when deleting a large number of files +* Fix Docker build process to source GOSU keys from updated GPG key location +* Fix application crash due to a conversion overflow when calculating file offset for session uploads +* Fix Docker Alpine build failing due to filesystem permissions issue due to Docker build system and Alpine Linux 3.14 incompatibility +* Fix that Business Shared Folders with parentheses are ignored + +### Updated +* Updated Lock Bot to run daily +* Updated documentation (various) + +## 2.4.12 - 2021-5-28 +### Fixed +* Fix an unhandled Error 412 when uploading modified files to OneDrive Business Accounts +* Fix 'sync_list' handling of inclusions when name is included in another folders name +* Fix that options --upload-only & --remove-source-files are ignored on an upload session restore +* Fix to add file check when adding item to database if using --upload-only --remove-source-files +* Fix application crash when SharePoint displayName is being withheld + +### Updated +* Updated Lock Bot to use GitHub Actions +* Updated documentation (various) + +## 2.4.11 - 2021-4-07 +### Fixed +* Fix support for '/*' regardless of location within sync_list file +* Fix 429 response handling correctly check for 'retry-after' response header and use set value +* Fix 'sync_list' path handling for sub item matching, so that items in parent are not implicitly matched when there is no wildcard present +* Fix --get-O365-drive-id to use 'nextLink' value if present when searching for specific SharePoint site names +* Fix OneDrive Business Shared Folder existing name conflict check +* Fix incorrect error message 'Item cannot be deleted from OneDrive because it was not found in the local database' when item is actually present +* Fix application crash when unable to rename folder structure due to unhandled file-system issue +* Fix uploading documents to Shared Business Folders when the shared folder exists on a SharePoint site due to Microsoft Sharepoint 'enrichment' of files +* Fix that a file record is kept in database when using --no-remote-delete & --remove-source-files + +### Added +* Added support in --get-O365-drive-id to provide the 'drive_id' for multiple 'document libraries' within a single Shared Library Site + +### Removed +* Removed the depreciated config option 'force_http_11' which was flagged as depreciated by PR #549 in v2.3.6 (June 2019) + +### Updated +* Updated error output of --get-O365-drive-id to provide more details why an error occurred if a SharePoint site lacks the details we need to perform the match +* Updated Docker build files for Raspberry Pi to dedicated armhf & aarch64 Dockerfiles +* Updated logging output when in --monitor mode, avoid outputting misleading logging when the new or modified item is a file, not a directory +* Updated documentation (various) + +## 2.4.10 - 2021-2-19 +### Fixed +* Catch database assertion when item path cannot be calculated +* Fix alpine Docker build so it uses the same golang alpine version +* Search all distinct drive id's rather than just default drive id for --get-file-link +* Use correct driveId value to query for changes when using --single-directory +* Improve upload handling of files for SharePoint sites and detecting when SharePoint modifies the file post upload +* Correctly handle '~' when present in 'log_dir' configuration option +* Fix logging output when handing downloaded new files +* Fix to use correct path offset for sync_list exclusion matching + +### Added +* Add upload speed metrics when files are uploaded and clarify that 'data to transfer' is what is needed to be downloaded from OneDrive +* Add new config option to rate limit connection to OneDrive +* Support new file maximum upload size of 250GB +* Support sync_list matching full path root wildcard with exclusions to simplify sync_list configuration + +### Updated +* Rename Office365.md --> SharePoint-Shared-Libraries.md which better describes this document +* Updated Dockerfile config for arm64 +* Updated documentation (various) + +## 2.4.9 - 2020-12-27 +### Fixed +* Fix to handle case where API provided deltaLink generates a further API error +* Fix application crash when unable to read a local file due to local file permissions +* Fix application crash when calculating the path length due to invalid UTF characters in local path +* Fix Docker build on Alpine due missing symbols due to using the edge version of ldc and ldc-runtime +* Fix application crash with --get-O365-drive-id when API response is restricted + +### Added +* Add debug log output of the configured URL's which will be used throughout the application to remove any ambiguity as to using incorrect URL's when making API calls +* Improve application startup when using --monitor when there is no network connection to the OneDrive API and only initialise application once OneDrive API is reachable +* Add Docker environment variable to allow --logout for re-authentication + +### Updated +* Remove duplicate code for error output functions and enhance error logging output +* Updated documentation + +## 2.4.8 - 2020-11-30 +### Fixed +* Fix to use config set option for 'remove_source_files' and 'skip_dir_strict_match' rather than ignore if set +* Fix download failure and crash due to incorrect local filesystem permissions when using mounted external devices +* Fix to not change permissions on pre-existing local directories +* Fix logging output when authentication authorisation fails to not say authorisation was successful +* Fix to check application_id before setting redirect URL when using specific Azure endpoints +* Fix application crash in --monitor mode due to 'Failed to stat file' when setgid is used on a directory and data cannot be read + +### Added +* Added advanced-usage.md to document advaced client usage such as multi account configurations and Windows dual-boot + +### Updated +* Updated --verbose logging output for config options when set +* Updated documentation (man page, USAGE.md, Office365.md, BusinessSharedFolders.md) + +## 2.4.7 - 2020-11-09 +### Fixed +* Fix debugging output for /delta changes available queries +* Fix logging output for modification comparison source data +* Fix Business Shared Folder handling to process only Shared Folders, not individually shared files +* Fix cleanup dryrun shm and wal files if they exist +* Fix --list-shared-folders to only show folders +* Fix to check for the presence of .nosync when processing DB entries +* Fix skip_dir matching when using --resync +* Fix uploading data to shared business folders when using --upload-only +* Fix to merge contents of SQLite WAL file into main database file on sync completion +* Fix to check if localModifiedTime is >= than item.mtime to avoid re-upload for equal modified time +* Fix to correctly set config directory permissions at first start + +### Added +* Added environment variable to allow easy HTTPS debug in docker +* Added environment variable to allow download-only mode in Docker +* Implement Feature: Allow config to specify a tenant id for non-multi-tenant applications +* Implement Feature: Adding support for authentication with single tenant custom applications +* Implement Feature: Configure specific File and Folder Permissions + +### Updated +* Updated documentation (readme.md, install.md, usage.md, bug_report.md) + +## 2.4.6 - 2020-10-04 +### Fixed +* Fix flagging of remaining free space when value is being restricted +* Fix --single-directory path handling when path does not exist locally +* Fix checking for 'Icon' path as no longer listed by Microsoft as an invalid file or folder name +* Fix removing child items on OneDrive when parent item responds with access denied +* Fix to handle deletion events for files when inotify events are missing +* Fix uninitialised value error as reported by valgrind +* Fix to handle deletion events for directories when inotify events are missing + +### Added +* Implement Feature: Create shareable link +* Implement Feature: Support wildcard within sync_list entries +* Implement Feature: Support negative patterns in sync_list for fine grained exclusions +* Implement Feature: Multiple skip_dir & skip_file configuration rules +* Add GUI notification to advise users when the client needs to be reauthenticated + +### Updated +* Updated documentation (readme.md, install.md, usage.md, bug_report.md) + +## 2.4.5 - 2020-08-13 +### Fixed +* Fixed fish auto completions installation destination + +## 2.4.4 - 2020-08-11 +### Fixed +* Fix 'skip_dir' & 'skip_file' pattern matching to ensure correct matching is performed +* Fix 'skip_dir' & 'skip_file' so that each directive is only used against directories or files as requried in --monitor +* Fix client hand when attempting to sync a Unix pipe file +* Fix --single-directory & 'sync_list' performance +* Fix erroneous 'return' statements which could prematurely end processing all changes returned from OneDrive +* Fix segfault when attempting to perform a comparison on an inotify event when determining if event path is directory or file +* Fix handling of Shared Folders to ensure these are checked against 'skip_dir' entries +* Fix 'Skipping uploading this new file as parent path is not in the database' when uploading to a Personal Shared Folder +* Fix how available free space is tracked when uploading files to OneDrive and Shared Folders +* Fix --single-directory handling of parent path matching if path is being seen for first time + +### Added +* Added Fish auto completions + +### Updated +* Increase maximum individual file size to 100GB due to Microsoft file limit increase +* Update Docker build files and align version of compiler across all Docker builds +* Update Docker documentation +* Update NixOS build information +* Update the 'Processing XXXX' output to display the full path +* Update logging output when a sync starts and completes when using --monitor +* Update Office 365 / SharePoint site search query and response if query return zero match + +## 2.4.3 - 2020-06-29 +### Fixed +* Check if symbolic link is relative to location path +* When using output logfile, fix inconsistent output spacing +* Perform initial sync at startup in monitor mode +* Handle a 'race' condition to process inotify events generated whilst performing DB or filesystem walk +* Fix segfault when moving folder outside the sync directory when using --monitor on Arch Linux + +### Added +* Added additional inotify event debugging +* Added support for loading system configs if there's no user config +* Added Ubuntu installation details to include installing the client from a PPA +* Added openSUSE installation details to include installing the client from a package +* Added support for comments in sync_list file +* Implement recursive deletion when Retention Policy is enabled on OneDrive Business Accounts +* Implement support for National cloud deployments +* Implement OneDrive Business Shared Folders Support + +### Updated +* Updated documentation files (various) +* Updated log output messaging when a full scan has been set or triggered +* Updated buildNormalizedPath complexity to simplify code +* Updated to only process OneDrive Personal Shared Folders only if account type is 'personal' + +## 2.4.2 - 2020-05-27 +### Fixed +* Fixed the catching of an unhandled exception when inotify throws an error +* Fixed an uncaught '100 Continue' response when files are being uploaded +* Fixed progress bar for uploads to be more accurate regarding percentage complete +* Fixed handling of database query enforcement if item is from a shared folder +* Fixed compiler depreciation of std.digest.digest +* Fixed checking & loading of configuration file sequence +* Fixed multiple issues reported by Valgrind +* Fixed double scan at application startup when using --monitor & --resync together +* Fixed when renaming a file locally, ensure that the target filename is valid before attempting to upload to OneDrive +* Fixed so that if a file is modified locally and --resync is used, rename the local file for data preservation to prevent local data loss + +### Added +* Implement 'bypass_data_preservation' enhancement + +### Changed +* Changed the monitor interval default to 300 seconds + +### Updated +* Updated the handling of out-of-space message when OneDrive is out of space +* Updated debug logging for retry wait times + +## 2.4.1 - 2020-05-02 +### Fixed +* Fixed the handling of renaming files to a name starting with a dot when skip_dotfiles = true +* Fixed the handling of parentheses from path or file names, when doing comparison with regex +* Fixed the handling of renaming dotfiles to another dotfile when skip_dotfile=true in monitor mode +* Fixed the handling of --dry-run and --resync together correctly as current database may be corrupt +* Fixed building on Alpine Linux under Docker +* Fixed the handling of --single-directory for --dry-run and --resync scenarios +* Fixed the handling of .nosync directive when downloading new files into existing directories that is (was) in sync +* Fixed the handling of zero-byte modified files for OneDrive Business +* Fixed skip_dotfiles handling of .folders when in monitor mode to prevent monitoring +* Fixed the handling of '.folder' -> 'folder' move when skip_dotfiles is enabled +* Fixed the handling of folders that cannot be read (permission error) if parent should be skipped +* Fixed the handling of moving folders from skipped directory to non-skipped directory via OneDrive web interface +* Fixed building on CentOS Linux under Docker +* Fixed Codacy reported issues: double quote to prevent globbing and word splitting +* Fixed an assertion when attempting to compute complex path comparison from shared folders +* Fixed the handling of .folders when being skipped via skip_dir + +### Added +* Implement Feature: Implement the ability to set --resync as a config option, default is false + +### Updated +* Update error logging to be consistent when initialising fails +* Update error logging output to handle HTML error response reasoning if present +* Update link to new Microsoft documentation +* Update logging output to differentiate between OneNote objects and other unsupported objects +* Update RHEL/CentOS spec file example +* Update known-issues.md regarding 'SSL_ERROR_SYSCALL, errno 104' +* Update progress bar to be more accurate when downloading large files +* Updated #658 and #865 handling of when to trigger a directory walk when changes occur on OneDrive +* Updated handling of when a full scan is requried due to utilising sync_list +* Updated handling of when OneDrive service throws a 429 or 504 response to retry original request after a delay + +## 2.4.0 - 2020-03-22 +### Fixed +* Fixed how the application handles 429 response codes from OneDrive (critical update) +* Fixed building on Alpine Linux under Docker +* Fixed how the 'username' is determined from the running process for logfile naming +* Fixed file handling when a failed download has occured due to exiting via CTRL-C +* Fixed an unhandled exception when OneDrive throws an error response on initialising +* Fixed the handling of moving files into a skipped .folder when skip_dotfiles = true +* Fixed the regex parsing of response URI to avoid potentially generating a bad request to OneDrive, leading to a 'AADSTS9002313: Invalid request. Request is malformed or invalid.' response. + +### Added +* Added a Dockerfile for building on Rasberry Pi / ARM platforms +* Implement Feature: warning on big deletes to safeguard data on OneDrive +* Implement Feature: delete local files after sync +* Implement Feature: perform skip_dir explicit match only +* Implement Feature: provide config file option for specifying the Client Identifier + +### Changed +* Updated the 'Client Identifier' to a new Application ID + +### Updated +* Updated relevant documentation (README.md, USAGE.md) to add new feature details and clarify existing information +* Update completions to include the --force-http-2 option +* Update to always log when a file is skipped due to the item being invalid +* Update application output when just authorising application to make information clearer +* Update logging output when using sync_list to be clearer as to what is actually being processed and why + +## 2.3.13 - 2019-12-31 +### Fixed +* Change the sync list override flag to false as default when not using sync_list +* Fix --dry-run output when using --upload-only & --no-remote-delete and deleting local files + +### Added +* Add a verbose log entry when a monitor sync loop with OneDrive starts & completes + +### Changed +* Remove logAndNotify for 'processing X changes' as it is excessive for each change bundle to inform the desktop of the number of changes the client is processing + +### Updated +* Updated INSTALL.md with Ubuntu 16.x i386 build instructions to reflect working configuration on legacy hardware +* Updated INSTALL.md with details of Linux packages +* Updated INSTALL.md build instructions for CentOS platforms + +## 2.3.12 - 2019-12-04 +### Fixed +* Retry session upload fragment when transient errors occur to prevent silent upload failure +* Update Microsoft restriction and limitations about windows naming files to include '~' for folder names +* Docker guide fixes, add multiple account setup instructions +* Check database for excluded sync_list items previously in scope +* Catch DNS resolution error +* Fix where an item now out of scope should be flagged for local delete +* Fix rebuilding of onedrive, but ensure version is properly updated +* Update Ubuntu i386 build instructions to use DMD using preferred method + +### Added +* Add debug message to when a message is sent to dbus or notification daemon +* Add i386 instructions for legacy low memory platforms using LDC + +## 2.3.11 - 2019-11-05 +### Fixed +* Fix typo in the documentation regarding invalid config when upgrading from 'skilion' codebase +* Fix handling of skip_dir, skip_file & sync_list config options +* Fix typo in the documentation regarding sync_list +* Fix log output to be consistent with sync_list exclusion +* Fix 'Processing X changes' output to be more reflective of actual activity when using sync_list +* Remove unused and unexported SED variable in Makefile.in +* Handle curl exceptions and timeouts better with backoff/retry logic +* Update skip_dir pattern matching when using wildcards +* Fix when a full rescan is performed when using sync_list +* Fix 'Key not found: name' when computing skip_dir path +* Fix call from --monitor to observe --no-remote-delete +* Fix unhandled exception when monitor initialisation failure occurs due to too many open local files +* Fix unhandled 412 error response from OneDrive API when moving files right after upload +* Fix --monitor when used with --download-only. This fixes a regression introduced in 12947d1. +* Fix if --single-directory is being used, and we are using --monitor, only set inotify watches on the single directory + +### Changed +* Move JSON logging output from error messages to debug output + +## 2.3.10 - 2019-10-01 +### Fixed +* Fix searching for 'name' when deleting a synced item, if the OneDrive API does not return the expected details in the API call +* Fix abnormal termination when no Internet connection +* Fix downloading of files from OneDrive Personal Shared Folders when the OneDrive API responds with unexpected additional path data +* Fix logging of 'initialisation' of client to actually when the attempt to initialise is performed +* Fix when using a sync_list file, using deltaLink will actually 'miss' changes (moves & deletes) on OneDrive as using sync_list discards changes +* Fix OneDrive API status code 500 handling when uploading files as error message is not correct +* Fix crash when resume_upload file is not a valid JSON +* Fix crash when a file system exception is generated when attempting to update the file date & time and this fails + +### Added +* If there is a case-insensitive match error, also return the remote name from the response +* Make user-agent string a configuration option & add to config file +* Set default User-Agent to 'OneDrive Client for Linux v{version}' + +### Changed +* Make verbose logging output optional on Docker +* Enable --resync & debug client output via environment variables on Docker + +## 2.3.9 - 2019-09-01 +### Fixed +* Catch a 403 Forbidden exception when querying Sharepoint Library Names +* Fix unhandled error exceptions that cause application to exit / crash when uploading files +* Fix JSON object validation for queries made against OneDrive where a JSON response is expected and where that response is to be used and expected to be valid +* Fix handling of 5xx responses from OneDrive when uploading via a session + +### Added +* Detect the need for --resync when config changes either via config file or cli override + +### Changed +* Change minimum required version of LDC to v1.12.0 + +### Removed +* Remove redundant logging output due to change in how errors are reported from OneDrive + +## 2.3.8 - 2019-08-04 +### Fixed +* Fix unable to download all files when OneDrive fails to return file level details used to validate file integrity +* Included the flag "-m" to create the home directory when creating the user +* Fix entrypoint.sh to work with "sudo docker run" +* Fix docker build error on stretch +* Fix hidden directories in 'root' from having prefix removed +* Fix Sharepoint Document Library handling for .txt & .csv files +* Fix logging for init.d service +* Fix OneDrive response missing required 'id' element when uploading images +* Fix 'Unexpected character '<'. (Line 1:1)' when OneDrive has an exception error +* Fix error when creating the sync dir fails when there is no permission to create the sync dir + +### Added +* Add explicit check for hashes to be returned in cases where OneDrive API fails to provide them despite requested to do so +* Add comparison with sha1 if OneDrive provides that rather than quickXor +* Add selinux configuration details for a sync folder outside of the home folder +* Add date tag on docker.hub +* Add back CentOS 6 install & uninstall to Makefile +* Add a check to handle moving items out of sync_list sync scope & delete locally if true +* Implement --get-file-link which will return the weburl of a file which has been synced to OneDrive + +### Changed +* Change unauthorized-api exit code to 3 +* Update LDC to v1.16.0 for Travis CI testing +* Use replace function for modified Sharepoint Document Library files rather than delete and upload as new file, preserving file history +* Update Sharepoint modified file handling for files > 4Mb in size + +### Removed +* Remove -d shorthand for --download-only to avoid confusion with other GNU applications where -d stands for 'debug' + +## 2.3.7 - 2019-07-03 +### Fixed +* Fix not all files being downloaded due to OneDrive query failure +* False DB update which potentially could had lead to false data loss on OneDrive + +## 2.3.6 - 2019-07-03 (DO NOT USE) +### Fixed +* Fix JSONValue object validation +* Fix building without git being available +* Fix some spelling/grammatical errors +* Fix OneDrive error response on creating upload session + +### Added +* Add download size & hash check to ensure downloaded files are valid and not corrupt +* Added --force-http-2 to use HTTP/2 if desired + +### Changed +* Depreciated --force-http-1.1 (enabled by default) due to OneDrive inconsistent behavior with HTTP/2 protocol + +## 2.3.5 - 2019-06-19 +### Fixed +* Handle a directory in the sync_dir when no permission to access +* Get rid of forced root necessity during installation +* Fix broken autoconf code for --enable-XXX options +* Fix so that skip_size check should only be used if configured +* Fix a OneDrive Internal Error exception occurring before attempting to download a file + +### Added +* Check for supported version of D compiler + +## 2.3.4 - 2019-06-13 +### Fixed +* Fix 'Local files not deleted' when using bad 'skip_file' entry +* Fix --dry-run logging output for faking downloading new files +* Fix install unit files to correct location on RHEL/CentOS 7 +* Fix up unit file removal on all platforms +* Fix setting times on a file by adding a check to see if the file was actually downloaded before attempting to set the times on the file +* Fix an unhandled curl exception when OneDrive throws an internal timeout error +* Check timestamp to ensure that latest timestamp is used when comparing OneDrive changes +* Fix handling responses where cTag JSON elements are missing +* Fix Docker entrypoint.sh failures when GID is defined but not UID + +### Added +* Add autoconf based build system +* Add an encoding validation check before any path length checks are performed as if the path contains any invalid UTF-8 sequences +* Implement --sync-root-files to sync all files in the OneDrive root when using a sync_list file that would normally exclude these files from being synced +* Implement skip_size feature request +* Implement feature request to support file based OneDrive authorization (request | response) + +### Updated +* Better handle initialisation issues when OneDrive / MS Graph is experiencing problems that generate 401 & 5xx error codes +* Enhance error message when unable to connect to Microsoft OneDrive service when the local CA SSL certificate(s) have issues +* Update Dockerfile to correctly build on Docker Hub +* Rework directory layout and re-factor MD files for readability + +## 2.3.3 - 2019-04-16 +### Fixed +* Fix --upload-only check for Sharepoint uploads +* Fix check to ensure item root we flag as 'root' actually is OneDrive account 'root' +* Handle object error response from OneDrive when uploading to OneDrive Business +* Fix handling of some OneDrive accounts not providing 'quota' details +* Fix 'resume_upload' handling in the event of bad OneDrive response + +### Added +* Add debugging for --get-O365-drive-id function +* Add shell (bash,zsh) completion support +* Add config options for command line switches to allow for better config handling in docker containers + +### Updated +* Implement more meaningful 5xx error responses +* Update onedrive.logrotate indentations and comments +* Update 'min_notif_changes' to 'min_notify_changes' + +## 2.3.2 - 2019-04-02 +### Fixed +* Reduce scanning the entire local system in monitor mode for local changes +* Resolve file creation loop when working directly in the synced folder and Microsoft Sharepoint + +### Added +* Add 'monitor_fullscan_frequency' config option to set the frequency of performing a full disk scan when in monitor mode + +### Updated +* Update default 'skip_file' to include tmp and lock files generated by LibreOffice +* Update database version due to changing defaults of 'skip_file' which will force a rebuild and use of new skip_file default regex + +## 2.3.1 - 2019-03-26 +### Fixed +* Resolve 'make install' issue where rebuild of application would occur due to 'version' being flagged as .PHONY +* Update readme build instructions to include 'make clean;' before build to ensure that 'version' is cleanly removed and can be updated correctly +* Update Debian Travis CI build URL's + +## 2.3.0 - 2019-03-25 +### Fixed +* Resolve application crash if no 'size' value is returned when uploading a new file +* Resolve application crash if a 5xx error is returned when uploading a new file +* Resolve not 'refreshing' version file when rebuilding +* Resolve unexpected application processing by preventing use of --synchronize & --monitor together +* Resolve high CPU usage when performing DB reads +* Update error logging around directory case-insensitive match +* Update Travis CI and ARM dependencies for LDC 1.14.0 +* Update Makefile due to build failure if building from release archive file +* Update logging as to why a OneDrive object was skipped + +### Added +* Implement config option 'skip_dir' + +## 2.2.6 - 2019-03-12 +### Fixed +* Resolve application crash when unable to delete remote folders when business retention policies are enabled +* Resolve deprecation warning: loop index implicitly converted from size_t to int +* Resolve warnings regarding 'bashisms' +* Resolve handling of notification failure is dbus server has not started or available +* Resolve handling of response JSON to ensure that 'id' key element is always checked for +* Resolve excessive & needless logging in monitor mode +* Resolve compiling with LDC on Alpine as musl lacks some standard interfaces +* Resolve notification issues when offline and cannot act on changes +* Resolve Docker entrypoint.sh to accept command line arguments +* Resolve to create a new upload session on reinit +* Resolve where on OneDrive query failure, default root and drive id is used if a response is not returned +* Resolve Key not found: nextExpectedRanges when attempting session uploads and incorrect response is returned +* Resolve application crash when re-using an authentication URI twice after previous --logout +* Resolve creating a folder on a shared personal folder appears successful but returns a JSON error +* Resolve to treat mv of new file as upload of mv target +* Update Debian i386 build dependencies +* Update handling of --get-O365-drive-id to print out all 'site names' that match the explicit search entry rather than just the last match +* Update Docker readme & documentation +* Update handling of validating local file permissions for new file uploads +### Added +* Add support for install & uninstall on RHEL / CentOS 6.x +* Add support for when notifications are enabled, display the number of OneDrive changes to process if any are found +* Add 'config' option 'min_notif_changes' for minimum number of changes to notify on, default = 5 +* Add additional Docker container builds utilising a smaller OS footprint +* Add configurable interval of logging in monitor mode +* Implement new CLI option --skip-dot-files to skip .files and .folders if option is used +* Implement new CLI option --check-for-nosync to ignore folder when special file (.nosync) present +* Implement new CLI option --dry-run + +## 2.2.5 - 2019-01-16 +### Fixed +* Update handling of HTTP 412 - Precondition Failed errors +* Update --display-config to display sync_list if configured +* Add a check for 'id' key on metadata update to prevent 'std.json.JSONException@std/json.d(494): Key not found: id' +* Update handling of 'remote' folder designation as 'root' items +* Ensure that remote deletes are handled correctly +* Handle 'Item not found' exception when unable to query OneDrive 'root' for changes +* Add handling for JSON response error when OneDrive API returns a 404 due to OneDrive API regression +* Fix items highlighted by codacy review +### Added +* Add --force-http-1.1 flag to downgrade any HTTP/2 curl operations to HTTP 1.1 protocol +* Support building with ldc2 and usage of pkg-config for lib finding + +## 2.2.4 - 2018-12-28 +### Fixed +* Resolve JSONException when supplying --get-O365-drive-id option with a string containing spaces +* Resolve 'sync_dir' not read from 'config' file when run in Docker container +* Resolve logic where potentially a 'default' ~/OneDrive sync_dir could be set despite 'config' file configured for an alternate +* Make sure sqlite checkpointing works by properly finalizing statements +* Update logic handling of --single-directory to prevent inadvertent local data loss +* Resolve signal handling and database shutdown on SIGINT and SIGTERM +* Update man page +* Implement better help output formatting +### Added +* Add debug handling for sync_dir operations +* Add debug handling for homePath calculation +* Add debug handling for configDirBase calculation +* Add debug handling if syncDir is created +* Implement Feature Request: Add status command or switch + +## 2.2.3 - 2018-12-20 +### Fixed +* Fix syncdir option is ignored + +## 2.2.2 - 2018-12-20 +### Fixed +* Handle short lived files in monitor mode +* Provide better log messages, less noise on temporary timeouts +* Deal with items that disappear during upload +* Deal with deleted move targets +* Reinitialize sync engine after three failed attempts +* Fix activation of dmd for docker builds +* Fix to check displayName rather than description for --get-O365-drive-id +* Fix checking of config file keys for validity +* Fix exception handling when missing parameter from usage option +### Added +* Notification support via libnotify +* Add very verbose (debug) mode by double -v -v +* Implement option --display-config + +## 2.2.1 - 2018-12-04 +### Fixed +* Gracefully handle connection errors in monitor mode +* Fix renaming of files when syncing +* Installation of doc files, addition of man page +* Adjust timeout values for libcurl +* Continue in monitor mode when sync timed out +* Fix unreachable statements +* Update Makefile to better support packaging +* Allow starting offline in monitor mode +### Added +* Implement --get-O365-drive-id to get correct SharePoint Shared Library (#248) +* Docker buildfiles for onedrive service (#262) + +## 2.2.0 - 2018-11-24 +### Fixed +* Updated client to output additional logging when debugging +* Resolve database assertion failure due to authentication +* Resolve unable to create folders on shared OneDrive Personal accounts +### Added +* Implement feature request to Sync from Microsoft SharePoint +* Implement feature request to specify a logging directory if logging is enabled +### Changed +* Change '--download' to '--download-only' to align with '--upload-only' +* Change logging so that logging to a separate file is no longer the default + +## 2.1.6 - 2018-11-15 +### Fixed +* Updated HTTP/2 transport handling when using curl 7.62.0 for session uploads +### Added +* Added PKGBUILD for makepkg for building packages under Arch Linux + +## 2.1.5 - 2018-11-11 +### Fixed +* Resolve 'Key not found: path' when syncing from some shared folders due to OneDrive API change +* Resolve to only upload changes on remote folder if the item is in the database - dont assert if false +* Resolve files will not download or upload when using curl 7.62.0 due to HTTP/2 being set as default for all curl operations +* Resolve to handle HTTP request returned status code 412 (Precondition Failed) for session uploads to OneDrive Personal Accounts +* Resolve unable to remove '~/.config/onedrive/resume_upload: No such file or directory' if there is a session upload error and the resume file does not get created +* Resolve handling of response codes when using 2 different systems when using '--upload-only' but the same OneDrive account and uploading the same filename to the same location +### Updated +* Updated Travis CI building on LDC v1.11.0 for ARMHF builds +* Updated Makefile to use 'install -D -m 644' rather than 'cp -raf' +* Updated default config to be aligned to code defaults + +## 2.1.4 - 2018-10-10 +### Fixed +* Resolve syncing of OneDrive Personal Shared Folders due to OneDrive API change +* Resolve incorrect systemd installation location(s) in Makefile + +## 2.1.3 - 2018-10-04 +### Fixed +* Resolve File download fails if the file is marked as malware in OneDrive +* Resolve high CPU usage when running in monitor mode +* Resolve how default path is set when running under systemd on headless systems +* Resolve incorrectly nested configDir in X11 systems +* Resolve Key not found: driveType +* Resolve to validate filename length before download to conform with Linux FS limits +* Resolve file handling to look for HTML ASCII codes which will cause uploads to fail +* Resolve Key not found: expirationDateTime on session resume +### Added +* Update Travis CI building to test build on ARM64 + +## 2.1.2 - 2018-08-27 +### Fixed +* Resolve skipping of symlinks in monitor mode +* Resolve Gateway Timeout - JSONValue is not an object +* Resolve systemd/user is not supported on CentOS / RHEL +* Resolve HTTP request returned status code 429 (Too Many Requests) +* Resolve handling of maximum path length calculation +* Resolve 'The parent item is not in the local database' +* Resolve Correctly handle file case sensitivity issues in same folder +* Update unit files documentation link + +## 2.1.1 - 2018-08-14 +### Fixed +* Fix handling no remote delete of remote directories when using --no-remote-delete +* Fix handling of no permission to access a local file / corrupt local file +* Fix application crash when unable to access login.microsoft.com upon application startup +### Added +* Build instructions for openSUSE Leap 15.0 + +## 2.1.0 - 2018-08-10 +### Fixed +* Fix handling of database exit scenarios when there is zero disk space left on drive where the items database resides +* Fix handling of incorrect database permissions +* Fix handling of different database versions to automatically re-create tables if version mis-match +* Fix handling timeout when accessing the Microsoft OneDrive Service +* Fix localFileModifiedTime to not use fraction seconds +### Added +* Implement Feature: Add a progress bar for large uploads & downloads +* Implement Feature: Make checkinterval for monitor configurable +* Implement Feature: Upload Only Option that does not perform remote delete +* Implement Feature: Add ability to skip symlinks +* Add dependency, ebuild and build instructions for Gentoo distributions +### Changed +* Build instructions for x86, x86_64 and ARM32 platforms +* Travis CI files to automate building on x32, x64 and ARM32 architectures +* Travis CI files to test built application against valid, invalid and problem files from previous issues + +## 2.0.2 - 2018-07-18 +### Fixed +* Fix systemd service install for builds with DESTDIR defined +* Fix 'HTTP 412 - Precondition Failed' error handling +* Gracefully handle OneDrive account password change +* Update logic handling of --upload-only and --local-first + +## 2.0.1 - 2018-07-11 +### Fixed +* Resolve computeQuickXorHash generates a different hash when files are > 64Kb + +## 2.0.0 - 2018-07-10 +### Fixed +* Resolve conflict resolution issue during syncing - the client does not handle conflicts very well & keeps on adding the hostname to files +* Resolve skilion #356 by adding additional check for 409 response from OneDrive +* Resolve multiple versions of file shown on website after single upload +* Resolve to gracefully fail when 'onedrive' process cannot get exclusive database lock +* Resolve 'Key not found: fileSystemInfo' when then item is a remote item (OneDrive Personal) +* Resolve skip_file config entry needs to be checked for any characters to escape +* Resolve Microsoft Naming Convention not being followed correctly +* Resolve Error when trying to upload a file with weird non printable characters present +* Resolve Crash if file is locked by online editing (status code 423) +* Resolve Resolve compilation issue with dmd-2.081.0 +* Resolve skip_file configuration doesn't handle spaces or specified directory paths +### Added +* Implement Feature: Add a flag to detect when the sync-folder is missing +* Implement Travis CI for code testing +### Changed +* Update Makefile to use DESTDIR variables +* Update OneDrive Business maximum path length from 256 to 400 +* Update OneDrive Business allowed characters for files and folders +* Update sync_dir handling to use the absolute path for setting parameter to something other than ~/OneDrive via config file or command line +* Update Fedora build instructions + +## 1.1.2 - 2018-05-17 +### Fixed +* Fix 4xx errors including (412 pre-condition, 409 conflict) +* Fix Key not found: lastModifiedDateTime (OneDrive API change) +* Fix configuration directory not found when run via init.d +* Fix skilion Issues #73, #121, #132, #224, #257, #294, #295, #297, #298, #300, #306, #315, #320, #329, #334, #337, #341 +### Added +* Add logging - log client activities to a file (/var/log/onedrive/%username%.onedrive.log or ~/onedrive.log) +* Add https debugging as a flag +* Add `--synchronize` to prevent from syncing when just blindly running the application +* Add individual folder sync +* Add sync from local directory first rather than download first then upload +* Add upload long path check +* Add upload only +* Add check for max upload file size before attempting upload +* Add systemd unit files for single & multi user configuration +* Add init.d file for older init.d based services +* Add Microsoft naming conventions and namespace validation for items that will be uploaded +* Add remaining free space counter at client initialisation to avoid out of space upload issue +* Add large file upload size check to align to OneDrive file size limitations +* Add upload file size validation & retry if does not match +* Add graceful handling of some fatal errors (OneDrive 5xx error handling) + +## Unreleased - 2018-02-19 +### Fixed +* Crash when the delta link is expired +### Changed +* Disabled buffering on stdout + +## 1.1.1 - 2018-01-20 +### Fixed +* Wrong regex for parsing authentication uri + +## 1.1.0 - 2018-01-19 +### Added +* Support for shared folders (OneDrive Personal only) +* `--download` option to only download changes +* `DC` variable in Makefile to chose the compiler +### Changed +* Print logs on stdout instead of stderr +* Improve log messages + +## 1.0.1 - 2017-08-01 +### Added +* `--syncdir` option +### Changed +* `--version` output simplified +* Updated README +### Fixed +* Fix crash caused by remotely deleted and recreated directories + +## 1.0.0 - 2017-07-14 +### Added +* `--version` option diff --git a/config b/config new file mode 100644 index 00000000..ebbbb638 --- /dev/null +++ b/config @@ -0,0 +1,57 @@ +# Configuration for OneDrive Linux Client +# This file contains the list of supported configuration fields +# with their default values. +# All values need to be enclosed in quotes +# When changing a config option below, remove the '#' from the start of the line +# For explanations of all config options below see docs/usage.md or the man page. +# +# sync_dir = "~/OneDrive" +# skip_file = "~*|.~*|*.tmp" +# monitor_interval = "300" +# skip_dir = "" +# log_dir = "/var/log/onedrive/" +# drive_id = "" +# upload_only = "false" +# check_nomount = "false" +# check_nosync = "false" +# download_only = "false" +# disable_notifications = "false" +# disable_upload_validation = "false" +# enable_logging = "false" +# force_http_11 = "false" +# local_first = "false" +# no_remote_delete = "false" +# skip_symlinks = "false" +# debug_https = "false" +# skip_dotfiles = "false" +# skip_size = "1000" +# dry_run = "false" +# min_notify_changes = "5" +# monitor_log_frequency = "6" +# monitor_fullscan_frequency = "12" +# sync_root_files = "false" +# classify_as_big_delete = "1000" +# user_agent = "" +# remove_source_files = "false" +# skip_dir_strict_match = "false" +# application_id = "" +# resync = "false" +# resync_auth = "false" +# bypass_data_preservation = "false" +# azure_ad_endpoint = "" +# azure_tenant_id = "common" +# sync_business_shared_folders = "false" +# sync_dir_permissions = "700" +# sync_file_permissions = "600" +# rate_limit = "131072" +# operation_timeout = "3600" +# webhook_enabled = "false" +# webhook_public_url = "" +# webhook_listening_host = "" +# webhook_listening_port = "8888" +# webhook_expiration_interval = "86400" +# webhook_renewal_interval = "43200" +# space_reservation = "50" +# display_running_config = "false" +# read_only_auth_scope = "false" +# cleanup_local_files = "false" diff --git a/docs/advanced-usage.md b/docs/advanced-usage.md new file mode 100644 index 00000000..2701909d --- /dev/null +++ b/docs/advanced-usage.md @@ -0,0 +1,302 @@ +# Advanced Configuration of the OneDrive Free Client +This document covers the following scenarios: +* [Configuring the client to use multiple OneDrive accounts / configurations](#configuring-the-client-to-use-multiple-onedrive-accounts--configurations) +* [Configuring the client to use multiple OneDrive accounts / configurations using Docker](#configuring-the-client-to-use-multiple-onedrive-accounts--configurations-using-docker) +* [Configuring the client for use in dual-boot (Windows / Linux) situations](#configuring-the-client-for-use-in-dual-boot-windows--linux-situations) +* [Configuring the client for use when 'sync_dir' is a mounted directory](#configuring-the-client-for-use-when-sync_dir-is-a-mounted-directory) +* [Upload data from the local ~/OneDrive folder to a specific location on OneDrive](#upload-data-from-the-local-onedrive-folder-to-a-specific-location-on-onedrive) + +## Configuring the client to use multiple OneDrive accounts / configurations +Essentially, each OneDrive account or SharePoint Shared Library which you require to be synced needs to have its own and unique configuration, local sync directory and service files. To do this, the following steps are needed: +1. Create a unique configuration folder for each onedrive client configuration that you need +2. Copy to this folder a copy of the default configuration file +3. Update the default configuration file as required, changing the required minimum config options and any additional options as needed to support your multi-account configuration +4. Authenticate the client using the new configuration directory +5. Test the configuration using '--display-config' and '--dry-run' +6. Sync the OneDrive account data as required using `--synchronize` or `--monitor` +7. Configure a unique systemd service file for this account configuration + +### 1. Create a unique configuration folder for each onedrive client configuration that you need +Make the configuration folder as required for this new configuration, for example: +```text +mkdir ~/.config/my-new-config +``` + +### 2. Copy to this folder a copy of the default configuration file +Copy to this folder a copy of the default configuration file by downloading this file from GitHub and saving this file in the directory created above: +```text +wget https://raw.githubusercontent.com/abraunegg/onedrive/master/config -O ~/.config/my-new-config/config +``` + +### 3. Update the default configuration file +The following config options *must* be updated to ensure that individual account data is not cross populated with other OneDrive accounts or other configurations: +* sync_dir + +Other options that may require to be updated, depending on the OneDrive account that is being configured: +* drive_id +* application_id +* sync_business_shared_folders +* skip_dir +* skip_file +* Creation of a 'sync_list' file if required +* Creation of a 'business_shared_folders' file if required + +### 4. Authenticate the client +Authenticate the client using the specific configuration file: +```text +onedrive --confdir="~/.config/my-new-config" +``` +You will be asked to open a specific URL by using your web browser where you will have to login into your Microsoft Account and give the application the permission to access your files. After giving permission to the application, you will be redirected to a blank page. Copy the URI of the blank page into the application. +```text +[user@hostname ~]$ onedrive --confdir="~/.config/my-new-config" +Configuration file successfully loaded +Configuring Global Azure AD Endpoints +Authorize this app visiting: + +https://..... + +Enter the response uri: + +``` + +### 5. Display and Test the configuration +Test the configuration using '--display-config' and '--dry-run'. By doing so, this allows you to test any configuration that you have currently made, enabling you to fix this configuration before using the configuration. + +#### Display the configuration +```text +onedrive --confdir="~/.config/my-new-config" --display-config +``` + +#### Test the configuration by performing a dry-run +```text +onedrive --confdir="~/.config/my-new-config" --synchronize --verbose --dry-run +``` + +If both of these operate as per your expectation, the configuration of this client setup is complete and validated. If not, amend your configuration as required. + +### 6. Sync the OneDrive account data as required +Sync the data for the new account configuration as required: +```text +onedrive --confdir="~/.config/my-new-config" --synchronize --verbose +``` +or +```text +onedrive --confdir="~/.config/my-new-config" --monitor --verbose +``` + +* `--synchronize` does a one-time sync +* `--monitor` keeps the application running and monitoring for changes both local and remote + +### 7. Automatic syncing of new OneDrive configuration +In order to automatically start syncing your OneDrive accounts, you will need to create a service file for each account. From the applicable 'systemd folder' where the applicable systemd service file exists: +* RHEL / CentOS: `/usr/lib/systemd/system` +* Others: `/usr/lib/systemd/user` and `/lib/systemd/system` + +### Step1: Create a new systemd service file +#### Red Hat Enterprise Linux, CentOS Linux +Copy the required service file to a new name: +```text +sudo cp /usr/lib/systemd/system/onedrive.service /usr/lib/systemd/system/onedrive-my-new-config +``` +or +```text +sudo cp /usr/lib/systemd/system/onedrive@.service /usr/lib/systemd/system/onedrive-my-new-config@.service +``` + +#### Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +Copy the required service file to a new name: +```text +sudo cp /usr/lib/systemd/user/onedrive.service /usr/lib/systemd/user/onedrive-my-new-config.service +``` +or +```text +sudo cp /lib/systemd/system/onedrive@.service /lib/systemd/system/onedrive-my-new-config@.service +``` + +### Step 2: Edit new systemd service file +Edit the new systemd file, updating the line beginning with `ExecStart` so that the confdir mirrors the one you used above: +```text +ExecStart=/usr/local/bin/onedrive --monitor --confdir="/full/path/to/config/dir" +``` + +Example: +```text +ExecStart=/usr/local/bin/onedrive --monitor --confdir="/home/myusername/.config/my-new-config" +``` + +**Note:** When running the client manually, `--confdir="~/.config/......` is acceptable. In a systemd configuration file, the full path must be used. The `~` must be expanded. + +### Step 3: Enable the new systemd service +Once the file is correctly editied, you can enable the new systemd service using the following commands. + +#### Red Hat Enterprise Linux, CentOS Linux +```text +systemctl enable onedrive-my-new-config +systemctl start onedrive-my-new-config +``` + +#### Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +```text +systemctl --user enable onedrive-my-new-config +systemctl --user start onedrive-my-new-config +``` +or +```text +systemctl --user enable onedrive-my-new-config@myusername.service +systemctl --user start onedrive-my-new-config@myusername.service +``` + +### Step 4: Viewing systemd status and logs for the custom service +#### Viewing systemd service status - Red Hat Enterprise Linux, CentOS Linux +```text +systemctl status onedrive-my-new-config +``` + +#### Viewing systemd service status - Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +```text +systemctl --user status onedrive-my-new-config +``` + +#### Viewing journalctl systemd logs - Red Hat Enterprise Linux, CentOS Linux +```text +journalctl --unit=onedrive-my-new-config -f +``` + +#### Viewing journalctl systemd logs - Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +```text +journalctl --user --unit=onedrive-my-new-config -f +``` + +### Step 5: (Optional) Run custom systemd service at boot without user login +In some cases it may be desirable for the systemd service to start without having to login as your 'user' + +All the systemd steps above that utilise the `--user` option, will run the systemd service as your particular user. As such, the systemd service will not start unless you actually login to your system. + +To avoid this issue, you need to reconfigure your 'user' account so that the systemd services you have created will startup without you having to login to your system: +```text +loginctl enable-linger +``` + +Example: +```text +alex@ubuntu-headless:~$ loginctl enable-linger alex +``` + +Repeat these steps for each OneDrive new account that you wish to use. + +## Configuring the client to use multiple OneDrive accounts / configurations using Docker +In some situations it may be desirable to run multiple Docker containers at the same time, each with their own configuration. + +To run the Docker container successfully, it needs two unique Docker volumes to operate: +* Your configuration Docker volumes +* Your data Docker volume + +When running multiple Docker containers, this is no different - each Docker container must have it's own configuration and data volume. + +### High level steps: +1. Create the required unique Docker volumes for the configuration volume +2. Create the required unique local path used for the Docker data volume +3. Start the multiple Docker containers with the required configuration for each container + +#### Create the required unique Docker volumes for the configuration volume +Create the required unique Docker volumes for the configuration volume(s): +```text +docker volume create onedrive_conf_sharepoint_site1 +docker volume create onedrive_conf_sharepoint_site2 +docker volume create onedrive_conf_sharepoint_site3 +... +docker volume create onedrive_conf_sharepoint_site50 +``` + +#### Create the required unique local path used for the Docker data volume +Create the required unique local path used for the Docker data volume +```text +mkdir -p /use/full/local/path/no/tilda/SharePointSite1 +mkdir -p /use/full/local/path/no/tilda/SharePointSite2 +mkdir -p /use/full/local/path/no/tilda/SharePointSite3 +... +mkdir -p /use/full/local/path/no/tilda/SharePointSite50 +``` + +#### Start the Docker container with the required configuration (example) +```text +docker run -it --name onedrive -v onedrive_conf_sharepoint_site1:/onedrive/conf -v "/use/full/local/path/no/tilda/SharePointSite1:/onedrive/data" driveone/onedrive:latest +docker run -it --name onedrive -v onedrive_conf_sharepoint_site2:/onedrive/conf -v "/use/full/local/path/no/tilda/SharePointSite2:/onedrive/data" driveone/onedrive:latest +docker run -it --name onedrive -v onedrive_conf_sharepoint_site3:/onedrive/conf -v "/use/full/local/path/no/tilda/SharePointSite3:/onedrive/data" driveone/onedrive:latest +... +docker run -it --name onedrive -v onedrive_conf_sharepoint_site50:/onedrive/conf -v "/use/full/local/path/no/tilda/SharePointSite50:/onedrive/data" driveone/onedrive:latest +``` + +#### TIP +To avoid 're-authenticating' and 'authorising' each individual Docker container, if all the Docker containers are using the 'same' OneDrive credentials, you can re-use the 'refresh_token' from one Docker container to another by copying this file to the configuration Docker volume of each Docker container. + +If the account credentials are different .. you will need to re-authenticate each Docker container individually. + +## Configuring the client for use in dual-boot (Windows / Linux) situations +When dual booting Windows and Linux, depending on the Windows OneDrive account configuration, the 'Files On-Demand' option may be enabled when running OneDrive within your Windows environment. + +When this option is enabled in Windows, if you are sharing this location between your Windows and Linux systems, all files will be a 0 byte link, and cannot be used under Linux. + +To fix the problem of windows turning all files (that should be kept offline) into links, you have to uncheck a specific option in the onedrive settings window. The option in question is `Save space and download files as you use them`. + +To find this setting, open the onedrive pop-up window from the taskbar, click "Help & Settings" > "Settings". This opens a new window. Go to the tab "Settings" and look for the section "Files On-Demand". + +After unchecking the option and clicking "OK", the Windows OneDrive client should restart itself and start actually downloading your files so they will truely be available on your disk when offline. These files will then be fully accessible under Linux and the Linux OneDrive client. + +| OneDrive Personal | Onedrive Business
SharePoint | +|---|---| +| ![Uncheck-Personal](./images/personal-files-on-demand.png) | ![Uncheck-Business](./images/business-files-on-demand.png) | + +## Configuring the client for use when 'sync_dir' is a mounted directory +In some environments, your setup might be that your configured 'sync_dir' is pointing to another mounted file system - a NFS|CIFS location, an external drive (USB stuc, eSATA etc). As such, you configure your 'sync_dir' as follows: +```text +sync_dir = "/path/to/mountpoint/OneDrive" +``` + +The issue here is - how does the client react if the mount point gets removed - network loss, device removal? + +The client has zero knowledge of any event that causes a mountpoint to become unavailable, thus, the client (if you are running as a service) will assume that you deleted the files, thus, will go ahead and delete all your files on OneDrive. This is most certainly an undesirable action. + +There are a few options here which you can configure in your 'config' file to assist you to prevent this sort of item from occuring: +1. classify_as_big_delete +2. check_nomount +3. check_nosync + +**Note:** Before making any change to your configuration, stop any sync process & stop any onedrive systemd service from running. + +### classify_as_big_delete +By default, this uses a value of 1000 files|folders. An undesirable unmount if you have more than 1000 files, this default level will prevent the client from executing the online delete. Modify this value up or down as desired + +### check_nomount & check_nosync +These two options are really the right safe guards to use. + +In your 'mount point', *before* you mount your external folder|device, create empty `.nosync` file, so that this is the *only* file present in the mount location before you mount your data to your mount point. When you mount your data, this '.nosync' file will not be visible, but, if the device you are mounting goes away - this '.nosync' file is the only file visible. + +Next, in your 'config' file, configure the following options: `check_nomount = "true"` and `check_nosync = "true"` + +What this will do is tell the client, if at *any* point you see this file - stop syncing - thus, protecting your online data from being deleted by the mounted device being suddenly unavailable. + +After making this sort of change - test with `--dry-run` so you can see the impacts of your mount point being unavailable, and how the client is now reacting. Once you are happy with how the system will react, restart your sync processes. + + +## Upload data from the local ~/OneDrive folder to a specific location on OneDrive +In some environments, you may not want your local ~/OneDrive folder to be uploaded directly to the root of your OneDrive account online. + +Unfortunatly, the OneDrive API lacks any facility to perform a re-direction of data during upload. + +The workaround for this is to structure your local filesystem and reconfigure your client to achieve the desired goal. + +### High level steps: +1. Create a new folder, for example `/opt/OneDrive` +2. Configure your application config 'sync_dir' to look at this folder +3. Inside `/opt/OneDrive` create the folder you wish to sync the data online to, for example: `/opt/OneDrive/RemoteOnlineDestination` +4. Configure the application to only sync `/opt/OneDrive/RemoteDestination` via 'sync_list' +5. Symbolically link `~/OneDrive` -> `/opt/OneDrive/RemoteOnlineDestination` + +### Outcome: +* Your `~/OneDrive` will look / feel as per normal +* The data will be stored online under `/RemoteOnlineDestination` + +### Testing: +* Validate your configuration with `onedrive --display-config` +* Test your configuration with `onedrive --dry-run` diff --git a/docs/application-security.md b/docs/application-security.md new file mode 100644 index 00000000..7c22c4f1 --- /dev/null +++ b/docs/application-security.md @@ -0,0 +1,97 @@ +# OneDrive Client for Linux Application Security +This document details the following information: + +* Why is this application an 'unverified publisher'? +* Application Security and Permission Scopes +* How to change Permission Scopes +* How to review your existing application access consent + +## Why is this application an 'unverified publisher'? +Publisher Verification, as per the Microsoft [process](https://learn.microsoft.com/en-us/azure/active-directory/develop/publisher-verification-overview) has actually been configured, and, actually has been verified! + +### Verified Publisher Configuration Evidence +As per the image below, the Azure portal shows that the 'Publisher Domain' has actually been verified: +![confirmed_verified_publisher](./images/confirmed_verified_publisher.jpg) + +* The 'Publisher Domain' is: https://abraunegg.github.io/ +* The required 'Microsoft Identity Association' is: https://abraunegg.github.io/.well-known/microsoft-identity-association.json + +## Application Security and Permission Scopes +There are 2 main components regarding security for this application: +* Azure Application Permissions +* User Authentication Permissions + +Keeping this in mind, security options should follow the security principal of 'least privilege': +> The principle that a security architecture should be designed so that each entity +> is granted the minimum system resources and authorizations that the entity needs +> to perform its function. + +Reference: [https://csrc.nist.gov/glossary/term/least_privilege](https://csrc.nist.gov/glossary/term/least_privilege) + +As such, the following API permissions are used by default: + +### Default Azure Application Permissions + +| API / Permissions name | Type | Description | Admin consent required | +|---|---|---|---| +| Files.Read | Delegated | Have read-only access to user files | No | +| Files.Read.All | Delegated | Have read-only access to all files user can access | No | +| Sites.Read.All | Delegated | Have read-only access to all items in all site collections | No | +| offline_access | Delegated | Maintain access to data you have given it access to | No | + +![default_authentication_scopes](./images/default_authentication_scopes.jpg) + +### Default User Authentication Permissions + +When a user authenticates with Microsoft OneDrive, additional account permissions are provided by service to give the user specific access to their data. These are delegated permissions provided by the platform: + +| API / Permissions name | Type | Description | Admin consent required | +|---|---|---|---| +| Files.ReadWrite | Delegated | Have full access to user files | No | +| Files.ReadWrite.All | Delegated | Have full access to all files user can access | No | +| Sites.ReadWrite.All | Delegated | Have full access to all items in all site collections | No | +| offline_access | Delegated | Maintain access to data you have given it access to | No | + +When these delegated API permissions are combined, these provide the effective authentication scope for the OneDrive Client for Linux to access your data. The resulting effective 'default' permissions will be: + +| API / Permissions name | Type | Description | Admin consent required | +|---|---|---|---| +| Files.ReadWrite | Delegated | Have full access to user files | No | +| Files.ReadWrite.All | Delegated | Have full access to all files user can access | No | +| Sites.ReadWrite.All | Delegated | Have full access to all items in all site collections | No | +| offline_access | Delegated | Maintain access to data you have given it access to | No | + +These 'default' permissions will allow the OneDrive Client for Linux to read, write and delete data associated with your OneDrive Account. + +## Configuring read-only access to your OneDrive data +In some situations, it may be desirable to configure the OneDrive Client for Linux totally in read-only operation. + +To change the application to 'read-only' access, add the following to your configuration file: +```text +read_only_auth_scope = "true" +``` +This will change the user authentication scope request to use read-only access. + +**Note:** When changing this value, you *must* re-authenticate the client using the `--reauth` option to utilise the change in authentication scopes. + +When using read-only authentication scopes, the uploading of any data or local change to OneDrive will fail with the following error: +``` +2022-Aug-06 13:16:45.3349625 ERROR: Microsoft OneDrive API returned an error with the following message: +2022-Aug-06 13:16:45.3351661 Error Message: HTTP request returned status code 403 (Forbidden) +2022-Aug-06 13:16:45.3352467 Error Reason: Access denied +2022-Aug-06 13:16:45.3352838 Error Timestamp: 2022-06-12T13:16:45 +2022-Aug-06 13:16:45.3353171 API Request ID: +``` + +As such, it is also advisable for you to add the following to your configuration file so that 'uploads' are prevented: +```text +download_only = "true" +``` + +**Important:** Additionally when using 'read_only_auth_scope' you also will need to remove your existing application access consent otherwise old authentication consent will be valid and will be used. This will mean the application will technically have the consent to upload data. See below on how to remove your prior application consent. + +## Reviewing your existing application access consent + +To review your existing application access consent, you need to access the following URL: https://account.live.com/consent/Manage + +From here, you are able to review what applications have been given what access to your data, and remove application access as required. diff --git a/docs/build-rpm-howto.md b/docs/build-rpm-howto.md new file mode 100644 index 00000000..5439c366 --- /dev/null +++ b/docs/build-rpm-howto.md @@ -0,0 +1,379 @@ +# RPM Package Build Process +The instuctions below have been tested on the following systems: +* CentOS 7 x86_64 +* CentOS 8 x86_64 + +These instructions should also be applicable for RedHat & Fedora platforms, or any other RedHat RPM based distribution. + +## Prepare Package Development Environment (CentOS 7, 8) +Install the following dependencies on your build system: +```text +sudo yum groupinstall -y 'Development Tools' +sudo yum install -y libcurl-devel +sudo yum install -y sqlite-devel +sudo yum install -y libnotify-devel +sudo yum install -y wget +sudo yum install -y http://downloads.dlang.org/releases/2.x/2.088.0/dmd-2.088.0-0.fedora.x86_64.rpm +mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS} +``` + +## Build RPM from spec file +Build the RPM from the provided spec file: +```text +wget https://github.com/abraunegg/onedrive/archive/refs/tags/v2.4.22.tar.gz -O ~/rpmbuild/SOURCES/v2.4.22.tar.gz +wget https://raw.githubusercontent.com/abraunegg/onedrive/master/contrib/spec/onedrive.spec.in -O ~/rpmbuild/SPECS/onedrive.spec +rpmbuild -ba ~/rpmbuild/SPECS/onedrive.spec +``` + +## RPM Build Example Results +Below are example output results of building, installing and running the RPM package on the respective platforms: + +### CentOS 7 +```text +[alex@localhost ~]$ rpmbuild -ba ~/rpmbuild/SPECS/onedrive.spec +Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.wi6Tdz ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd /home/alex/rpmbuild/BUILD ++ rm -rf onedrive-2.4.15 ++ /usr/bin/tar -xf - ++ /usr/bin/gzip -dc /home/alex/rpmbuild/SOURCES/v2.4.15.tar.gz ++ STATUS=0 ++ '[' 0 -ne 0 ']' ++ cd onedrive-2.4.15 ++ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w . ++ exit 0 +Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.dyeEuM ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd onedrive-2.4.15 ++ CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' ++ export CFLAGS ++ CXXFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' ++ export CXXFLAGS ++ FFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -I/usr/lib64/gfortran/modules' ++ export FFLAGS ++ FCFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -I/usr/lib64/gfortran/modules' ++ export FCFLAGS ++ LDFLAGS='-Wl,-z,relro ' ++ export LDFLAGS ++ '[' 1 == 1 ']' ++ '[' x86_64 == ppc64le ']' +++ find . -name config.guess -o -name config.sub ++ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info +configure: WARNING: unrecognized options: --disable-dependency-tracking +checking for a BSD-compatible install... /usr/bin/install -c +checking for x86_64-redhat-linux-gnu-pkg-config... no +checking for pkg-config... /usr/bin/pkg-config +checking pkg-config is at least version 0.9.0... yes +checking for dmd... dmd +checking version of D compiler... 2.087.0 +checking for curl... yes +checking for sqlite... yes +configure: creating ./config.status +config.status: creating Makefile +config.status: creating contrib/pacman/PKGBUILD +config.status: creating contrib/spec/onedrive.spec +config.status: creating onedrive.1 +config.status: creating contrib/systemd/onedrive.service +config.status: creating contrib/systemd/onedrive@.service +configure: WARNING: unrecognized options: --disable-dependency-tracking ++ make +if [ -f .git/HEAD ] ; then \ + git describe --tags > version ; \ +else \ + echo v2.4.15 > version ; \ +fi +dmd -w -g -O -J. -L-lcurl -L-lsqlite3 -L-ldl src/config.d src/itemdb.d src/log.d src/main.d src/monitor.d src/onedrive.d src/qxor.d src/selective.d src/sqlite.d src/sync.d src/upload.d src/util.d src/progress.d src/arsd/cgi.d -ofonedrive ++ exit 0 +Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.L3JbHy ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ '[' /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64 '!=' / ']' ++ rm -rf /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64 +++ dirname /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64 ++ mkdir -p /home/alex/rpmbuild/BUILDROOT ++ mkdir /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64 ++ cd onedrive-2.4.15 ++ /usr/bin/make install DESTDIR=/home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64 PREFIX=/home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64 +/usr/bin/install -c -D onedrive /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/bin/onedrive +/usr/bin/install -c -D onedrive.1 /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/man/man1/onedrive.1 +/usr/bin/install -c -D -m 644 contrib/logrotate/onedrive.logrotate /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/etc/logrotate.d/onedrive +mkdir -p /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/doc/onedrive +/usr/bin/install -c -D -m 644 README.md config LICENSE CHANGELOG.md docs/Docker.md docs/INSTALL.md docs/SharePoint-Shared-Libraries.md docs/USAGE.md docs/BusinessSharedFolders.md docs/advanced-usage.md /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/doc/onedrive +/usr/bin/install -c -d -m 0755 /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/lib/systemd/user /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/lib/systemd/system +/usr/bin/install -c -m 0644 contrib/systemd/onedrive@.service /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/lib/systemd/system +/usr/bin/install -c -m 0644 contrib/systemd/onedrive.service /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/lib/systemd/system ++ /usr/lib/rpm/check-buildroot ++ /usr/lib/rpm/redhat/brp-compress ++ /usr/lib/rpm/redhat/brp-strip /usr/bin/strip ++ /usr/lib/rpm/redhat/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump ++ /usr/lib/rpm/redhat/brp-strip-static-archive /usr/bin/strip ++ /usr/lib/rpm/brp-python-bytecompile /usr/bin/python 1 ++ /usr/lib/rpm/redhat/brp-python-hardlink ++ /usr/lib/rpm/redhat/brp-java-repack-jars +Processing files: onedrive-2.4.15-1.el7.x86_64 +Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.cpSXho ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd onedrive-2.4.15 ++ DOCDIR=/home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/doc/onedrive-2.4.15 ++ export DOCDIR ++ /usr/bin/mkdir -p /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/doc/onedrive-2.4.15 ++ cp -pr README.md /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/doc/onedrive-2.4.15 ++ cp -pr LICENSE /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/doc/onedrive-2.4.15 ++ cp -pr CHANGELOG.md /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64/usr/share/doc/onedrive-2.4.15 ++ exit 0 +Provides: config(onedrive) = 2.4.15-1.el7 onedrive = 2.4.15-1.el7 onedrive(x86-64) = 2.4.15-1.el7 +Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 +Requires(post): systemd +Requires(preun): systemd +Requires(postun): systemd +Requires: ld-linux-x86-64.so.2()(64bit) ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) libc.so.6()(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.15)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3.2)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libc.so.6(GLIBC_2.6)(64bit) libc.so.6(GLIBC_2.8)(64bit) libc.so.6(GLIBC_2.9)(64bit) libcurl.so.4()(64bit) libdl.so.2()(64bit) libdl.so.2(GLIBC_2.2.5)(64bit) libgcc_s.so.1()(64bit) libgcc_s.so.1(GCC_3.0)(64bit) libgcc_s.so.1(GCC_4.2.0)(64bit) libm.so.6()(64bit) libm.so.6(GLIBC_2.2.5)(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) libpthread.so.0(GLIBC_2.3.2)(64bit) libpthread.so.0(GLIBC_2.3.4)(64bit) librt.so.1()(64bit) librt.so.1(GLIBC_2.2.5)(64bit) libsqlite3.so.0()(64bit) rtld(GNU_HASH) +Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el7.x86_64 +Wrote: /home/alex/rpmbuild/SRPMS/onedrive-2.4.15-1.el7.src.rpm +Wrote: /home/alex/rpmbuild/RPMS/x86_64/onedrive-2.4.15-1.el7.x86_64.rpm +Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.nWoW33 ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd onedrive-2.4.15 ++ exit 0 +[alex@localhost ~]$ sudo yum -y install /home/alex/rpmbuild/RPMS/x86_64/onedrive-2.4.15-1.el7.x86_64.rpm +Loaded plugins: fastestmirror +Examining /home/alex/rpmbuild/RPMS/x86_64/onedrive-2.4.15-1.el7.x86_64.rpm: onedrive-2.4.15-1.el7.x86_64 +Marking /home/alex/rpmbuild/RPMS/x86_64/onedrive-2.4.15-1.el7.x86_64.rpm to be installed +Resolving Dependencies +--> Running transaction check +---> Package onedrive.x86_64 0:2.4.15-1.el7 will be installed +--> Finished Dependency Resolution + +Dependencies Resolved + +============================================================================================================================================================================================== + Package Arch Version Repository Size +============================================================================================================================================================================================== +Installing: + onedrive x86_64 2.4.15-1.el7 /onedrive-2.4.15-1.el7.x86_64 7.2 M + +Transaction Summary +============================================================================================================================================================================================== +Install 1 Package + +Total size: 7.2 M +Installed size: 7.2 M +Downloading packages: +Running transaction check +Running transaction test +Transaction test succeeded +Running transaction + Installing : onedrive-2.4.15-1.el7.x86_64 1/1 + Verifying : onedrive-2.4.15-1.el7.x86_64 1/1 + +Installed: + onedrive.x86_64 0:2.4.15-1.el7 + +Complete! +[alex@localhost ~]$ which onedrive +/usr/bin/onedrive +[alex@localhost ~]$ onedrive --version +onedrive v2.4.15 +[alex@localhost ~]$ onedrive --display-config +onedrive version = v2.4.15 +Config path = /home/alex/.config/onedrive +Config file found in config path = false +Config option 'check_nosync' = false +Config option 'sync_dir' = /home/alex/OneDrive +Config option 'skip_dir' = +Config option 'skip_file' = ~*|.~*|*.tmp +Config option 'skip_dotfiles' = false +Config option 'skip_symlinks' = false +Config option 'monitor_interval' = 300 +Config option 'min_notify_changes' = 5 +Config option 'log_dir' = /var/log/onedrive/ +Config option 'classify_as_big_delete' = 1000 +Config option 'upload_only' = false +Config option 'no_remote_delete' = false +Config option 'remove_source_files' = false +Config option 'sync_root_files' = false +Selective sync 'sync_list' configured = false +Business Shared Folders configured = false +[alex@localhost ~]$ +``` + +### CentOS 8 +```text +[alex@localhost ~]$ rpmbuild -ba ~/rpmbuild/SPECS/onedrive.spec +Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.UINFyE ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd /home/alex/rpmbuild/BUILD ++ rm -rf onedrive-2.4.15 ++ /usr/bin/gzip -dc /home/alex/rpmbuild/SOURCES/v2.4.15.tar.gz ++ /usr/bin/tar -xof - ++ STATUS=0 ++ '[' 0 -ne 0 ']' ++ cd onedrive-2.4.15 ++ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w . ++ exit 0 +Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.cX1WQa ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd onedrive-2.4.15 ++ CFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' ++ export CFLAGS ++ CXXFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' ++ export CXXFLAGS ++ FFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -I/usr/lib64/gfortran/modules' ++ export FFLAGS ++ FCFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -I/usr/lib64/gfortran/modules' ++ export FCFLAGS ++ LDFLAGS='-Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' ++ export LDFLAGS ++ '[' 1 = 1 ']' ++++ dirname ./configure +++ find . -name config.guess -o -name config.sub ++ '[' 1 = 1 ']' ++ '[' x '!=' 'x-Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld' ']' +++ find . -name ltmain.sh ++ ./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info +configure: WARNING: unrecognized options: --disable-dependency-tracking +checking for a BSD-compatible install... /usr/bin/install -c +checking for x86_64-redhat-linux-gnu-pkg-config... /usr/bin/x86_64-redhat-linux-gnu-pkg-config +checking pkg-config is at least version 0.9.0... yes +checking for dmd... dmd +checking version of D compiler... 2.087.0 +checking for curl... yes +checking for sqlite... yes +configure: creating ./config.status +config.status: creating Makefile +config.status: creating contrib/pacman/PKGBUILD +config.status: creating contrib/spec/onedrive.spec +config.status: creating onedrive.1 +config.status: creating contrib/systemd/onedrive.service +config.status: creating contrib/systemd/onedrive@.service +configure: WARNING: unrecognized options: --disable-dependency-tracking ++ make +if [ -f .git/HEAD ] ; then \ + git describe --tags > version ; \ +else \ + echo v2.4.15 > version ; \ +fi +dmd -w -g -O -J. -L-lcurl -L-lsqlite3 -L-ldl src/config.d src/itemdb.d src/log.d src/main.d src/monitor.d src/onedrive.d src/qxor.d src/selective.d src/sqlite.d src/sync.d src/upload.d src/util.d src/progress.d src/arsd/cgi.d -ofonedrive ++ exit 0 +Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.dNFPdx ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ '[' /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64 '!=' / ']' ++ rm -rf /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64 +++ dirname /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64 ++ mkdir -p /home/alex/rpmbuild/BUILDROOT ++ mkdir /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64 ++ cd onedrive-2.4.15 ++ /usr/bin/make install DESTDIR=/home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64 'INSTALL=/usr/bin/install -p' PREFIX=/home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64 +/usr/bin/install -p -D onedrive /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/bin/onedrive +/usr/bin/install -p -D onedrive.1 /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/man/man1/onedrive.1 +/usr/bin/install -p -D -m 644 contrib/logrotate/onedrive.logrotate /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/etc/logrotate.d/onedrive +mkdir -p /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/doc/onedrive +/usr/bin/install -p -D -m 644 README.md config LICENSE CHANGELOG.md docs/Docker.md docs/INSTALL.md docs/SharePoint-Shared-Libraries.md docs/USAGE.md docs/BusinessSharedFolders.md docs/advanced-usage.md /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/doc/onedrive +/usr/bin/install -p -d -m 0755 /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/lib/systemd/user /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/lib/systemd/system +/usr/bin/install -p -m 0644 contrib/systemd/onedrive@.service /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/lib/systemd/system +/usr/bin/install -p -m 0644 contrib/systemd/onedrive.service /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/lib/systemd/system ++ /usr/lib/rpm/check-buildroot ++ /usr/lib/rpm/redhat/brp-ldconfig +/sbin/ldconfig: Warning: ignoring configuration file that cannot be opened: /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/etc/ld.so.conf: No such file or directory ++ /usr/lib/rpm/brp-compress ++ /usr/lib/rpm/brp-strip /usr/bin/strip ++ /usr/lib/rpm/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump ++ /usr/lib/rpm/brp-strip-static-archive /usr/bin/strip ++ /usr/lib/rpm/brp-python-bytecompile 1 ++ /usr/lib/rpm/brp-python-hardlink ++ PYTHON3=/usr/libexec/platform-python ++ /usr/lib/rpm/redhat/brp-mangle-shebangs +Processing files: onedrive-2.4.15-1.el8.x86_64 +Executing(%doc): /bin/sh -e /var/tmp/rpm-tmp.TnFKbZ ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd onedrive-2.4.15 ++ DOCDIR=/home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/doc/onedrive ++ export LC_ALL=C ++ LC_ALL=C ++ export DOCDIR ++ /usr/bin/mkdir -p /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/doc/onedrive ++ cp -pr README.md /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/doc/onedrive ++ cp -pr LICENSE /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/doc/onedrive ++ cp -pr CHANGELOG.md /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64/usr/share/doc/onedrive ++ exit 0 +warning: File listed twice: /usr/share/doc/onedrive +warning: File listed twice: /usr/share/doc/onedrive/CHANGELOG.md +warning: File listed twice: /usr/share/doc/onedrive/LICENSE +warning: File listed twice: /usr/share/doc/onedrive/README.md +Provides: config(onedrive) = 2.4.15-1.el8 onedrive = 2.4.15-1.el8 onedrive(x86-64) = 2.4.15-1.el8 +Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1 +Requires(post): systemd +Requires(preun): systemd +Requires(postun): systemd +Requires: ld-linux-x86-64.so.2()(64bit) ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) libc.so.6()(64bit) libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.15)(64bit) libc.so.6(GLIBC_2.2.5)(64bit) libc.so.6(GLIBC_2.3.2)(64bit) libc.so.6(GLIBC_2.3.4)(64bit) libc.so.6(GLIBC_2.4)(64bit) libc.so.6(GLIBC_2.6)(64bit) libc.so.6(GLIBC_2.8)(64bit) libc.so.6(GLIBC_2.9)(64bit) libcurl.so.4()(64bit) libdl.so.2()(64bit) libdl.so.2(GLIBC_2.2.5)(64bit) libgcc_s.so.1()(64bit) libgcc_s.so.1(GCC_3.0)(64bit) libgcc_s.so.1(GCC_4.2.0)(64bit) libm.so.6()(64bit) libm.so.6(GLIBC_2.2.5)(64bit) libpthread.so.0()(64bit) libpthread.so.0(GLIBC_2.2.5)(64bit) libpthread.so.0(GLIBC_2.3.2)(64bit) libpthread.so.0(GLIBC_2.3.4)(64bit) librt.so.1()(64bit) librt.so.1(GLIBC_2.2.5)(64bit) libsqlite3.so.0()(64bit) rtld(GNU_HASH) +Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/alex/rpmbuild/BUILDROOT/onedrive-2.4.15-1.el8.x86_64 +Wrote: /home/alex/rpmbuild/SRPMS/onedrive-2.4.15-1.el8.src.rpm +Wrote: /home/alex/rpmbuild/RPMS/x86_64/onedrive-2.4.15-1.el8.x86_64.rpm +Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.FAMTFz ++ umask 022 ++ cd /home/alex/rpmbuild/BUILD ++ cd onedrive-2.4.15 ++ exit 0 +[alex@localhost ~]$ sudo yum -y install /home/alex/rpmbuild/RPMS/x86_64/onedrive-2.4.15-1.el8.x86_64.rpm +Last metadata expiration check: 0:04:07 ago on Fri 14 Jan 2022 14:22:13 EST. +Dependencies resolved. +============================================================================================================================================================================================== + Package Architecture Version Repository Size +============================================================================================================================================================================================== +Installing: + onedrive x86_64 2.4.15-1.el8 @commandline 1.5 M + +Transaction Summary +============================================================================================================================================================================================== +Install 1 Package + +Total size: 1.5 M +Installed size: 7.1 M +Downloading Packages: +Running transaction check +Transaction check succeeded. +Running transaction test +Transaction test succeeded. +Running transaction + Preparing : 1/1 + Installing : onedrive-2.4.15-1.el8.x86_64 1/1 + Running scriptlet: onedrive-2.4.15-1.el8.x86_64 1/1 + Verifying : onedrive-2.4.15-1.el8.x86_64 1/1 + +Installed: + onedrive-2.4.15-1.el8.x86_64 + +Complete! +[alex@localhost ~]$ which onedrive +/usr/bin/onedrive +[alex@localhost ~]$ onedrive --version +onedrive v2.4.15 +[alex@localhost ~]$ onedrive --display-config +onedrive version = v2.4.15 +Config path = /home/alex/.config/onedrive +Config file found in config path = false +Config option 'check_nosync' = false +Config option 'sync_dir' = /home/alex/OneDrive +Config option 'skip_dir' = +Config option 'skip_file' = ~*|.~*|*.tmp +Config option 'skip_dotfiles' = false +Config option 'skip_symlinks' = false +Config option 'monitor_interval' = 300 +Config option 'min_notify_changes' = 5 +Config option 'log_dir' = /var/log/onedrive/ +Config option 'classify_as_big_delete' = 1000 +Config option 'upload_only' = false +Config option 'no_remote_delete' = false +Config option 'remove_source_files' = false +Config option 'sync_root_files' = false +Selective sync 'sync_list' configured = false +Business Shared Folders configured = false +[alex@localhost ~]$ +``` diff --git a/docs/business-shared-folders.md b/docs/business-shared-folders.md new file mode 100644 index 00000000..3f042943 --- /dev/null +++ b/docs/business-shared-folders.md @@ -0,0 +1,192 @@ +# How to configure OneDrive Business Shared Folder Sync +## Application Version +Before reading this document, please ensure you are running application version [![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) or greater. Use `onedrive --version` to determine what application version you are using and upgrade your client if required. + +## Process Overview +Syncing OneDrive Business Shared Folders requires additional configuration for your 'onedrive' client: +1. List available shared folders to determine which folder you wish to sync & to validate that you have access to that folder +2. Create a new file called 'business_shared_folders' in your config directory which contains a list of the shared folders you wish to sync +3. Test the configuration using '--dry-run' +4. Sync the OneDrive Business Shared folders as required + +## Listing available OneDrive Business Shared Folders +List the available OneDrive Business Shared folders with the following command: +```text +onedrive --list-shared-folders +``` + This will return a listing of all OneDrive Business Shared folders which have been shared with you and by whom. This is important for conflict resolution: +```text +Initializing the Synchronization Engine ... + +Listing available OneDrive Business Shared Folders: +--------------------------------------- +Shared Folder: SharedFolder0 +Shared By: Firstname Lastname +--------------------------------------- +Shared Folder: SharedFolder1 +Shared By: Firstname Lastname +--------------------------------------- +Shared Folder: SharedFolder2 +Shared By: Firstname Lastname +--------------------------------------- +Shared Folder: SharedFolder0 +Shared By: Firstname Lastname (user@domain) +--------------------------------------- +Shared Folder: SharedFolder1 +Shared By: Firstname Lastname (user@domain) +--------------------------------------- +Shared Folder: SharedFolder2 +Shared By: Firstname Lastname (user@domain) +... +``` + +## Configuring OneDrive Business Shared Folders +1. Create a new file called 'business_shared_folders' in your config directory +2. On each new line, list the OneDrive Business Shared Folder you wish to sync +```text +[alex@centos7full onedrive]$ cat ~/.config/onedrive/business_shared_folders +# comment +Child Shared Folder +# Another comment +Top Level to Share +[alex@centos7full onedrive]$ +``` +3. Validate your configuration with `onedrive --display-config`: +```text +Configuration file successfully loaded +onedrive version = v2.4.3 +Config path = /home/alex/.config/onedrive-business/ +Config file found in config path = true +Config option 'check_nosync' = false +Config option 'sync_dir' = /home/alex/OneDriveBusiness +Config option 'skip_dir' = +Config option 'skip_file' = ~*|.~*|*.tmp +Config option 'skip_dotfiles' = false +Config option 'skip_symlinks' = false +Config option 'monitor_interval' = 300 +Config option 'min_notify_changes' = 5 +Config option 'log_dir' = /var/log/onedrive/ +Config option 'classify_as_big_delete' = 1000 +Config option 'sync_root_files' = false +Selective sync 'sync_list' configured = false +Business Shared Folders configured = true +business_shared_folders contents: +# comment +Child Shared Folder +# Another comment +Top Level to Share +``` + +## Performing a sync of OneDrive Business Shared Folders +Perform a standalone sync using the following command: `onedrive --synchronize --sync-shared-folders --verbose`: +```text +onedrive --synchronize --sync-shared-folders --verbose +Using 'user' Config Dir: /home/alex/.config/onedrive-business/ +Using 'system' Config Dir: +Configuration file successfully loaded +Initializing the OneDrive API ... +Configuring Global Azure AD Endpoints +Opening the item database ... +All operations will be performed in: /home/alex/OneDriveBusiness +Application version: v2.4.3 +Account Type: business +Default Drive ID: b!bO8V7s9SSk6r7mWHpIjURotN33W1W2tEv3OXV_oFIdQimEdOHR-1So7CqeT1MfHA +Default Root ID: 01WIXGO5V6Y2GOVW7725BZO354PWSELRRZ +Remaining Free Space: 1098316220277 +Fetching details for OneDrive Root +OneDrive Root exists in the database +Initializing the Synchronization Engine ... +Syncing changes from OneDrive ... +Applying changes of Path ID: 01WIXGO5V6Y2GOVW7725BZO354PWSELRRZ +Number of items from OneDrive to process: 0 +Attempting to sync OneDrive Business Shared Folders +Syncing this OneDrive Business Shared Folder: Child Shared Folder +OneDrive Business Shared Folder - Shared By: test user +Applying changes of Path ID: 01JRXHEZMREEB3EJVHNVHKNN454Q7DFXPR +Adding OneDrive root details for processing +Adding OneDrive folder details for processing +Adding 4 OneDrive items for processing from OneDrive folder +Adding 2 OneDrive items for processing from /Child Shared Folder/Cisco VDI Whitepaper +Adding 2 OneDrive items for processing from /Child Shared Folder/SMPP_Shared +Processing 11 OneDrive items to ensure consistent local state +Syncing this OneDrive Business Shared Folder: Top Level to Share +OneDrive Business Shared Folder - Shared By: test user (testuser@mynasau3.onmicrosoft.com) +Applying changes of Path ID: 01JRXHEZLRMXHKBYZNOBF3TQOPBXS3VZMA +Adding OneDrive root details for processing +Adding OneDrive folder details for processing +Adding 4 OneDrive items for processing from OneDrive folder +Adding 3 OneDrive items for processing from /Top Level to Share/10-Files +Adding 2 OneDrive items for processing from /Top Level to Share/10-Files/Cisco VDI Whitepaper +Adding 2 OneDrive items for processing from /Top Level to Share/10-Files/Images +Adding 8 OneDrive items for processing from /Top Level to Share/10-Files/Images/JPG +Adding 8 OneDrive items for processing from /Top Level to Share/10-Files/Images/PNG +Adding 2 OneDrive items for processing from /Top Level to Share/10-Files/SMPP +Processing 31 OneDrive items to ensure consistent local state +Uploading differences of ~/OneDriveBusiness +Processing root +The directory has not changed +Processing SMPP_Local +The directory has not changed +Processing SMPP-IF-SPEC_v3_3-24858.pdf +The file has not changed +Processing SMPP_v3_4_Issue1_2-24857.pdf +The file has not changed +Processing new_local_file.txt +The file has not changed +Processing root +The directory has not changed +... +The directory has not changed +Processing week02-03-Combinational_Logic-v1.pptx +The file has not changed +Uploading new items of ~/OneDriveBusiness +Applying changes of Path ID: 01WIXGO5V6Y2GOVW7725BZO354PWSELRRZ +Number of items from OneDrive to process: 0 +Attempting to sync OneDrive Business Shared Folders +Syncing this OneDrive Business Shared Folder: Child Shared Folder +OneDrive Business Shared Folder - Shared By: test user +Applying changes of Path ID: 01JRXHEZMREEB3EJVHNVHKNN454Q7DFXPR +Adding OneDrive root details for processing +Adding OneDrive folder details for processing +Adding 4 OneDrive items for processing from OneDrive folder +Adding 2 OneDrive items for processing from /Child Shared Folder/Cisco VDI Whitepaper +Adding 2 OneDrive items for processing from /Child Shared Folder/SMPP_Shared +Processing 11 OneDrive items to ensure consistent local state +Syncing this OneDrive Business Shared Folder: Top Level to Share +OneDrive Business Shared Folder - Shared By: test user (testuser@mynasau3.onmicrosoft.com) +Applying changes of Path ID: 01JRXHEZLRMXHKBYZNOBF3TQOPBXS3VZMA +Adding OneDrive root details for processing +Adding OneDrive folder details for processing +Adding 4 OneDrive items for processing from OneDrive folder +Adding 3 OneDrive items for processing from /Top Level to Share/10-Files +Adding 2 OneDrive items for processing from /Top Level to Share/10-Files/Cisco VDI Whitepaper +Adding 2 OneDrive items for processing from /Top Level to Share/10-Files/Images +Adding 8 OneDrive items for processing from /Top Level to Share/10-Files/Images/JPG +Adding 8 OneDrive items for processing from /Top Level to Share/10-Files/Images/PNG +Adding 2 OneDrive items for processing from /Top Level to Share/10-Files/SMPP +Processing 31 OneDrive items to ensure consistent local state +``` + +**Note:** Whenever you modify the `business_shared_folders` file you must perform a `--resync` of your database to clean up stale entries due to changes in your configuration. + +## Enable / Disable syncing of OneDrive Business Shared Folders +Performing a sync of the configured OneDrive Business Shared Folders can be enabled / disabled via adding the following to your configuration file. + +### Enable syncing of OneDrive Business Shared Folders via config file +```text +sync_business_shared_folders = "true" +``` + +### Disable syncing of OneDrive Business Shared Folders via config file +```text +sync_business_shared_folders = "false" +``` + +## Known Issues +Shared folders, shared with you from people outside of your 'organisation' are unable to be synced. This is due to the Microsoft Graph API not presenting these folders. + +Shared folders that match this scenario, when you view 'Shared' via OneDrive online, will have a 'world' symbol as per below: + +![shared_with_me](./images/shared_with_me.JPG) + +This issue is being tracked by: [#966](https://github.com/abraunegg/onedrive/issues/966) diff --git a/docs/docker.md b/docs/docker.md new file mode 100644 index 00000000..2b59eea0 --- /dev/null +++ b/docs/docker.md @@ -0,0 +1,320 @@ +# Run the OneDrive Client for Linux under Docker +This client can be run as a Docker container, with 3 available container base options for you to choose from: + +| Container Base | Docker Tag | Description | i686 | x86_64 | ARMHF | AARCH64 | +|----------------|-------------|----------------------------------------------------------------|:------:|:------:|:-----:|:-------:| +| Alpine Linux | edge-alpine | Docker container based on Alpine 3.18 using 'master' |❌|✔|❌|✔| +| Alpine Linux | alpine | Docker container based on Alpine 3.18 using latest release |❌|✔|❌|✔| +| Debian | debian | Docker container based on Debian Stable using latest release |✔|✔|✔|✔| +| Debian | edge | Docker container based on Debian Stable using 'master' |✔|✔|✔|✔| +| Debian | edge-debian | Docker container based on Debian Stable using 'master' |✔|✔|✔|✔| +| Debian | latest | Docker container based on Debian Stable using latest release |✔|✔|✔|✔| +| Fedora | edge-fedora | Docker container based on Fedora 38 using 'master' |❌|✔|❌|✔| +| Fedora | fedora | Docker container based on Fedora 38 using latest release |❌|✔|❌|✔| + +These containers offer a simple monitoring-mode service for the OneDrive Client for Linux. + +The instructions below have been validated on: +* Red Hat Enterprise Linux 8.x +* Ubuntu Server 22.04 + +The instructions below will utilise the 'edge' tag, however this can be substituted for any of the other docker tags such as 'latest' from the table above if desired. + +The 'edge' Docker Container will align closer to all documentation and features, where as 'latest' is the release version from a static point in time. The 'latest' tag however may contain bugs and/or issues that will have been fixed, and those fixes are contained in 'edge'. + +Additionally there are specific version release tags for each release. Refer to https://hub.docker.com/r/driveone/onedrive/tags for any other Docker tags you may be interested in. + +## Basic Setup +### 0. Install docker using your distribution platform's instructions +1. Ensure that SELinux has been disabled on your system. A reboot may be required to ensure that this is correctly disabled. +2. Install Docker as per required for your platform. Refer to https://docs.docker.com/engine/install/ for assistance. +3. Obtain your normal, non-root user UID and GID by using the `id` command +4. As your normal, non-root user, ensure that you can run `docker run hello-world` *without* using `sudo` + +Once the above 4 steps are complete and you can successfully run `docker run hello-world` without sudo, only then proceed to 'Pulling and Running the Docker Image' + +## Pulling and Running the Docker Image +### 1. Pull the image +```bash +docker pull driveone/onedrive:edge +``` + +**NOTE:** SELinux context needs to be configured or disabled for Docker to be able to write to OneDrive host directory. + +### 2. Prepare config volume +The Docker container requries 2 Docker volumes: +* Config Volume +* Data Volume + +Create the config volume with the following command: +```bash +docker volume create onedrive_conf +``` + +This will create a docker volume labeled `onedrive_conf`, where all configuration of your onedrive account will be stored. You can add a custom config file and other things later. + +The second docker volume is for your data folder and is created in the next step. This volume needs to be a path to a directory on your local filesystem, and this is where your data will be stored from OneDrive. Keep in mind that: + +* The owner of this specified folder must not be root +* The owner of this specified folder must have permissions for its parent directory + +**NOTE:** Issues occur when this target folder is a mounted folder of an external system (NAS, SMB mount, USB Drive etc) as the 'mount' itself is owned by 'root'. If this is your use case, you *must* ensure your normal user can mount your desired target without having the target mounted by 'root'. If you do not fix this, your Docker container will fail to start with the following error message: +```bash +ROOT level privileges prohibited! +``` + +### 3. First run +The 'onedrive' client within the Docker container needs to be authorized with your Microsoft account. This is achieved by initially running docker in interactive mode. + +Run the docker image with the commands below and make sure to change `ONEDRIVE_DATA_DIR` to the actual onedrive data directory on your filesystem that you wish to use (e.g. `"/home/abraunegg/OneDrive"`). +```bash +export ONEDRIVE_DATA_DIR="${HOME}/OneDrive" +mkdir -p ${ONEDRIVE_DATA_DIR} +docker run -it --name onedrive -v onedrive_conf:/onedrive/conf \ + -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" \ + -e "ONEDRIVE_UID=${ONEDRIVE_UID}" \ + -e "ONEDRIVE_GID=${ONEDRIVE_GID}" \ + driveone/onedrive:edge +``` +**Important:** The 'target' folder of `ONEDRIVE_DATA_DIR` must exist before running the Docker container, otherwise, Docker will create the target folder, and the folder will be given 'root' permissions, which then causes the Docker container to fail upon startup with the following error message: +```bash +ROOT level privileges prohibited! +``` +**NOTE:** It is also highly advisable for you to replace `${ONEDRIVE_UID}` and `${ONEDRIVE_GID}` with your actual UID and GID as specified by your `id` command output to avoid any any potential user or group conflicts. + +**Example:** +```bash +export ONEDRIVE_UID=`id -u` +export ONEDRIVE_GID=`id -g` +export ONEDRIVE_DATA_DIR="${HOME}/OneDrive" +mkdir -p ${ONEDRIVE_DATA_DIR} +docker run -it --name onedrive -v onedrive_conf:/onedrive/conf \ + -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" \ + -e "ONEDRIVE_UID=${ONEDRIVE_UID}" \ + -e "ONEDRIVE_GID=${ONEDRIVE_GID}" \ + driveone/onedrive:edge +``` + +When the Docker container successfully starts: +* You will be asked to open a specific link using your web browser +* Login to your Microsoft Account and give the application the permission +* After giving the permission, you will be redirected to a blank page +* Copy the URI of the blank page into the application prompt to authorise the application + +Once the 'onedrive' application is authorised, the client will automatically start monitoring your `ONEDRIVE_DATA_DIR` for data changes to be uploaded to OneDrive. Files stored on OneDrive will be downloaded to this location. + +If the client is working as expected, you can detach from the container with Ctrl+p, Ctrl+q. + +### 4. Docker Container Status, stop, and restart +Check if the monitor service is running + +```bash +docker ps -f name=onedrive +``` + +Show monitor run logs + +```bash +docker logs onedrive +``` + +Stop running monitor + +```bash +docker stop onedrive +``` + +Resume monitor + +```bash +docker start onedrive +``` + +Remove onedrive Docker container + +```bash +docker rm -f onedrive +``` +## Advanced Setup + +### 5. Docker-compose +Also supports docker-compose schemas > 3. +In the following example it is assumed you have a `ONEDRIVE_DATA_DIR` environment variable and a `onedrive_conf` volume. +However, you can also use bind mounts for the configuration folder, e.g. `export ONEDRIVE_CONF="${HOME}/OneDriveConfig"`. + +``` +version: "3" +services: + onedrive: + image: driveone/onedrive:edge + restart: unless-stopped + environment: + - ONEDRIVE_UID=${PUID} + - ONEDRIVE_GID=${PGID} + volumes: + - onedrive_conf:/onedrive/conf + - ${ONEDRIVE_DATA_DIR}:/onedrive/data +``` + +Note that you still have to perform step 3: First Run. + +### 6. Edit the config +The 'onedrive' client should run in default configuration, however you can change this default configuration by placing a custom config file in the `onedrive_conf` docker volume. First download the default config from [here](https://raw.githubusercontent.com/abraunegg/onedrive/master/config) +Then put it into your onedrive_conf volume path, which can be found with: + +```bash +docker volume inspect onedrive_conf +``` + +Or you can map your own config folder to the config volume. Make sure to copy all files from the docker volume into your mapped folder first. + +The detailed document for the config can be found here: [Configuration](https://github.com/abraunegg/onedrive/blob/master/docs/usage.md#configuration) + +### 7. Sync multiple accounts +There are many ways to do this, the easiest is probably to +1. Create a second docker config volume (replace `Work` with your desired name): `docker volume create onedrive_conf_Work` +2. And start a second docker monitor container (again replace `Work` with your desired name): +``` +export ONEDRIVE_DATA_DIR_WORK="/home/abraunegg/OneDriveWork" +mkdir -p ${ONEDRIVE_DATA_DIR_WORK} +docker run -it --restart unless-stopped --name onedrive_Work -v onedrive_conf_Work:/onedrive/conf -v "${ONEDRIVE_DATA_DIR_WORK}:/onedrive/data" driveone/onedrive:edge +``` + +## Run or update with one script +If you are experienced with docker and onedrive, you can use the following script: + +```bash +# Update ONEDRIVE_DATA_DIR with correct OneDrive directory path +ONEDRIVE_DATA_DIR="${HOME}/OneDrive" +# Create directory if non-existant +mkdir -p ${ONEDRIVE_DATA_DIR} + +firstRun='-d' +docker pull driveone/onedrive:edge +docker inspect onedrive_conf > /dev/null 2>&1 || { docker volume create onedrive_conf; firstRun='-it'; } +docker inspect onedrive > /dev/null 2>&1 && docker rm -f onedrive +docker run $firstRun --restart unless-stopped --name onedrive -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" driveone/onedrive:edge +``` + +## Environment Variables +| Variable | Purpose | Sample Value | +| ---------------- | --------------------------------------------------- |:--------------------------------------------------------------------------------------------------------------------------------:| +| ONEDRIVE_UID | UserID (UID) to run as | 1000 | +| ONEDRIVE_GID | GroupID (GID) to run as | 1000 | +| ONEDRIVE_VERBOSE | Controls "--verbose" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_DEBUG | Controls "--verbose --verbose" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_DEBUG_HTTPS | Controls "--debug-https" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_RESYNC | Controls "--resync" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_DOWNLOADONLY | Controls "--download-only" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_UPLOADONLY | Controls "--upload-only" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_NOREMOTEDELETE | Controls "--no-remote-delete" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_LOGOUT | Controls "--logout" switch. Default is 0 | 1 | +| ONEDRIVE_REAUTH | Controls "--reauth" switch. Default is 0 | 1 | +| ONEDRIVE_AUTHFILES | Controls "--auth-files" option. Default is "" | "authUrl:responseUrl" | +| ONEDRIVE_AUTHRESPONSE | Controls "--auth-response" option. Default is "" | See [here](https://github.com/abraunegg/onedrive/blob/master/docs/usage.md#authorize-the-application-with-your-onedrive-account) | +| ONEDRIVE_DISPLAY_CONFIG | Controls "--display-running-config" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_SINGLE_DIRECTORY | Controls "--single-directory" option. Default = "" | "mydir" | + +### Usage Examples +**Verbose Output:** +```bash +docker container run -e ONEDRIVE_VERBOSE=1 -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" driveone/onedrive:edge +``` +**Debug Output:** +```bash +docker container run -e ONEDRIVE_DEBUG=1 -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" driveone/onedrive:edge +``` +**Perform a --resync:** +```bash +docker container run -e ONEDRIVE_RESYNC=1 -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" driveone/onedrive:edge +``` +**Perform a --resync and --verbose:** +```bash +docker container run -e ONEDRIVE_RESYNC=1 -e ONEDRIVE_VERBOSE=1 -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" driveone/onedrive:edge +``` +**Perform a --logout and re-authenticate:** +```bash +docker container run -it -e ONEDRIVE_LOGOUT=1 -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" driveone/onedrive:edge +``` + +## Build instructions +### Build Environment Requirements +* Build environment must have at least 1GB of memory & 2GB swap space + +There are 2 ways to validate this requirement: +* Modify the file `/etc/dphys-swapfile` and edit the `CONF_SWAPSIZE`, for example: `CONF_SWAPSIZE=2048`. A reboot is required to make this change effective. +* Dynamically allocate a swapfile for building: +```bash +cd /var +sudo fallocate -l 1.5G swapfile +sudo chmod 600 swapfile +sudo mkswap swapfile +sudo swapon swapfile +# make swap permanent +sudo nano /etc/fstab +# add "/swapfile swap swap defaults 0 0" at the end of file +# check it has been assigned +swapon -s +free -h +``` + +### Building a custom Docker image +You can also build your own image instead of pulling the one from [hub.docker.com](https://hub.docker.com/r/driveone/onedrive): +```bash +git clone https://github.com/abraunegg/onedrive +cd onedrive +docker build . -t local-onedrive -f contrib/docker/Dockerfile +``` + +There are alternate, smaller images available by building +Dockerfile-debian or Dockerfile-alpine. These [multi-stage builder pattern](https://docs.docker.com/develop/develop-images/multistage-build/) +Dockerfiles require Docker version at least 17.05. + +#### How to build and run a custom Docker image based on Debian +``` bash +docker build . -t local-ondrive-debian -f contrib/docker/Dockerfile-debian +docker container run -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" local-ondrive-debian:latest +``` + +#### How to build and run a custom Docker image based on Alpine Linux +``` bash +docker build . -t local-ondrive-alpine -f contrib/docker/Dockerfile-alpine +docker container run -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" local-ondrive-alpine:latest +``` + +#### How to build and run a custom Docker image for ARMHF (Raspberry Pi) +Compatible with: +* Raspberry Pi +* Raspberry Pi 2 +* Raspberry Pi Zero +* Raspberry Pi 3 +* Raspberry Pi 4 +``` bash +docker build . -t local-onedrive-armhf -f contrib/docker/Dockerfile-debian +docker container run -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" local-onedrive-armhf:latest +``` + +#### How to build and run a custom Docker image for AARCH64 Platforms +``` bash +docker build . -t local-onedrive-aarch64 -f contrib/docker/Dockerfile-debian +docker container run -v onedrive_conf:/onedrive/conf -v "${ONEDRIVE_DATA_DIR}:/onedrive/data" local-onedrive-aarch64:latest +``` + +#### How to support double-byte languages +In some geographic regions, you may need to change and/or update the locale specification of the Docker container to better support the local language used for your local filesystem. To do this, follow the example below: +``` +FROM driveone/onedrive + +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update +RUN apt-get install -y locales + +RUN echo "ja_JP.UTF-8 UTF-8" > /etc/locale.gen && \ + locale-gen ja_JP.UTF-8 && \ + dpkg-reconfigure locales && \ + /usr/sbin/update-locale LANG=ja_JP.UTF-8 + +ENV LC_ALL ja_JP.UTF-8 +``` +The above example changes the Docker container to support Japanese. To support your local language, change `ja_JP.UTF-8` to the required entry. diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 00000000..3f00ae21 --- /dev/null +++ b/docs/install.md @@ -0,0 +1,277 @@ +# Installing or Upgrading using Distribution Packages or Building the OneDrive Client for Linux from source + +## Installing or Upgrading using Distribution Packages +This project has been packaged for the following Linux distributions as per below. The current client release is: [![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) + +Only the current release version or greater is supported. Earlier versions are not supported and should not be installed or used. + +#### Important Note: +Distribution packages may be of an older release when compared to the latest release that is [available](https://github.com/abraunegg/onedrive/releases). If any package version indicator below is 'red' for your distribution, it is recommended that you build from source. Do not install the software from the available distribution package. If a package is out of date, please contact the package maintainer for resolution. + +| Distribution | Package Name & Package Link |   PKG_Version   |  i686  | x86_64 | ARMHF | AARCH64 | Extra Details | +|---------------------------------|------------------------------------------------------------------------------|:---------------:|:----:|:------:|:-----:|:-------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Alpine Linux | [onedrive](https://pkgs.alpinelinux.org/packages?name=onedrive&branch=edge) |Alpine Linux Edge package|❌|✔|❌|✔ | | +| Arch Linux

Manjaro Linux | [onedrive-abraunegg](https://aur.archlinux.org/packages/onedrive-abraunegg/) |AUR package|✔|✔|✔|✔ | Install via: `pamac build onedrive-abraunegg` from the Arch Linux User Repository (AUR)

**Note:** If asked regarding a provider for 'd-runtime' and 'd-compiler', select 'liblphobos' and 'ldc'

**Note:** System must have at least 1GB of memory & 1GB swap space +| Debian 11 | [onedrive](https://packages.debian.org/bullseye/source/onedrive) |Debian 11 package|✔|✔|✔|✔| **Note:** Do not install from Debian Package Repositories

It is recommended that for Debian 11 that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) | +| Debian 12 | [onedrive](https://packages.debian.org/bookworm/source/onedrive) |Debian 12 package|✔|✔|✔|✔| **Note:** Do not install from Debian Package Repositories

It is recommended that for Debian 12 that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) | +| Fedora | [onedrive](https://koji.fedoraproject.org/koji/packageinfo?packageID=26044) |Fedora Rawhide package|✔|✔|✔|✔| | +| Gentoo | [onedrive](https://gpo.zugaina.org/net-misc/onedrive) | No API Available |✔|✔|❌|❌| | +| Homebrew | [onedrive](https://formulae.brew.sh/formula/onedrive) | Homebrew package |❌|✔|❌|❌| | +| Linux Mint 20.x | [onedrive](https://community.linuxmint.com/software/view/onedrive) |Ubuntu 20.04 package |❌|✔|✔|✔| **Note:** Do not install from Linux Mint Repositories

It is recommended that for Linux Mint that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) | +| Linux Mint 21.x | [onedrive](https://community.linuxmint.com/software/view/onedrive) |Ubuntu 22.04 package |❌|✔|✔|✔| **Note:** Do not install from Linux Mint Repositories

It is recommended that for Linux Mint that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) | +| NixOS | [onedrive](https://search.nixos.org/packages?channel=20.09&from=0&size=50&sort=relevance&query=onedrive)|nixpkgs unstable package|❌|✔|❌|❌| Use package `onedrive` either by adding it to `configuration.nix` or by using the command `nix-env -iA .onedrive`. This does not install a service. To install a service, use unstable channel (will stabilize in 20.09) and add `services.onedrive.enable=true` in `configuration.nix`. You can also add a custom package using the `services.onedrive.package` option (recommended since package lags upstream). Enabling the service installs a default package too (based on the channel). You can also add multiple onedrive accounts trivially, see [documentation](https://github.com/NixOS/nixpkgs/pull/77734#issuecomment-575874225). | +| OpenSuSE | [onedrive](https://software.opensuse.org/package/onedrive) |openSUSE Tumbleweed package|✔|✔|❌|❌| | +| OpenSuSE Build Service | [onedrive](https://build.opensuse.org/package/show/home:npreining:debian-ubuntu-onedrive/onedrive) | No API Available |✔|✔|✔|✔| Package Build Service for Debian and Ubuntu | +| Raspbian | [onedrive](https://archive.raspbian.org/raspbian/pool/main/o/onedrive/) |Raspbian Stable package |❌|❌|✔|✔| **Note:** Do not install from Raspbian Package Repositories

It is recommended that for Raspbian that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) | +| Slackware | [onedrive](https://slackbuilds.org/result/?search=onedrive&sv=) |SlackBuilds package|✔|✔|❌|❌| | +| Solus | [onedrive](https://dev.getsol.us/search/query/FB7PIf1jG9Z9/#R) |Solus package|✔|✔|❌|❌| | +| Ubuntu 20.04 | [onedrive](https://packages.ubuntu.com/focal/onedrive) |Ubuntu 20.04 package |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe

It is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) | +| Ubuntu 22.04 | [onedrive](https://packages.ubuntu.com/jammy/onedrive) |Ubuntu 22.04 package |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe

It is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) | +| Ubuntu 23.04 | [onedrive](https://packages.ubuntu.com/lunar/onedrive) |Ubuntu 23.04 package |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe

It is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) | +| Void Linux | [onedrive](https://voidlinux.org/packages/?arch=x86_64&q=onedrive) |Void Linux x86_64 package|✔|✔|❌|❌| | + +#### Important information for all Ubuntu and Ubuntu based distribution users: +This information is specifically for the following platforms and distributions: +* Ubuntu +* Lubuntu +* Linux Mint +* POP OS +* Peppermint OS + +Whilst there are [onedrive](https://packages.ubuntu.com/search?keywords=onedrive&searchon=names&suite=all§ion=all) Universe packages available for Ubuntu, do not install 'onedrive' from these Universe packages. The default Universe packages are out-of-date and are not supported and should not be used. If you wish to use a package, it is highly recommended that you utilise the [OpenSuSE Build Service](ubuntu-package-install.md) to install packages for these platforms. If the OpenSuSE Build Service does not cater for your version, your only option is to build from source. + +If you wish to change this situation so that you can just use the Universe packages via 'apt install onedrive', consider becoming the Ubuntu package maintainer and contribute back to your community. + +## Building from Source - High Level Requirements +* Build environment must have at least 1GB of memory & 1GB swap space +* Install the required distribution package dependencies +* [libcurl](http://curl.haxx.se/libcurl/) +* [SQLite 3](https://www.sqlite.org/) >= 3.7.15 +* [Digital Mars D Compiler (DMD)](http://dlang.org/download.html) or [LDC – the LLVM-based D Compiler](https://github.com/ldc-developers/ldc) + +**Note:** DMD version >= 2.088.0 or LDC version >= 1.18.0 is required to compile this application + +### Example for installing DMD Compiler +```text +curl -fsS https://dlang.org/install.sh | bash -s dmd +``` + +### Example for installing LDC Compiler +```text +curl -fsS https://dlang.org/install.sh | bash -s ldc +``` + +## Distribution Package Dependencies +### Dependencies: Ubuntu 16.x +Ubuntu Linux 16.x LTS reached the end of its five-year LTS window on April 30th 2021 and is no longer supported. + +### Dependencies: Ubuntu 18.x / Lubuntu 18.x +Ubuntu Linux 18.x LTS reached the end of its five-year LTS window on May 31th 2023 and is no longer supported. + +### Dependencies: Debian 9 +Debian 9 reached the end of its five-year support window on June 30th 2022 and is no longer supported. + +### Dependencies: Ubuntu 20.x -> Ubuntu 23.x / Debian 10 -> Debian 12 - x86_64 +These dependencies are also applicable for all Ubuntu based distributions such as: +* Lubuntu +* Linux Mint +* POP OS +* Peppermint OS +```text +sudo apt install build-essential +sudo apt install libcurl4-openssl-dev libsqlite3-dev pkg-config git curl +curl -fsS https://dlang.org/install.sh | bash -s dmd +``` +For notifications the following is also necessary: +```text +sudo apt install libnotify-dev +``` + +### Dependencies: CentOS 6.x / RHEL 6.x +CentOS 6.x and RHEL 6.x reached End of Life status on November 30th 2020 and is no longer supported. + +### Dependencies: Fedora < Version 18 / CentOS 7.x / RHEL 7.x +```text +sudo yum groupinstall 'Development Tools' +sudo yum install libcurl-devel sqlite-devel +curl -fsS https://dlang.org/install.sh | bash -s dmd-2.099.0 +``` +For notifications the following is also necessary: +```text +sudo yum install libnotify-devel +``` + +### Dependencies: Fedora > Version 18 / CentOS 8.x / RHEL 8.x / RHEL 9.x +```text +sudo dnf groupinstall 'Development Tools' +sudo dnf install libcurl-devel sqlite-devel +curl -fsS https://dlang.org/install.sh | bash -s dmd +``` +For notifications the following is also necessary: +```text +sudo dnf install libnotify-devel +``` + +### Dependencies: Arch Linux & Manjaro Linux +```text +sudo pacman -S make pkg-config curl sqlite ldc +``` +For notifications the following is also necessary: +```text +sudo pacman -S libnotify +``` + +### Dependencies: Raspbian (ARMHF) and Ubuntu 22.x / Debian 11 / Debian 12 / Raspbian (ARM64) +**Note:** The minimum LDC compiler version required to compile this application is now 1.18.0, which is not available for Debian Buster or distributions based on Debian Buster. You are advised to first upgrade your platform distribution to one that is based on Debian Bullseye (Debian 11) or later. + +These instructions were validated using: +* `Linux raspberrypi 5.10.92-v8+ #1514 SMP PREEMPT Mon Jan 17 17:39:38 GMT 2022 aarch64` (2022-01-28-raspios-bullseye-armhf-lite) using Raspberry Pi 3B (revision 1.2) +* `Linux raspberrypi 5.10.92-v8+ #1514 SMP PREEMPT Mon Jan 17 17:39:38 GMT 2022 aarch64` (2022-01-28-raspios-bullseye-arm64-lite) using Raspberry Pi 3B (revision 1.2) +* `Linux ubuntu 5.15.0-1005-raspi #5-Ubuntu SMP PREEMPT Mon Apr 4 12:21:48 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux` (ubuntu-22.04-preinstalled-server-arm64+raspi) using Raspberry Pi 3B (revision 1.2) + +**Note:** Build environment must have at least 1GB of memory & 1GB swap space. Check with `swapon`. + +```text +sudo apt install build-essential +sudo apt install libcurl4-openssl-dev libsqlite3-dev pkg-config git curl ldc +``` +For notifications the following is also necessary: +```text +sudo apt install libnotify-dev +``` + +### Dependencies: Gentoo +```text +sudo emerge app-portage/layman +sudo layman -a dlang +``` +Add ebuild from contrib/gentoo to a local overlay to use. + +For notifications the following is also necessary: +```text +sudo emerge x11-libs/libnotify +``` + +### Dependencies: OpenSuSE Leap 15.0 +```text +sudo zypper addrepo https://download.opensuse.org/repositories/devel:languages:D/openSUSE_Leap_15.0/devel:languages:D.repo +sudo zypper refresh +sudo zypper install gcc git libcurl-devel sqlite3-devel dmd phobos-devel phobos-devel-static +``` +For notifications the following is also necessary: +```text +sudo zypper install libnotify-devel +``` + +### Dependencies: OpenSuSE Leap 15.1 +```text +sudo zypper addrepo https://download.opensuse.org/repositories/devel:languages:D/openSUSE_Leap_15.1/devel:languages:D.repo +sudo zypper refresh +sudo zypper install gcc git libcurl-devel sqlite3-devel dmd phobos-devel phobos-devel-static +``` +For notifications the following is also necessary: +```text +sudo zypper install libnotify-devel +``` + +### Dependencies: OpenSuSE Leap 15.2 +```text +sudo zypper refresh +sudo zypper install gcc git libcurl-devel sqlite3-devel dmd phobos-devel phobos-devel-static +``` +For notifications the following is also necessary: +```text +sudo zypper install libnotify-devel +``` + +## Compilation & Installation +### High Level Steps +1. Install the platform dependencies for your Linux OS +2. Activate your DMD or LDC compiler +3. Clone the GitHub repository, run configure and make, then install +4. Deactivate your DMD or LDC compiler + +### Building using DMD Reference Compiler +Before cloning and compiling, if you have installed DMD via curl for your OS, you will need to activate DMD as per example below: +```text +Run `source ~/dlang/dmd-2.088.0/activate` in your shell to use dmd-2.088.0. +This will setup PATH, LIBRARY_PATH, LD_LIBRARY_PATH, DMD, DC, and PS1. +Run `deactivate` later on to restore your environment. +``` +Without performing this step, the compilation process will fail. + +**Note:** Depending on your DMD version, substitute `2.088.0` above with your DMD version that is installed. + +```text +git clone https://github.com/abraunegg/onedrive.git +cd onedrive +./configure +make clean; make; +sudo make install +``` + +### Build options +Notifications can be enabled using the `configure` switch `--enable-notifications`. + +Systemd service files are installed in the appropriate directories on the system, +as provided by `pkg-config systemd` settings. If the need for overriding the +deduced path are necessary, the two options `--with-systemdsystemunitdir` (for +the Systemd system unit location), and `--with-systemduserunitdir` (for the +Systemd user unit location) can be specified. Passing in `no` to one of these +options disabled service file installation. + +By passing `--enable-debug` to the `configure` call, `onedrive` gets built with additional debug +information, useful (for example) to get `perf`-issued figures. + +By passing `--enable-completions` to the `configure` call, shell completion functions are +installed for `bash`, `zsh` and `fish`. The installation directories are determined +as far as possible automatically, but can be overridden by passing +`--with-bash-completion-dir=`, `--with-zsh-completion-dir=`, and +`--with-fish-completion-dir=` to `configure`. + +### Building using a different compiler (for example [LDC](https://wiki.dlang.org/LDC)) +#### ARMHF Architecture (Raspbian) and ARM64 Architecture (Ubuntu 22.x / Debian 11 / Raspbian) +**Note:** The minimum LDC compiler version required to compile this application is now 1.18.0, which is not available for Debian Buster or distributions based on Debian Buster. You are advised to first upgrade your platform distribution to one that is based on Debian Bullseye (Debian 11) or later. + +**Note:** Build environment must have at least 1GB of memory & 1GB swap space. Check with `swapon`. +```text +git clone https://github.com/abraunegg/onedrive.git +cd onedrive +./configure DC=/usr/bin/ldmd2 +make clean; make +sudo make install +``` + +## Upgrading the client +If you have installed the client from a distribution package, the client will be updated when the distribution package is updated by the package maintainer and will be updated to the new application version when you perform your package update. + +If you have built the client from source, to upgrade your client, it is recommended that you first uninstall your existing 'onedrive' binary (see below), then re-install the client by re-cloning, re-compiling and re-installing the client again to install the new version. + +**Note:** Following the uninstall process will remove all client components including *all* systemd files, including any custom files created for specific access such as SharePoint Libraries. + +You can optionally choose to not perform this uninstallation step, and simply re-install the client by re-cloning, re-compiling and re-installing the client again - however the risk here is that you end up with two onedrive client binaries on your system, and depending on your system search path preferences, this will determine which binary is used. + +**Important:** Before performing any upgrade, it is highly recommended for you to stop any running systemd service if applicable to ensure that these services are restarted using the updated client version. + +Post re-install, to confirm that you have the new version of the client installed, use `onedrive --version` to determine the client version that is now installed. + +## Uninstalling the client +### Uninstalling the client if installed from distribution package +Follow your distribution documentation to uninstall the package that you installed + +### Uninstalling the client if installed and built from source +From within your GitHub repository clone, perform the following to remove the 'onedrive' binary: +```text +sudo make uninstall +``` + +If you are not upgrading your client, to remove your application state and configuration, perform the following additional step: +``` +rm -rf ~/.config/onedrive +``` +**Note:** If you are using the `--confdir option`, substitute `~/.config/onedrive` for the correct directory storing your client configuration. + +If you want to just delete the application key, but keep the items database: +```text +rm -f ~/.config/onedrive/refresh_token +``` diff --git a/docs/known-issues.md b/docs/known-issues.md new file mode 100644 index 00000000..6d970ff9 --- /dev/null +++ b/docs/known-issues.md @@ -0,0 +1,54 @@ +# Known Issues +The below are known issues with this client: + +## Moving files into different folders should not cause data to delete and be re-uploaded +**Issue Tracker:** [#876](https://github.com/abraunegg/onedrive/issues/876) + +**Description:** + +When running the client in standalone mode (`--synchronize`) moving folders that are successfully synced around between subsequent standalone syncs causes a deletion & re-upload of data to occur. + +**Explanation:** + +Technically, the client is 'working' correctly, as, when moving files, you are 'deleting' them from the current location, but copying them to the 'new location'. As the client is running in standalone sync mode, there is no way to track what OS operations have been done when the client is not running - thus, this is why the 'delete and upload' is occurring. + +**Workaround:** + +If the tracking of moving data to new local directories is requried, it is better to run the client in service mode (`--monitor`) rather than in standalone mode, as the 'move' of files can then be handled at the point when it occurs, so that the data is moved to the new location on OneDrive without the need to be deleted and re-uploaded. + +## Application 'stops' running without any visible reason +**Issue Tracker:** [#494](https://github.com/abraunegg/onedrive/issues/494), [#753](https://github.com/abraunegg/onedrive/issues/753), [#792](https://github.com/abraunegg/onedrive/issues/792), [#884](https://github.com/abraunegg/onedrive/issues/884), [#1162](https://github.com/abraunegg/onedrive/issues/1162), [#1408](https://github.com/abraunegg/onedrive/issues/1408), [#1520](https://github.com/abraunegg/onedrive/issues/1520), [#1526](https://github.com/abraunegg/onedrive/issues/1526) + +**Description:** + +When running the client and performing an upload or download operation, the application just stops working without any reason or explanation. If `echo $?` is used after the application has exited without visible reason, an error level of 141 may be provided. + +Additionally, this issue has mainly been seen when the client is operating against Microsoft's Europe Data Centre's. + +**Explanation:** + +The client is heavily dependant on Curl and OpenSSL to perform the activities with the Microsoft OneDrive service. Generally, when this issue occurs, the following is found in the HTTPS Debug Log: +``` +OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 104 +``` +The only way to determine that this is the cause of the application ceasing to work is to generate a HTTPS debug log using the following additional flags: +``` +--verbose --verbose --debug-https +``` + +This is indicative of the following: +* Some sort of flaky Internet connection somewhere between you and the OneDrive service +* Some sort of 'broken' HTTPS transparent inspection service inspecting your traffic somewhere between you and the OneDrive service + +**How to resolve:** + +The best avenue of action here are: +* Ensure your OS is as up-to-date as possible +* Get support from your OS vendor +* Speak to your ISP or Help Desk for assistance +* Open a ticket with OpenSSL and/or Curl teams to better handle this sort of connection failure +* Generate a HTTPS Debug Log for this application and open a new support request with Microsoft and provide the debug log file for their analysis. + +If you wish to diagnose this issue further, refer to the following: + +https://maulwuff.de/research/ssl-debugging.html diff --git a/docs/national-cloud-deployments.md b/docs/national-cloud-deployments.md new file mode 100644 index 00000000..6b348388 --- /dev/null +++ b/docs/national-cloud-deployments.md @@ -0,0 +1,145 @@ +# How to configure access to specific Microsoft Azure deployments +## Application Version +Before reading this document, please ensure you are running application version [![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) or greater. Use `onedrive --version` to determine what application version you are using and upgrade your client if required. + +## Process Overview +In some cases it is a requirement to utilise specific Microsoft Azure cloud deployments to conform with data and security reuqirements that requires data to reside within the geographic borders of that country. +Current national clouds that are supported are: +* Microsoft Cloud for US Government +* Microsoft Cloud Germany +* Azure and Office365 operated by 21Vianet in China + +In order to successfully use these specific Microsoft Azure deployments, the following steps are required: +1. Register an application with the Microsoft identity platform using the Azure portal +2. Configure the new application with the appropriate authentication scopes +3. Validate that the authentication / redirect URI is correct for your application registration +4. Configure the onedrive client to use the new application id as provided during application registration +5. Configure the onedrive client to use the right Microsoft Azure deployment region that your application was registered with +6. Authenticate the client + +## Step 1: Register a new application with Microsoft Azure +1. Log into your applicable Microsoft Azure Portal with your applicable Office365 identity: + +| National Cloud Environment | Microsoft Azure Portal | +|---|---| +| Microsoft Cloud for US Government | https://portal.azure.com/ | +| Microsoft Cloud Germany | https://portal.azure.com/ | +| Azure and Office365 operated by 21Vianet | https://portal.azure.cn/ | + +2. Select 'Azure Active Directory' as the service you wish to configure +3. Under 'Manage', select 'App registrations' to register a new application +4. Click 'New registration' +5. Type in the appropriate details required as per below: + +![application_registration](./images/application_registration.jpg) + +6. To save the application registration, click 'Register' and something similar to the following will be displayed: + +![application_registration_done](./images/application_registration_done.jpg) + +**Note:** The Application (client) ID UUID as displayed after client registration, is what is required as the 'application_id' for Step 4 below. + +## Step 2: Configure application authentication scopes +Configure the API permissions as per the following: + +| API / Permissions name | Type | Description | Admin consent required | +|---|---|---|---| +| Files.ReadWrite | Delegated | Have full access to user files | No | +| Files.ReadWrite.All | Delegated | Have full access to all files user can access | No | +| Sites.ReadWrite.All | Delegated | Have full access to all items in all site collections | No | +| offline_access | Delegated | Maintain access to data you have given it access to | No | + +![authentication_scopes](./images/authentication_scopes.jpg) + +## Step 3: Validate that the authentication / redirect URI is correct +Add the appropriate redirect URI for your Azure deployment: + +![authentication_response_uri](./images/authentication_response_uri.jpg) + +A valid entry for the response URI should be one of: +* https://login.microsoftonline.us/common/oauth2/nativeclient (Microsoft Cloud for US Government) +* https://login.microsoftonline.de/common/oauth2/nativeclient (Microsoft Cloud Germany) +* https://login.chinacloudapi.cn/common/oauth2/nativeclient (Azure and Office365 operated by 21Vianet in China) + +For a single-tenant application, it may be necessary to use your specific tenant id instead of "common": +* https://login.microsoftonline.us/example.onmicrosoft.us/oauth2/nativeclient (Microsoft Cloud for US Government) +* https://login.microsoftonline.de/example.onmicrosoft.de/oauth2/nativeclient (Microsoft Cloud Germany) +* https://login.chinacloudapi.cn/example.onmicrosoft.cn/oauth2/nativeclient (Azure and Office365 operated by 21Vianet in China) + +## Step 4: Configure the onedrive client to use new application registration +Update to your 'onedrive' configuration file (`~/.config/onedrive/config`) the following: +```text +application_id = "insert valid entry here" +``` + +This will reconfigure the client to use the new application registration you have created. + +**Example:** +```text +application_id = "22c49a0d-d21c-4792-aed1-8f163c982546" +``` + +## Step 5: Configure the onedrive client to use the specific Microsoft Azure deployment +Update to your 'onedrive' configuration file (`~/.config/onedrive/config`) the following: +```text +azure_ad_endpoint = "insert valid entry here" +``` + +Valid entries are: +* USL4 (Microsoft Cloud for US Government) +* USL5 (Microsoft Cloud for US Government - DOD) +* DE (Microsoft Cloud Germany) +* CN (Azure and Office365 operated by 21Vianet in China) + +This will configure your client to use the correct Azure AD and Graph endpoints as per [https://docs.microsoft.com/en-us/graph/deployments](https://docs.microsoft.com/en-us/graph/deployments) + +**Example:** +```text +azure_ad_endpoint = "USL4" +``` + +If the Microsoft Azure deployment does not support multi-tenant applications, update to your 'onedrive' configuration file (`~/.config/onedrive/config`) the following: +```text +azure_tenant_id = "insert valid entry here" +``` + +This will configure your client to use the specified tenant id in its Azure AD and Graph endpoint URIs, instead of "common". +The tenant id may be the GUID Directory ID (formatted "00000000-0000-0000-0000-000000000000"), or the fully qualified tenant name (e.g. "example.onmicrosoft.us"). +The GUID Directory ID may be located in the Azure administation page as per [https://docs.microsoft.com/en-us/onedrive/find-your-office-365-tenant-id](https://docs.microsoft.com/en-us/onedrive/find-your-office-365-tenant-id). Note that you may need to go to your national-deployment-specific administration page, rather than following the links within that document. +The tenant name may be obtained by following the PowerShell instructions on [https://docs.microsoft.com/en-us/onedrive/find-your-office-365-tenant-id](https://docs.microsoft.com/en-us/onedrive/find-your-office-365-tenant-id); it is shown as the "TenantDomain" upon completion of the "Connect-AzureAD" command. + +**Example:** +```text +azure_tenant_id = "example.onmicrosoft.us" +# or +azure_tenant_id = "0c4be462-a1ab-499b-99e0-da08ce52a2cc" +``` + +## Step 6: Authenticate the client +Run the application without any additional command switches. + +You will be asked to open a specific URL by using your web browser where you will have to login into your Microsoft Account and give the application the permission to access your files. After giving permission to the application, you will be redirected to a blank page. Copy the URI of the blank page into the application. +```text +[user@hostname ~]$ onedrive + +Authorize this app visiting: + +https://..... + +Enter the response uri: + +``` + +**Example:** +``` +[user@hostname ~]$ onedrive +Authorize this app visiting: + +https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=22c49a0d-d21c-4792-aed1-8f163c982546&scope=Files.ReadWrite%20Files.ReadWrite.all%20Sites.ReadWrite.All%20offline_access&response_type=code&redirect_uri=https://login.microsoftonline.com/common/oauth2/nativeclient + +Enter the response uri: https://login.microsoftonline.com/common/oauth2/nativeclient?code= + +Application has been successfully authorised, however no additional command switches were provided. + +Please use --help for further assistance in regards to running this application. +``` diff --git a/docs/podman.md b/docs/podman.md new file mode 100644 index 00000000..d0689314 --- /dev/null +++ b/docs/podman.md @@ -0,0 +1,289 @@ +# Run the OneDrive Client for Linux under Podman +This client can be run as a Podman container, with 3 available container base options for you to choose from: + +| Container Base | Docker Tag | Description | i686 | x86_64 | ARMHF | AARCH64 | +|----------------|-------------|----------------------------------------------------------------|:------:|:------:|:-----:|:-------:| +| Alpine Linux | edge-alpine | Podman container based on Alpine 3.18 using 'master' |❌|✔|❌|✔| +| Alpine Linux | alpine | Podman container based on Alpine 3.18 using latest release |❌|✔|❌|✔| +| Debian | debian | Podman container based on Debian Stable using latest release |✔|✔|✔|✔| +| Debian | edge | Podman container based on Debian Stable using 'master' |✔|✔|✔|✔| +| Debian | edge-debian | Podman container based on Debian Stable using 'master' |✔|✔|✔|✔| +| Debian | latest | Podman container based on Debian Stable using latest release |✔|✔|✔|✔| +| Fedora | edge-fedora | Podman container based on Fedora 38 using 'master' |❌|✔|❌|✔| +| Fedora | fedora | Podman container based on Fedora 38 using latest release |❌|✔|❌|✔| + +These containers offer a simple monitoring-mode service for the OneDrive Client for Linux. + +The instructions below have been validated on: +* Fedora 35 + +The instructions below will utilise the 'latest' tag, however this can be substituted for any of the other docker tags from the table above if desired. + +Additionally there are specific version release tags for each release. Refer to https://hub.docker.com/r/driveone/onedrive/tags for any other Docker tags you may be interested in. + +**Note:** The below instructions for podman have only been tested as the root user while running the containers themselves as non-root users. + +## Basic Setup +### 0. Install podman using your distribution platform's instructions if not already installed +1. Ensure that SELinux has been disabled on your system. A reboot may be required to ensure that this is correctly disabled. +2. Install Podman as per requried for your platform +3. Obtain your normal, non-root user UID and GID by using the `id` command or select another non-root id to run the container as + +**NOTE:** SELinux context needs to be configured or disabled for Podman to be able to write to OneDrive host directory. + +### 1.1 Prepare data volume +The container requries 2 Podman volumes: +* Config Volume +* Data Volume + +The first volume is for your data folder and is created in the next step. This volume needs to be a path to a directory on your local filesystem, and this is where your data will be stored from OneDrive. Keep in mind that: + +* The owner of this specified folder must not be root +* Podman will attempt to change the permissions of the volume to the user the container is configured to run as + +**NOTE:** Issues occur when this target folder is a mounted folder of an external system (NAS, SMB mount, USB Drive etc) as the 'mount' itself is owed by 'root'. If this is your use case, you *must* ensure your normal user can mount your desired target without having the target mounted by 'root'. If you do not fix this, your Podman container will fail to start with the following error message: +```bash +ROOT level privileges prohibited! +``` + +### 1.2 Prepare config volume +Although not required, you can prepare the config volume before starting the container. Otherwise it will be created automatically during initial startup of the container. + +Create the config volume with the following command: +```bash +podman volume create onedrive_conf +``` + +This will create a podman volume labeled `onedrive_conf`, where all configuration of your onedrive account will be stored. You can add a custom config file and other things later. + +### 2. First run +The 'onedrive' client within the container needs to be authorized with your Microsoft account. This is achieved by initially running podman in interactive mode. + +Run the podman image with the commands below and make sure to change `ONEDRIVE_DATA_DIR` to the actual onedrive data directory on your filesystem that you wish to use (e.g. `"/home/abraunegg/OneDrive"`). + +It is a requirement that the container be run using a non-root uid and gid, you must insert a non-root UID and GID (e.g.` export ONEDRIVE_UID=1000` and export `ONEDRIVE_GID=1000`). + +```bash +export ONEDRIVE_DATA_DIR="${HOME}/OneDrive" +export ONEDRIVE_UID=1000 +export ONEDRIVE_GID=1000 +mkdir -p ${ONEDRIVE_DATA_DIR} +podman run -it --name onedrive --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" \ + -v onedrive_conf:/onedrive/conf:U,Z \ + -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" \ + driveone/onedrive:latest +``` +**Important:** The 'target' folder of `ONEDRIVE_DATA_DIR` must exist before running the podman container + +**If you plan to use podmans built in auto-updating of container images described in step 5, you must pass an additional argument to set a label during the first run.** + +**Important:** In some scenarios, 'podman' sets the configuration and data directories to a different UID & GID as specified. To resolve this situation, you must run 'podman' with the `--userns=keep-id` flag to ensure 'podman' uses the UID and GID as specified. + +The run command would look instead look like as follows: +``` +podman run -it --name onedrive --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" \ + -v onedrive_conf:/onedrive/conf:U,Z \ + -v "onedrive-test-data:/onedrive/data:U,Z" \ + -e PODMAN=1 \ + --label "io.containers.autoupdate=image" + driveone/onedrive:latest +``` + +When the Podman container successfully starts: +* You will be asked to open a specific link using your web browser +* Login to your Microsoft Account and give the application the permission +* After giving the permission, you will be redirected to a blank page +* Copy the URI of the blank page into the application prompt to authorise the application + +Once the 'onedrive' application is authorised, the client will automatically start monitoring your `ONEDRIVE_DATA_DIR` for data changes to be uploaded to OneDrive. Files stored on OneDrive will be downloaded to this location. + +If the client is working as expected, you can detach from the container with Ctrl+p, Ctrl+q. + +### 4. Podman Container Status, stop, and restart +Check if the monitor service is running + +```bash +podman ps -f name=onedrive +``` + +Show monitor run logs + +```bash +podman logs onedrive +``` + +Stop running monitor + +```bash +podman stop onedrive +``` + +Resume monitor + +```bash +podman start onedrive +``` + +Remove onedrive container + +```bash +podman rm -f onedrive +``` +## Advanced Setup + +### 5. Systemd Service & Auto Updating + +Podman supports running containers as a systemd service and also auto updating of the container images. Using the existing running container you can generate a systemd unit file to be installed by the **root** user. To have your container image auto-update with podman, it must first be created with the label `"io.containers.autoupdate=image"` mentioned in step 2. + +``` +cd /tmp +podman generate systemd --new --restart-policy on-failure --name -f onedrive +/tmp/container-onedrive.service + +# copy the generated systemd unit file to the systemd path and reload the daemon + +cp -Z ~/container-onedrive.service /usr/lib/systemd/system +systemctl daemon-reload + +#optionally enable it to startup on boot + +systemctl enable container-onedrive.service + +#check status + +systemctl status container-onedrive + +#start/stop/restart container as a systemd service + +systemctl stop container-onedrive +systemctl start container-onedrive +``` + +To update the image using podman (Ad-hoc) +``` +podman auto-update +``` + +To update the image using systemd (Automatic/Scheduled) +``` +# Enable the podman-auto-update.timer service at system start: + +systemctl enable podman-auto-update.timer + +# Start the service + +systemctl start podman-auto-update.timer + +# Containers with the autoupdate label will be updated on the next scheduled timer + +systemctl list-timers --all +``` + +### 6. Edit the config +The 'onedrive' client should run in default configuration, however you can change this default configuration by placing a custom config file in the `onedrive_conf` podman volume. First download the default config from [here](https://raw.githubusercontent.com/abraunegg/onedrive/master/config) +Then put it into your onedrive_conf volume path, which can be found with: + +```bash +podman volume inspect onedrive_conf +``` +Or you can map your own config folder to the config volume. Make sure to copy all files from the volume into your mapped folder first. + +The detailed document for the config can be found here: [Configuration](https://github.com/abraunegg/onedrive/blob/master/docs/usage.md#configuration) + +### 7. Sync multiple accounts +There are many ways to do this, the easiest is probably to +1. Create a second podman config volume (replace `Work` with your desired name): `podman volume create onedrive_conf_Work` +2. And start a second podman monitor container (again replace `Work` with your desired name): +``` +export ONEDRIVE_DATA_DIR_WORK="/home/abraunegg/OneDriveWork" +mkdir -p ${ONEDRIVE_DATA_DIR_WORK} +podman run -it --restart unless-stopped --name onedrive_work \ + -v onedrive_conf_Work:/onedrive/conf \ + -v "${ONEDRIVE_DATA_DIR_WORK}:/onedrive/data" \ + --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" \ + driveone/onedrive:latest +``` + +## Environment Variables +| Variable | Purpose | Sample Value | +| ---------------- | --------------------------------------------------- |:-------------:| +| ONEDRIVE_UID | UserID (UID) to run as | 1000 | +| ONEDRIVE_GID | GroupID (GID) to run as | 1000 | +| ONEDRIVE_VERBOSE | Controls "--verbose" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_DEBUG | Controls "--verbose --verbose" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_DEBUG_HTTPS | Controls "--debug-https" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_RESYNC | Controls "--resync" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_DOWNLOADONLY | Controls "--download-only" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_UPLOADONLY | Controls "--upload-only" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_NOREMOTEDELETE | Controls "--no-remote-delete" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_LOGOUT | Controls "--logout" switch. Default is 0 | 1 | +| ONEDRIVE_REAUTH | Controls "--reauth" switch. Default is 0 | 1 | +| ONEDRIVE_AUTHFILES | Controls "--auth-files" option. Default is "" | "authUrl:responseUrl" | +| ONEDRIVE_AUTHRESPONSE | Controls "--auth-response" option. Default is "" | See [here](https://github.com/abraunegg/onedrive/blob/master/docs/usage.md#authorize-the-application-with-your-onedrive-account) | +| ONEDRIVE_DISPLAY_CONFIG | Controls "--display-running-config" switch on onedrive sync. Default is 0 | 1 | +| ONEDRIVE_SINGLE_DIRECTORY | Controls "--single-directory" option. Default = "" | "mydir" | + +### Usage Examples +**Verbose Output:** +```bash +podman run -e ONEDRIVE_VERBOSE=1 -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" driveone/onedrive:latest +``` +**Debug Output:** +```bash +podman run -e ONEDRIVE_DEBUG=1 -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" driveone/onedrive:latest +``` +**Perform a --resync:** +```bash +podman run -e ONEDRIVE_RESYNC=1 -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" driveone/onedrive:latest +``` +**Perform a --resync and --verbose:** +```bash +podman run -e ONEDRIVE_RESYNC=1 -e ONEDRIVE_VERBOSE=1 -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" driveone/onedrive:latest +``` +**Perform a --logout and re-authenticate:** +```bash +podman run -it -e ONEDRIVE_LOGOUT=1 -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" driveone/onedrive:latest +``` + +## Build instructions +### Building a custom Podman image +You can also build your own image instead of pulling the one from [hub.docker.com](https://hub.docker.com/r/driveone/onedrive): +```bash +git clone https://github.com/abraunegg/onedrive +cd onedrive +podman build . -t local-onedrive -f contrib/docker/Dockerfile +``` + +There are alternate, smaller images available by building +Dockerfile-debian or Dockerfile-alpine. These [multi-stage builder pattern](https://docs.docker.com/develop/develop-images/multistage-build/) +Dockerfiles require Docker version at least 17.05. + +#### How to build and run a custom Podman image based on Debian +``` bash +podman build . -t local-ondrive-debian -f contrib/docker/Dockerfile-debian +podman run -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" local-ondrive-debian:latest +``` + +#### How to build and run a custom Podman image based on Alpine Linux +``` bash +podman build . -t local-ondrive-alpine -f contrib/docker/Dockerfile-alpine +podman run -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" local-ondrive-alpine:latest +``` + +#### How to build and run a custom Podman image for ARMHF (Raspberry Pi) +Compatible with: +* Raspberry Pi +* Raspberry Pi 2 +* Raspberry Pi Zero +* Raspberry Pi 3 +* Raspberry Pi 4 +``` bash +podman build . -t local-onedrive-armhf -f contrib/docker/Dockerfile-debian +podman run -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" local-onedrive-armhf:latest +``` + +#### How to build and run a custom Podman image for AARCH64 Platforms +``` bash +podman build . -t local-onedrive-aarch64 -f contrib/docker/Dockerfile-debian +podman run -v onedrive_conf:/onedrive/conf:U,Z -v "${ONEDRIVE_DATA_DIR}:/onedrive/data:U,Z" --user "${ONEDRIVE_UID}:${ONEDRIVE_GID}" local-onedrive-aarch64:latest +``` diff --git a/docs/privacy-policy.md b/docs/privacy-policy.md new file mode 100644 index 00000000..64fe1dd3 --- /dev/null +++ b/docs/privacy-policy.md @@ -0,0 +1,65 @@ +# Privacy Policy +Effective Date: May 16 2018 + +## Introduction + +This Privacy Policy outlines how OneDrive Client for Linux ("we," "our," or "us") collects, uses, and protects information when you use our software ("OneDrive Client for Linux"). We respect your privacy and are committed to ensuring the confidentiality and security of any information you provide while using the Software. + +## Information We Do Not Collect + +We want to be transparent about the fact that we do not collect any personal data, usage data, or tracking data through the Software. This means: + +1. **No Personal Data**: We do not collect any information that can be used to personally identify you, such as your name, email address, phone number, or physical address. + +2. **No Usage Data**: We do not collect data about how you use the Software, such as the features you use, the duration of your sessions, or any interactions within the Software. + +3. **No Tracking Data**: We do not use cookies or similar tracking technologies to monitor your online behavior or track your activities across websites or apps. + +## How We Use Your Information + +Since we do not collect any personal, usage, or tracking data, there is no information for us to use for any purpose. + +## Third-Party Services + +The Software may include links to third-party websites or services, but we do not have control over the privacy practices or content of these third-party services. We encourage you to review the privacy policies of any third-party services you access through the Software. + +## Children's Privacy + +Since we do not collect any personal, usage, or tracking data, there is no restriction on the use of this application by anyone under the age of 18. + +## Information You Choose to Share + +While we do not collect personal data, usage data, or tracking data through the Software, there may be instances where you voluntarily choose to share information with us, particularly when submitting bug reports. These bug reports may contain sensitive information such as account details, file names, and directory names. It's important to note that these details are included in the logs and debug logs solely for the purpose of diagnosing and resolving technical issues with the Software. + +We want to emphasize that, even in these cases, we do not have access to your actual data. The logs and debug logs provided in bug reports are used exclusively for technical troubleshooting and debugging purposes. We take measures to treat this information with the utmost care, and it is only accessible to our technical support and development teams. We do not use this information for any other purpose, and we have strict security measures in place to protect it. + +## Protecting Your Sensitive Data + +We are committed to safeguarding your sensitive data and maintaining its confidentiality. To ensure its protection: + +1. **Limited Access**: Only authorized personnel within our technical support and development teams have access to the logs and debug logs containing sensitive data, and they are trained in handling this information securely. + +2. **Data Encryption**: We use industry-standard encryption protocols to protect the transmission and storage of sensitive data. + +3. **Data Retention**: We retain bug report data for a limited time necessary for resolving the reported issue. Once the issue is resolved, we promptly delete or anonymize the data. + +4. **Security Measures**: We employ robust security measures to prevent unauthorized access, disclosure, or alteration of sensitive data. + +By submitting a bug report, you acknowledge and consent to the inclusion of sensitive information in logs and debug logs for the sole purpose of addressing technical issues with the Software. + +## Your Responsibilities + +While we take measures to protect your sensitive data, it is essential for you to exercise caution when submitting bug reports. Please refrain from including any sensitive or personally identifiable information that is not directly related to the technical issue you are reporting. You have the option to redact or obfuscate sensitive details in bug reports to further protect your data. + +## Changes to this Privacy Policy + +We may update this Privacy Policy from time to time to reflect changes in our practices or for other operational, legal, or regulatory reasons. We will notify you of any material changes by posting the updated Privacy Policy on our website or through the Software. We encourage you to review this Privacy Policy periodically. + +## Contact Us + +If you have any questions or concerns about this Privacy Policy or our privacy practices, please contact us at support@mynas.com.au or via GitHub (https://github.com/abraunegg/onedrive) + +## Conclusion + +By using the Software, you agree to the terms outlined in this Privacy Policy. If you do not agree with any part of this policy, please discontinue the use of the Software. + diff --git a/docs/sharepoint-libraries.md b/docs/sharepoint-libraries.md new file mode 100644 index 00000000..d1714d4e --- /dev/null +++ b/docs/sharepoint-libraries.md @@ -0,0 +1,228 @@ +# How to configure OneDrive SharePoint Shared Library sync +**WARNING:** Several users have reported files being overwritten causing data loss as a result of using this client with SharePoint Libraries when running as a systemd service. + +When this has been investigated, the following has been noted as potential root causes: +* File indexing application such as Baloo File Indexer or Tracker3 constantly indexing your OneDrive data +* The use of WPS Office and how it 'saves' files by deleting the existing item and replaces it with the saved data + +Additionally there could be a yet unknown bug with the client, however all debugging and data provided previously shows that an 'external' process to the 'onedrive' application modifies the files triggering the undesirable upload to occur. + +**Possible Preventative Actions:** +* Disable all File Indexing for your SharePoint Library data. It is out of scope to detail on how you should do this. +* Disable using a systemd service for syncing your SharePoint Library data. +* Do not use WPS Office to edit your documents. Use OpenOffice or LibreOffice as these do not exhibit the same 'delete to save' action that WPS Office has. + +Additionally, please use caution when using this client with SharePoint. + +## Application Version +Before reading this document, please ensure you are running application version [![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) or greater. Use `onedrive --version` to determine what application version you are using and upgrade your client if required. + +## Process Overview +Syncing a OneDrive SharePoint library requires additional configuration for your 'onedrive' client: +1. Login to OneDrive and under 'Shared Libraries' obtain the shared library name +2. Query that shared library name using the client to obtain the required configuration details +3. Create a unique local folder which will be the SharePoint Library 'root' +4. Configure the client's config file with the required 'drive_id' +5. Test the configuration using '--dry-run' +6. Sync the SharePoint Library as required + +**Note:** The `--get-O365-drive-id` process below requires a fully configured 'onedrive' configuration so that the applicable Drive ID for the given Office 365 SharePoint Shared Library can be determined. It is highly recommended that you do not use the application 'default' configuration directory for any SharePoint Site, and configure separate items for each site you wish to use. + +## 1. Listing available OneDrive SharePoint Libraries +Login to the OneDrive web interface and determine which shared library you wish to configure the client for: +![shared_libraries](./images/SharedLibraries.jpg) + +## 2. Query OneDrive API to obtain required configuration details +Run the following command using the 'onedrive' client to query the OneDrive API to obtain the required 'drive_id' of the SharePoint Library that you wish to sync: +```text +onedrive --get-O365-drive-id '' +``` +This will return something similar to the following: +```text +Configuration file successfully loaded +Configuring Global Azure AD Endpoints +Initializing the Synchronization Engine ... +Office 365 Library Name Query: +----------------------------------------------- +Site Name: +Library Name: +drive_id: b!6H_y8B...xU5 +Library URL: +----------------------------------------------- +``` +If there are no matches to the site you are attempting to search, the following will be displayed: +```text +Configuration file successfully loaded +Configuring Global Azure AD Endpoints +Initializing the Synchronization Engine ... +Office 365 Library Name Query: blah + +ERROR: The requested SharePoint site could not be found. Please check it's name and your permissions to access the site. + +The following SharePoint site names were returned: + * + * + ... + * +``` +This list of site names can be used as a basis to search for the correct site for which you are searching + +## 3. Create a new configuration directory and sync location for this SharePoint Library +Create a new configuration directory for this SharePoint Library in the following manner: +```text +mkdir ~/.config/SharePoint_My_Library_Name +``` + +Create a new local folder to store the SharePoint Library data in: +```text +mkdir ~/SharePoint_My_Library_Name +``` + +**Note:** Do not use spaces in the directory name, use '_' as a replacement + +## 4. Configure SharePoint Library config file with the required 'drive_id' & 'sync_dir' options +Download a copy of the default configuration file by downloading this file from GitHub and saving this file in the directory created above: +```text +wget https://raw.githubusercontent.com/abraunegg/onedrive/master/config -O ~/.config/SharePoint_My_Library_Name/config +``` + +Update your 'onedrive' configuration file (`~/.config/SharePoint_My_Library_Name/config`) with the local folder where you will store your data: +```text +sync_dir = "~/SharePoint_My_Library_Name" +``` + +Update your 'onedrive' configuration file(`~/.config/SharePoint_My_Library_Name/config`) with the 'drive_id' value obtained in the steps above: +```text +drive_id = "insert the drive_id value from above here" +``` +The OneDrive client will now be configured to sync this SharePoint shared library to your local system and the location you have configured. + +**Note:** After changing `drive_id`, you must perform a full re-synchronization by adding `--resync` to your existing command line. + +## 5. Validate and Test the configuration +Validate your new configuration using the `--display-config` option to validate you have configured the application correctly: +```text +onedrive --confdir="~/.config/SharePoint_My_Library_Name" --display-config +``` + +Test your new configuration using the `--dry-run` option to validate the application configuration: +```text +onedrive --confdir="~/.config/SharePoint_My_Library_Name" --synchronize --verbose --dry-run +``` + +**Note:** As this is a *new* configuration, the application will be required to be re-authorised the first time this command is run with the new configuration. + +## 6. Sync the SharePoint Library as required +Sync the SharePoint Library to your system with either `--synchronize` or `--monitor` operations: +```text +onedrive --confdir="~/.config/SharePoint_My_Library_Name" --synchronize --verbose +``` + +```text +onedrive --confdir="~/.config/SharePoint_My_Library_Name" --monitor --verbose +``` + +**Note:** As this is a *new* configuration, the application will be required to be re-authorised the first time this command is run with the new configuration. + +## 7. Enable custom systemd service for SharePoint Library +Systemd can be used to automatically run this configuration in the background, however, a unique systemd service will need to be setup for this SharePoint Library instance + +In order to automatically start syncing each SharePoint Library, you will need to create a service file for each SharePoint Library. From the applicable 'systemd folder' where the applicable systemd service file exists: +* RHEL / CentOS: `/usr/lib/systemd/system` +* Others: `/usr/lib/systemd/user` and `/lib/systemd/system` + +### Step1: Create a new systemd service file +#### Red Hat Enterprise Linux, CentOS Linux +Copy the required service file to a new name: +```text +sudo cp /usr/lib/systemd/system/onedrive.service /usr/lib/systemd/system/onedrive-SharePoint_My_Library_Name.service +``` +or +```text +sudo cp /usr/lib/systemd/system/onedrive@.service /usr/lib/systemd/system/onedrive-SharePoint_My_Library_Name@.service +``` + +#### Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +Copy the required service file to a new name: +```text +sudo cp /usr/lib/systemd/user/onedrive.service /usr/lib/systemd/user/onedrive-SharePoint_My_Library_Name.service +``` +or +```text +sudo cp /lib/systemd/system/onedrive@.service /lib/systemd/system/onedrive-SharePoint_My_Library_Name@.service +``` + +### Step 2: Edit new systemd service file +Edit the new systemd file, updating the line beginning with `ExecStart` so that the confdir mirrors the one you used above: +```text +ExecStart=/usr/local/bin/onedrive --monitor --confdir="/full/path/to/config/dir" +``` + +Example: +```text +ExecStart=/usr/local/bin/onedrive --monitor --confdir="/home/myusername/.config/SharePoint_My_Library_Name" +``` + +**Note:** When running the client manually, `--confdir="~/.config/......` is acceptable. In a systemd configuration file, the full path must be used. The `~` must be expanded. + +### Step 3: Enable the new systemd service +Once the file is correctly editied, you can enable the new systemd service using the following commands. + +#### Red Hat Enterprise Linux, CentOS Linux +```text +systemctl enable onedrive-SharePoint_My_Library_Name +systemctl start onedrive-SharePoint_My_Library_Name +``` + +#### Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +```text +systemctl --user enable onedrive-SharePoint_My_Library_Name +systemctl --user start onedrive-SharePoint_My_Library_Name +``` +or +```text +systemctl --user enable onedrive-SharePoint_My_Library_Name@myusername.service +systemctl --user start onedrive-SharePoint_My_Library_Name@myusername.service +``` + +### Step 4: Viewing systemd status and logs for the custom service +#### Viewing systemd service status - Red Hat Enterprise Linux, CentOS Linux +```text +systemctl status onedrive-SharePoint_My_Library_Name +``` + +#### Viewing systemd service status - Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +```text +systemctl --user status onedrive-SharePoint_My_Library_Name +``` + +#### Viewing journalctl systemd logs - Red Hat Enterprise Linux, CentOS Linux +```text +journalctl --unit=onedrive-SharePoint_My_Library_Name -f +``` + +#### Viewing journalctl systemd logs - Others such as Arch, Ubuntu, Debian, OpenSuSE, Fedora +```text +journalctl --user --unit=onedrive-SharePoint_My_Library_Name -f +``` + +### Step 5: (Optional) Run custom systemd service at boot without user login +In some cases it may be desirable for the systemd service to start without having to login as your 'user' + +All the systemd steps above that utilise the `--user` option, will run the systemd service as your particular user. As such, the systemd service will not start unless you actually login to your system. + +To avoid this issue, you need to reconfigure your 'user' account so that the systemd services you have created will startup without you having to login to your system: +```text +loginctl enable-linger +``` + +Example: +```text +alex@ubuntu-headless:~$ loginctl enable-linger alex +``` + +## 8. Configuration for a SharePoint Library is complete +The 'onedrive' client configuration for this particular SharePoint Library is now complete. + +# How to configure multiple OneDrive SharePoint Shared Library sync +Create a new configuration as per the process above. Repeat these steps for each SharePoint Library that you wish to use. diff --git a/docs/terms-of-service.md b/docs/terms-of-service.md new file mode 100644 index 00000000..cdf7c432 --- /dev/null +++ b/docs/terms-of-service.md @@ -0,0 +1,54 @@ +# OneDrive Client for Linux - Software Service Terms of Service + +## 1. Introduction + +These Terms of Service ("Terms") govern your use of the OneDrive Client for Linux ("Application") software and related Microsoft OneDrive services ("Service") provided by Microsoft. By accessing or using the Service, you agree to comply with and be bound by these Terms. If you do not agree to these Terms, please do not use the Service. + +## 2. License Compliance + +The OneDrive Client for Linux software is licensed under the GNU General Public License, version 3.0 (the "GPLv3"). Your use of the software must comply with the terms and conditions of the GPLv3. A copy of the GPLv3 can be found here: https://www.gnu.org/licenses/gpl-3.0.en.html + +## 3. Use of the Service + +### 3.1. Access and Accounts + +You may need to create an account or provide personal information to access certain features of the Service. You are responsible for maintaining the confidentiality of your account information and are solely responsible for all activities that occur under your account. + +### 3.2. Prohibited Activities + +You agree not to: + +- Use the Service in any way that violates applicable laws or regulations. +- Use the Service to engage in any unlawful, harmful, or fraudulent activity. +- Use the Service in any manner that disrupts, damages, or impairs the Service. + +## 4. Intellectual Property + +The OneDrive Client for Linux software is subject to the GPLv3, and you must respect all copyrights, trademarks, and other intellectual property rights associated with the software. Any contributions you make to the software must also comply with the GPLv3. + +## 5. Disclaimer of Warranties + +The OneDrive Client for Linux software is provided "as is" without any warranties, either expressed or implied. We do not guarantee that the use of the Application will be error-free or uninterrupted. + +Microsoft is not responsible for OneDrive Client for Linux. Any issues or problems with OneDrive Client for Linux should be raised on GitHub at https://github.com/abraunegg/onedrive or email support@mynas.com.au + +OneDrive Client for Linux is not responsible for the Microsoft OneDrive Service or the Microsoft Graph API Service that this Application utilizes. Any issue with either Microsoft OneDrive or Microsoft Graph API should be raised with Microsoft via their support channel in your country. + +## 6. Limitation of Liability + +To the fullest extent permitted by law, we shall not be liable for any direct, indirect, incidental, special, consequential, or punitive damages, or any loss of profits or revenues, whether incurred directly or indirectly, or any loss of data, use, goodwill, or other intangible losses, resulting from (a) your use or inability to use the Service, or (b) any other matter relating to the Service. + +This limitiation of liability explicitly relates to the use of the OneDrive Client for Linux software and does not affect your rights under the GPLv3. + +## 7. Changes to Terms + +We reserve the right to update or modify these Terms at any time without prior notice. Any changes will be effective immediately upon posting on GitHub. Your continued use of the Service after the posting of changes constitutes your acceptance of such changes. Changes can be reviewed on GitHub. + +## 8. Governing Law + +These Terms shall be governed by and construed in accordance with the laws of Australia, without regard to its conflict of law principles. + +## 9. Contact Us + +If you have any questions or concerns about these Terms, please contact us at https://github.com/abraunegg/onedrive or email support@mynas.com.au + diff --git a/docs/ubuntu-package-install.md b/docs/ubuntu-package-install.md new file mode 100644 index 00000000..abdcace9 --- /dev/null +++ b/docs/ubuntu-package-install.md @@ -0,0 +1,383 @@ +# Installation of 'onedrive' package on Debian and Ubuntu + +This document covers the appropriate steps to install the 'onedrive' client using the provided packages for Debian and Ubuntu. + +#### Important information for all Ubuntu and Ubuntu based distribution users: +This information is specifically for the following platforms and distributions: + +* Lubuntu +* Linux Mint +* POP OS +* Peppermint OS +* Raspbian +* Ubuntu + +Whilst there are [onedrive](https://packages.ubuntu.com/search?keywords=onedrive&searchon=names&suite=all§ion=all) Universe packages available for Ubuntu, do not install 'onedrive' from these Universe packages. The default Ubuntu Universe packages are out-of-date and are not supported and should not be used. + +## Determine which instructions to use +Ubuntu and its clones are based on various different releases, thus, you must use the correct instructions below, otherwise you may run into package dependancy issues and will be unable to install the client. + +### Step 1: Remove any configured PPA and associated 'onedrive' package and systemd service files +Many Internet 'help' pages provide inconsistent details on how to install the OneDrive Client for Linux. A number of these websites continue to point users to install the client via the yann1ck PPA repository however this PPA no longer exists and should not be used. + +To remove the PPA repository and the older client, perform the following actions: +```text +sudo apt remove onedrive +sudo add-apt-repository --remove ppa:yann1ck/onedrive +``` + +Additionally, Ubuntu and its clones have a bad habit of creating a 'default' systemd service file when installing the 'onedrive' package so that the client will automatically run the client post being authenticated. This systemd entry is erroneous and needs to be removed. +``` +Created symlink /etc/systemd/user/default.target.wants/onedrive.service → /usr/lib/systemd/user/onedrive.service. +``` +To remove this symbolic link, run the following command: +``` +sudo rm /etc/systemd/user/default.target.wants/onedrive.service +``` + +### Step 2: Ensure your system is up-to-date +Use a script, similar to the following to ensure your system is updated correctly: +```text +#!/bin/bash +rm -rf /var/lib/dpkg/lock-frontend +rm -rf /var/lib/dpkg/lock +apt-get update +apt-get upgrade -y +apt-get dist-upgrade -y +apt-get autoremove -y +apt-get autoclean -y +``` + +Run this script as 'root' by using `su -` to elevate to 'root'. Example below: +```text +Welcome to Ubuntu 20.04.1 LTS (GNU/Linux 5.4.0-48-generic x86_64) + + * Documentation: https://help.ubuntu.com + * Management: https://landscape.canonical.com + * Support: https://ubuntu.com/advantage + +425 updates can be installed immediately. +208 of these updates are security updates. +To see these additional updates run: apt list --upgradable + +Your Hardware Enablement Stack (HWE) is supported until April 2025. +Last login: Thu Jan 20 14:21:48 2022 from my.ip.address +alex@ubuntu-20-LTS:~$ su - +Password: +root@ubuntu-20-LTS:~# ls -la +total 28 +drwx------ 3 root root 4096 Oct 10 2020 . +drwxr-xr-x 20 root root 4096 Oct 10 2020 .. +-rw------- 1 root root 175 Jan 20 14:23 .bash_history +-rw-r--r-- 1 root root 3106 Dec 6 2019 .bashrc +drwx------ 2 root root 4096 Apr 23 2020 .cache +-rw-r--r-- 1 root root 161 Dec 6 2019 .profile +-rwxr-xr-x 1 root root 174 Oct 10 2020 update-os.sh +root@ubuntu-20-LTS:~# cat update-os.sh +#!/bin/bash +rm -rf /var/lib/dpkg/lock-frontend +rm -rf /var/lib/dpkg/lock +apt-get update +apt-get upgrade -y +apt-get dist-upgrade -y +apt-get autoremove -y +apt-get autoclean -y +root@ubuntu-20-LTS:~# ./update-os.sh +Hit:1 http://au.archive.ubuntu.com/ubuntu focal InRelease +Hit:2 http://au.archive.ubuntu.com/ubuntu focal-updates InRelease +Hit:3 http://au.archive.ubuntu.com/ubuntu focal-backports InRelease +Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease +Reading package lists... 96% +... +Sourcing file `/etc/default/grub' +Sourcing file `/etc/default/grub.d/init-select.cfg' +Generating grub configuration file ... +Found linux image: /boot/vmlinuz-5.13.0-27-generic +Found initrd image: /boot/initrd.img-5.13.0-27-generic +Found linux image: /boot/vmlinuz-5.4.0-48-generic +Found initrd image: /boot/initrd.img-5.4.0-48-generic +Found memtest86+ image: /boot/memtest86+.elf +Found memtest86+ image: /boot/memtest86+.bin +done +Removing linux-modules-5.4.0-26-generic (5.4.0-26.30) ... +Processing triggers for libc-bin (2.31-0ubuntu9.2) ... +Reading package lists... Done +Building dependency tree +Reading state information... Done +root@ubuntu-20-LTS:~# +``` + +Reboot your system after running this process before continuing with Step 3. +```text +reboot +``` + +### Step 3: Determine what your OS is based on +Determine what your OS is based on. To do this, run the following command: +```text +lsb_release -a +``` +**Example:** +```text +alex@ubuntu-system:~$ lsb_release -a +No LSB modules are available. +Distributor ID: Ubuntu +Description: Ubuntu 22.04 LTS +Release: 22.04 +Codename: jammy +``` + +### Step 4: Pick the correct instructions to use +If required, review the table below based on your 'lsb_release' information to pick the appropriate instructions to use: + +| Release & Codename | Instructions to use | +|--------------------|---------------------| +| Linux Mint 19.x | This platform is End-of-Life (EOL) and no longer supported. You must upgrade to Linux Mint 21.x | +| Linux Mint 20.x | Use [Ubuntu 20.04](#distribution-ubuntu-2004) instructions below | +| Linux Mint 21.x | Use [Ubuntu 22.04](#distribution-ubuntu-2204) instructions below | +| Debian 9 | This platform is End-of-Life (EOL) and no longer supported. You must upgrade to Debian 11 | +| Debian 10 | You must build from source or upgrade your Operating System to Debian 11 | +| Debian 11 | Use [Debian 11](#distribution-debian-11) instructions below | +| Debian 12 | Use [Debian 12](#distribution-debian-12) instructions below | +| Raspbian GNU/Linux 10 | You must build from source or upgrade your Operating System to Raspbian GNU/Linux 11 | +| Raspbian GNU/Linux 11 | Use [Debian 11](#distribution-debian-11) instructions below | +| Raspbian GNU/Linux 12 | Use [Debian 12](#distribution-debian-12) instructions below | +| Ubuntu 18.04 / Bionic | This platform is End-of-Life (EOL) and no longer supported. You must upgrade to Ubuntu 22.x | +| Ubuntu 20.04 / Focal | Use [Ubuntu 20.04](#distribution-ubuntu-2004) instructions below | +| Ubuntu 21.04 / Hirsute | Use [Ubuntu 21.04](#distribution-ubuntu-2104) instructions below | +| Ubuntu 21.10 / Impish | Use [Ubuntu 21.10](#distribution-ubuntu-2110) instructions below | +| Ubuntu 22.04 / Jammy | Use [Ubuntu 22.04](#distribution-ubuntu-2204) instructions below | +| Ubuntu 22.10 / Kinetic | Use [Ubuntu 22.10](#distribution-ubuntu-2210) instructions below | +| Ubuntu 23.04 / Lunar | Use [Ubuntu 23.04](#distribution-ubuntu-2304) instructions below | + +## Distribution Package Install Instructions + +### Distribution: Debian 11 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +|✔|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/Debian_11/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/obs-onedrive.gpg > /dev/null +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/obs-onedrive.gpg] https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/Debian_11/ ./" | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +### Distribution: Debian 12 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +|✔|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/Debian_12/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/obs-onedrive.gpg > /dev/null +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/obs-onedrive.gpg] https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/Debian_12/ ./" | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +### Distribution: Ubuntu 20.04 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +❌|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_20.04/Release.key | sudo apt-key add - +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo 'deb https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_20.04/ ./' | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +### Distribution: Ubuntu 21.04 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +❌|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_21.04/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/obs-onedrive.gpg > /dev/null +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/obs-onedrive.gpg] https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_21.04/ ./" | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +### Distribution: Ubuntu 21.10 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +❌|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_21.10/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/obs-onedrive.gpg > /dev/null +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/obs-onedrive.gpg] https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_21.10/ ./" | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +### Distribution: Ubuntu 22.04 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +❌|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_22.04/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/obs-onedrive.gpg > /dev/null +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/obs-onedrive.gpg] https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_22.04/ ./" | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +### Distribution: Ubuntu 22.10 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +❌|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_22.10/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/obs-onedrive.gpg > /dev/null +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/obs-onedrive.gpg] https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_22.10/ ./" | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +### Distribution: Ubuntu 23.04 +The packages support the following platform architectures: +|  i686  | x86_64 | ARMHF | AARCH64 | +|:----:|:------:|:-----:|:-------:| +❌|✔|✔|✔| | + +#### Step 1: Add the OpenSuSE Build Service repository release key +Add the OpenSuSE Build Service repository release key using the following command: +```text +wget -qO - https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_23.04/Release.key | gpg --dearmor | sudo tee /usr/share/keyrings/obs-onedrive.gpg > /dev/null +``` + +#### Step 2: Add the OpenSuSE Build Service repository +Add the OpenSuSE Build Service repository using the following command: +```text +echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/obs-onedrive.gpg] https://download.opensuse.org/repositories/home:/npreining:/debian-ubuntu-onedrive/xUbuntu_23.04/ ./" | sudo tee /etc/apt/sources.list.d/onedrive.list +``` + +#### Step 3: Update your apt package cache +Run: `sudo apt-get update` + +#### Step 4: Install 'onedrive' +Run: `sudo apt install --no-install-recommends --no-install-suggests onedrive` + +#### Step 5: Read 'Known Issues' with these packages +Read and understand the [known issues](#known-issues-with-installing-from-the-above-packages) with these packages below, taking any action that is needed. + +## Known Issues with Installing from the above packages + +### 1. The client may segfault | core-dump when exiting +When the client is run in `--monitor` mode manually, or when using the systemd service, the client may segfault on exit. + +This issue is caused by the way the 'onedrive' packages are built using the distribution LDC package & the default distribution compiler options which is the root cause for this issue. Refer to: https://bugs.launchpad.net/ubuntu/+source/ldc/+bug/1895969 + +**Additional references:** +* https://github.com/abraunegg/onedrive/issues/1053 +* https://github.com/abraunegg/onedrive/issues/1609 + +**Resolution Options:** +* Uninstall the package and build client from source diff --git a/docs/usage.md b/docs/usage.md new file mode 100644 index 00000000..b7a6c4cd --- /dev/null +++ b/docs/usage.md @@ -0,0 +1,1468 @@ +# Configuration and Usage of the OneDrive Free Client +## Application Version +Before reading this document, please ensure you are running application version [![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) or greater. Use `onedrive --version` to determine what application version you are using and upgrade your client if required. + +## Table of Contents +- [Using the client](#using-the-client) + * [Upgrading from 'skilion' client](#upgrading-from-skilion-client) + * [Local File and Folder Naming Conventions](#local-file-and-folder-naming-conventions) + * [curl compatibility](#curl-compatibility) + * [Authorize the application with your OneDrive Account](#authorize-the-application-with-your-onedrive-account) + * [Show your configuration](#show-your-configuration) + * [Testing your configuration](#testing-your-configuration) + * [Performing a sync](#performing-a-sync) + * [Performing a single directory sync](#performing-a-single-directory-sync) + * [Performing a 'one-way' download sync](#performing-a-one-way-download-sync) + * [Performing a 'one-way' upload sync](#performing-a-one-way-upload-sync) + * [Performing a selective sync via 'sync_list' file](#performing-a-selective-sync-via-sync_list-file) + * [Performing a --resync](#performing-a---resync) + * [Performing a --force-sync without a --resync or changing your configuration](#performing-a---force-sync-without-a---resync-or-changing-your-configuration) + * [Increasing logging level](#increasing-logging-level) + * [Client Activity Log](#client-activity-log) + * [Notifications](#notifications) + * [Handling a OneDrive account password change](#handling-a-onedrive-account-password-change) +- [Configuration](#configuration) + * [The default configuration](#the-default-configuration-file-is-listed-below) + * ['config' file configuration examples](#config-file-configuration-examples) + + [sync_dir](#sync_dir) + + [sync_dir directory and file permissions](#sync_dir-directory-and-file-permissions) + + [skip_dir](#skip_dir) + + [skip_file](#skip_file) + + [skip_dotfiles](#skip_dotfiles) + + [monitor_interval](#monitor_interval) + + [monitor_fullscan_frequency](#monitor_fullscan_frequency) + + [monitor_log_frequency](#monitor_log_frequency) + + [min_notify_changes](#min_notify_changes) + + [operation_timeout](#operation_timeout) + + [ip_protocol_version](#ip_protocol_version) + + [classify_as_big_delete](#classify_as_big_delete) + * [Configuring the client for 'single tenant application' use](#configuring-the-client-for-single-tenant-application-use) + * [Configuring the client to use older 'skilion' application identifier](#configuring-the-client-to-use-older-skilion-application-identifier) +- [Frequently Asked Configuration Questions](#frequently-asked-configuration-questions) + * [How to sync only specific or single directory?](#how-to-sync-only-specific-or-single-directory) + * [How to 'skip' directories from syncing?](#how-to-skip-directories-from-syncing) + * [How to 'skip' files from syncing?](#how-to-skip-files-from-syncing) + * [How to 'skip' dot files and folders from syncing?](#how-to-skip-dot-files-and-folders-from-syncing) + * [How to 'skip' files larger than a certain size from syncing?](#how-to-skip-files-larger-than-a-certain-size-from-syncing) + * [How to 'rate limit' the application to control bandwidth consumed for upload & download operations?](#how-to-rate-limit-the-application-to-control-bandwidth-consumed-for-upload--download-operations) + * [How to prevent your local disk from filling up?](#how-to-prevent-your-local-disk-from-filling-up) + * [How are symbolic links handled by the client?](#how-are-symbolic-links-handled-by-the-client) + * [How to sync shared folders (OneDrive Personal)?](#how-to-sync-shared-folders-onedrive-personal) + * [How to sync shared folders (OneDrive Business or Office 365)?](#how-to-sync-shared-folders-onedrive-business-or-office-365) + * [How to sync sharePoint / Office 365 Shared Libraries?](#how-to-sync-sharepoint--office-365-shared-libraries) + * [How to run a user systemd service at boot without user login?](#how-to-run-a-user-systemd-service-at-boot-without-user-login) + * [How to create a shareable link?](#how-to-create-a-shareable-link) + * [How to sync both Personal and Business accounts at the same time?](#how-to-sync-both-personal-and-business-accounts-at-the-same-time) + * [How to sync multiple SharePoint Libraries at the same time?](#how-to-sync-multiple-sharepoint-libraries-at-the-same-time) +- [Running 'onedrive' in 'monitor' mode](#running-onedrive-in-monitor-mode) + * [Use webhook to subscribe to remote updates in 'monitor' mode](#use-webhook-to-subscribe-to-remote-updates-in-monitor-mode) + * [More webhook configuration options](#more-webhook-configuration-options) + + [webhook_listening_host and webhook_listening_port](#webhook_listening_host-and-webhook_listening_port) + + [webhook_expiration_interval and webhook_renewal_interval](#webhook_expiration_interval-and-webhook_renewal_interval) +- [Running 'onedrive' as a system service](#running-onedrive-as-a-system-service) + * [OneDrive service running as root user via init.d](#onedrive-service-running-as-root-user-via-initd) + * [OneDrive service running as root user via systemd (Arch, Ubuntu, Debian, OpenSuSE, Fedora)](#onedrive-service-running-as-root-user-via-systemd-arch-ubuntu-debian-opensuse-fedora) + * [OneDrive service running as root user via systemd (Red Hat Enterprise Linux, CentOS Linux)](#onedrive-service-running-as-root-user-via-systemd-red-hat-enterprise-linux-centos-linux) + * [OneDrive service running as a non-root user via systemd (All Linux Distributions)](#onedrive-service-running-as-a-non-root-user-via-systemd-all-linux-distributions) + * [OneDrive service running as a non-root user via systemd (with notifications enabled) (Arch, Ubuntu, Debian, OpenSuSE, Fedora)](#onedrive-service-running-as-a-non-root-user-via-systemd-with-notifications-enabled-arch-ubuntu-debian-opensuse-fedora) + * [OneDrive service running as a non-root user via runit (antiX, Devuan, Artix, Void)](#onedrive-service-running-as-a-non-root-user-via-runit-antix-devuan-artix-void) +- [Additional Configuration](#additional-configuration) + * [Advanced Configuration of the OneDrive Free Client](#advanced-configuration-of-the-onedrive-free-client) + * [Access OneDrive service through a proxy](#access-onedrive-service-through-a-proxy) + * [Setup selinux for a sync folder outside of the home folder](#setup-selinux-for-a-sync-folder-outside-of-the-home-folder) +- [All available commands](#all-available-commands) + +## Using the client +### Upgrading from 'skilion' client +The 'skilion' version contains a significant number of defects in how the local sync state is managed. When upgrading from the 'skilion' version to this version, it is advisable to stop any service / onedrive process from running and then remove any `items.sqlite3` file from your configuration directory (`~/.config/onedrive/`) as this will force the creation of a new local cache file. + +Additionally, if you are using a 'config' file within your configuration directory (`~/.config/onedrive/`), please ensure that you update the `skip_file = ` option as per below: + +**Invalid configuration:** +```text +skip_file = ".*|~*" +``` +**Minimum valid configuration:** +```text +skip_file = "~*" +``` +**Default valid configuration:** +```text +skip_file = "~*|.~*|*.tmp" +``` + +Do not use a skip_file entry of `.*` as this will prevent correct searching of local changes to process. + +### Local File and Folder Naming Conventions +The files and directories in the synchronization directory must follow the [Windows naming conventions](https://docs.microsoft.com/windows/win32/fileio/naming-a-file). +The application will attempt to handle instances where you have two files with the same names but with different capitalization. Where there is a namespace clash, the file name which clashes will not be synced. This is expected behavior and won't be fixed. + +### curl compatibility +If your system utilises curl < 7.47.0, curl defaults to HTTP/1.1 for HTTPS operations. The client will use HTTP/1.1. + +If your system utilises curl >= 7.47.0 and < 7.62.0, curl will prefer HTTP/2 for HTTPS but will stick to HTTP/1.1 by default. The client will use HTTP/1.1 for HTTPS operations. + +If your system utilises curl >= 7.62.0, curl defaults to prefer HTTP/2 over HTTP/1.1 by default. The client will utilse HTTP/2 for most HTTPS operations and HTTP/1.1 for others. This difference is governed by the OneDrive platform and not this client. + +If you wish to explicitly use HTTP/1.1 you will need to use the `--force-http-11` flag or set the config option `force_http_11 = "true"` to force the application to use HTTP/1.1 otherwise all client operations will use whatever is the curl default for your distribution. + +### Authorize the application with your OneDrive Account +After installing the application you must authorize the application with your OneDrive Account. This is done by running the application without any additional command switches. + +Note that some companies require to explicitly add this app in [Microsoft MyApps portal](https://myapps.microsoft.com/). To add an (approved) app to your apps, click on the ellipsis in the top-right corner and choose "Request new apps". On the next page you can add this app. If its not listed, you should request through your IT department. + +You will be asked to open a specific URL by using your web browser where you will have to login into your Microsoft Account and give the application the permission to access your files. After giving permission to the application, you will be redirected to a blank page. Copy the URI of the blank page into the application. +```text +[user@hostname ~]$ onedrive + +Authorize this app visiting: + +https://..... + +Enter the response uri: + +``` + +**Example:** +``` +[user@hostname ~]$ onedrive +Authorize this app visiting: + +https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=22c49a0d-d21c-4792-aed1-8f163c982546&scope=Files.ReadWrite%20Files.ReadWrite.all%20Sites.ReadWrite.All%20offline_access&response_type=code&redirect_uri=https://login.microsoftonline.com/common/oauth2/nativeclient + +Enter the response uri: https://login.microsoftonline.com/common/oauth2/nativeclient?code= + +Application has been successfully authorised, however no additional command switches were provided. + +Please use 'onedrive --help' for further assistance in regards to running this application. +``` + +### Show your configuration +To validate your configuration the application will use, utilize the following: +```text +onedrive --display-config +``` +This will display all the pertinent runtime interpretation of the options and configuration you are using. Example output is as follows: +```text +Configuration file successfully loaded +onedrive version = vX.Y.Z-A-bcdefghi +Config path = /home/alex/.config/onedrive +Config file found in config path = true +Config option 'sync_dir' = /home/alex/OneDrive +Config option 'enable_logging' = false +... +Selective sync 'sync_list' configured = false +Config option 'sync_business_shared_folders' = false +Business Shared Folders configured = false +Config option 'webhook_enabled' = false +``` + +### Testing your configuration +You are able to test your configuration by utilising the `--dry-run` CLI option. No files will be downloaded, uploaded or removed, however the application will display what 'would' have occurred. For example: +```text +onedrive --synchronize --verbose --dry-run +DRY-RUN Configured. Output below shows what 'would' have occurred. +Loading config ... +Using Config Dir: /home/user/.config/onedrive +Initializing the OneDrive API ... +Opening the item database ... +All operations will be performed in: /home/user/OneDrive +Initializing the Synchronization Engine ... +Account Type: personal +Default Drive ID: +Default Root ID: +Remaining Free Space: 5368709120 +Fetching details for OneDrive Root +OneDrive Root exists in the database +Syncing changes from OneDrive ... +Applying changes of Path ID: +Uploading differences of . +Processing root +The directory has not changed +Uploading new items of . +OneDrive Client requested to create remote path: ./newdir +The requested directory to create was not found on OneDrive - creating remote directory: ./newdir +Successfully created the remote directory ./newdir on OneDrive +Uploading new file ./newdir/newfile.txt ... done. +Remaining free space: 5368709076 +Applying changes of Path ID: +``` + +**Note:** `--dry-run` can only be used with `--synchronize`. It cannot be used with `--monitor` and will be ignored. + +### Performing a sync +By default all files are downloaded in `~/OneDrive`. After authorizing the application, a sync of your data can be performed by running: +```text +onedrive --synchronize +``` +This will synchronize files from your OneDrive account to your `~/OneDrive` local directory. + +If you prefer to use your local files as stored in `~/OneDrive` as the 'source of truth' use the following sync command: +```text +onedrive --synchronize --local-first +``` + +### Performing a single directory sync +In some cases it may be desirable to sync a single directory under ~/OneDrive without having to change your client configuration. To do this use the following command: +```text +onedrive --synchronize --single-directory '' +``` + +Example: If the full path is `~/OneDrive/mydir`, the command would be `onedrive --synchronize --single-directory 'mydir'` + +### Performing a 'one-way' download sync +In some cases it may be desirable to 'download only' from OneDrive. To do this use the following command: +```text +onedrive --synchronize --download-only +``` + +### Performing a 'one-way' upload sync +In some cases it may be desirable to 'upload only' to OneDrive. To do this use the following command: +```text +onedrive --synchronize --upload-only +``` +**Note:** If a file or folder is present on OneDrive, that was previously synced and now does not exist locally, that item it will be removed from OneDrive. If the data on OneDrive should be kept, the following should be used: +```text +onedrive --synchronize --upload-only --no-remote-delete +``` +**Note:** The operation of 'upload only' does not request data from OneDrive about what 'other' data exists online. The client only knows about the data that 'this' client uploaded, thus any files or folders created or uploaded outside of this client will remain untouched online. + +### Performing a selective sync via 'sync_list' file +Selective sync allows you to sync only specific files and directories. +To enable selective sync create a file named `sync_list` in your application configuration directory (default is `~/.config/onedrive`). + +Important points to understand before using 'sync_list'. +* 'sync_list' excludes _everything_ by default on onedrive. +* 'sync_list' follows an _"exclude overrides include"_ rule, and requires **explicit inclusion**. +* Order exclusions before inclusions, so that anything _specifically included_ is included. +* How and where you place your `/` matters for excludes and includes in sub directories. + +Each line of the file represents a relative path from your `sync_dir`. All files and directories not matching any line of the file will be skipped during all operations. + +Additionally, the use of `/` is critically important to determine how a rule is interpreted. It is very similar to `**` wildcards, for those that are familiar with globbing patterns. +Here is an example of `sync_list`: +```text +# sync_list supports comments +# +# The ordering of entries is highly recommended - exclusions before inclusions +# +# Exclude temp folder(s) or file(s) under Documents folder(s), anywhere in Onedrive +!Documents/temp* +# +# Exclude secret data folder in root directory only +!/Secret_data/* +# +# Include everything else in root directory +/* +# +# Include my Backup folder(s) or file(s) anywhere on Onedrive +Backup +# +# Include my Backup folder in root +/Backup/ +# +# Include Documents folder(s) anywhere in Onedrive +Documents/ +# +# Include all PDF files in Documents folder(s), anywhere in Onedrive +Documents/*.pdf +# +# Include this single document in Documents folder(s), anywhere in Onedrive +Documents/latest_report.docx +# +# Include all Work/Project directories or files, inside 'Work' folder(s), anywhere in Onedrive +Work/Project* +# +# Include all "notes.txt" files, anywhere in Onedrive +notes.txt +# +# Include /Blender in the ~Onedrive root but not if elsewhere in Onedrive +/Blender +# +# Include these directories(or files) in 'Pictures' folder(s), that have a space in their name +Pictures/Camera Roll +Pictures/Saved Pictures +# +# Include these names if they match any file or folder +Cinema Soc +Codes +Textbooks +Year 2 +``` +The following are supported for pattern matching and exclusion rules: +* Use the `*` to wildcard select any characters to match for the item to be included +* Use either `!` or `-` characters at the start of the line to exclude an otherwise included item + + +**Note:** When enabling the use of 'sync_list' utilise the `--display-config` option to validate that your configuration will be used by the application, and test your configuration by adding `--dry-run` to ensure the client will operate as per your requirement. + +**Note:** After changing the sync_list, you must perform a full re-synchronization by adding `--resync` to your existing command line - for example: `onedrive --synchronize --resync` + +**Note:** In some circumstances, it may be required to sync all the individual files within the 'sync_dir', but due to frequent name change / addition / deletion of these files, it is not desirable to constantly change the 'sync_list' file to include / exclude these files and force a resync. To assist with this, enable the following in your configuration file: +```text +sync_root_files = "true" +``` +This will tell the application to sync any file that it finds in your 'sync_dir' root by default. + +### Performing a --resync +If you modify any of the following configuration items, you will be required to perform a `--resync` to ensure your client is syncing your data with the updated configuration: +* sync_dir +* skip_dir +* skip_file +* drive_id +* Modifying sync_list +* Modifying business_shared_folders + +Additionally, you may choose to perform a `--resync` if you feel that this action needs to be taken to ensure your data is in sync. If you are using this switch simply because you dont know the sync status, you can query the actual sync status using `--display-sync-status`. + +When using `--resync`, the following warning and advice will be presented: +```text +The use of --resync will remove your local 'onedrive' client state, thus no record will exist regarding your current 'sync status' +This has the potential to overwrite local versions of files with potentially older versions downloaded from OneDrive which can lead to data loss +If in-doubt, backup your local data first before proceeding with --resync + +Are you sure you wish to proceed with --resync? [Y/N] +``` + +To proceed with using `--resync`, you must type 'y' or 'Y' to allow the application to continue. + +**Note:** It is highly recommended to only use `--resync` if the application advises you to use it. Do not just blindly set the application to start with `--resync` as the default option. + +**Note:** In some automated environments (and it is 100% assumed you *know* what you are doing because of automation), in order to avoid this 'proceed with acknowledgement' requirement, add `--resync-auth` to automatically acknowledge the prompt. + +### Performing a --force-sync without a --resync or changing your configuration +In some cases and situations, you may have configured the application to skip certain files and folders using 'skip_file' and 'skip_dir' configuration. You then may have a requirement to actually sync one of these items, but do not wish to modify your configuration, nor perform an entire `--resync` twice. + +The `--force-sync` option allows you to sync a specific directory, ignoring your 'skip_file' and 'skip_dir' configuration and negating the requirement to perform a `--resync` + +In order to use this option, you must run the application manually in the following manner: +```text +onedrive --synchronize --single-directory '' --force-sync +``` + +When using `--force-sync`, the following warning and advice will be presented: +```text +WARNING: Overriding application configuration to use application defaults for skip_dir and skip_file due to --synchronize --single-directory --force-sync being used + +The use of --force-sync will reconfigure the application to use defaults. This may have untold and unknown future impacts. +By proceeding in using this option you accept any impacts including any data loss that may occur as a result of using --force-sync. + +Are you sure you wish to proceed with --force-sync [Y/N] +``` + +To proceed with using `--force-sync`, you must type 'y' or 'Y' to allow the application to continue. + +### Increasing logging level +When running a sync it may be desirable to see additional information as to the progress and operation of the client. To do this, use the following command: +```text +onedrive --synchronize --verbose +``` + +### Client Activity Log +When running onedrive all actions can be logged to a separate log file. This can be enabled by using the `--enable-logging` flag. By default, log files will be written to `/var/log/onedrive/` + +**Note:** You will need to ensure the existence of this directory, and that your user has the applicable permissions to write to this directory or the following warning will be printed: +```text +Unable to access /var/log/onedrive/ +Please manually create '/var/log/onedrive/' and set appropriate permissions to allow write access +The requested client activity log will instead be located in the users home directory +``` + +On many systems this can be achieved by +```text +sudo mkdir /var/log/onedrive +sudo chown root:users /var/log/onedrive +sudo chmod 0775 /var/log/onedrive +``` + +All log files will be in the format of `%username%.onedrive.log`, where `%username%` represents the user who ran the client. + +Additionally, you need to ensure that your user account is part of the 'users' group: +``` +cat /etc/group | grep users +``` + +If your user is not part of this group, then you need to add your user to this group: +``` +sudo usermod -a -G users +``` + +You then need to 'logout' of all sessions / SSH sessions to login again to have the new group access applied. + + +**Note:** +To use a different log directory rather than the default above, add the following as a configuration option to `~/.config/onedrive/config`: +```text +log_dir = "/path/to/location/" +``` +Trailing slash required + +An example of the log file is below: +```text +2018-Apr-07 17:09:32.1162837 Loading config ... +2018-Apr-07 17:09:32.1167908 No config file found, using defaults +2018-Apr-07 17:09:32.1170626 Initializing the OneDrive API ... +2018-Apr-07 17:09:32.5359143 Opening the item database ... +2018-Apr-07 17:09:32.5515295 All operations will be performed in: /root/OneDrive +2018-Apr-07 17:09:32.5518387 Initializing the Synchronization Engine ... +2018-Apr-07 17:09:36.6701351 Applying changes of Path ID: +2018-Apr-07 17:09:37.4434282 Adding OneDrive Root to the local database +2018-Apr-07 17:09:37.4478342 The item is already present +2018-Apr-07 17:09:37.4513752 The item is already present +2018-Apr-07 17:09:37.4550062 The item is already present +2018-Apr-07 17:09:37.4586444 The item is already present +2018-Apr-07 17:09:37.7663571 Adding OneDrive Root to the local database +2018-Apr-07 17:09:37.7739451 Fetching details for OneDrive Root +2018-Apr-07 17:09:38.0211861 OneDrive Root exists in the database +2018-Apr-07 17:09:38.0215375 Uploading differences of . +2018-Apr-07 17:09:38.0220464 Processing +2018-Apr-07 17:09:38.0224884 The directory has not changed +2018-Apr-07 17:09:38.0229369 Processing +2018-Apr-07 17:09:38.02338 The directory has not changed +2018-Apr-07 17:09:38.0237678 Processing +2018-Apr-07 17:09:38.0242285 The directory has not changed +2018-Apr-07 17:09:38.0245977 Processing +2018-Apr-07 17:09:38.0250788 The directory has not changed +2018-Apr-07 17:09:38.0254657 Processing +2018-Apr-07 17:09:38.0259923 The directory has not changed +2018-Apr-07 17:09:38.0263547 Uploading new items of . +2018-Apr-07 17:09:38.5708652 Applying changes of Path ID: +``` + +### Notifications +If notification support is compiled in, the following events will trigger a notification within the display manager session: +* Aborting a sync if .nosync file is found +* Cannot create remote directory +* Cannot upload file changes +* Cannot delete remote file / folder +* Cannot move remote file / folder + + +### Handling a OneDrive account password change +If you change your OneDrive account password, the client will no longer be authorised to sync, and will generate the following error: +```text +ERROR: OneDrive returned a 'HTTP 401 Unauthorized' - Cannot Initialize Sync Engine +``` +To re-authorise the client, follow the steps below: +1. If running the client as a service (init.d or systemd), stop the service +2. Run the command `onedrive --reauth`. This will clean up the previous authorisation, and will prompt you to re-authorise the client as per initial configuration. +3. Restart the client if running as a service or perform a manual sync + +The application will now sync with OneDrive with the new credentials. + +## Configuration + +Configuration is determined by three layers: the default values, values set in the configuration file, and values passed in via the command line. The default values provide a reasonable default, and configuration is optional. + +Most command line options have a respective configuration file setting. + +If you want to change the defaults, you can copy and edit the included config file into your configuration directory. Valid default directories for the config file are: +* `~/.config/onedrive` +* `/etc/onedrive` + +**Example:** +```text +mkdir -p ~/.config/onedrive +wget https://raw.githubusercontent.com/abraunegg/onedrive/master/config -O ~/.config/onedrive/config +nano ~/.config/onedrive/config +``` +This file does not get created by default, and should only be created if you want to change the 'default' operational parameters. + +See the [config](https://raw.githubusercontent.com/abraunegg/onedrive/master/config) file for the full list of options, and [All available commands](https://github.com/abraunegg/onedrive/blob/master/docs/usage.md#all-available-commands) for all possible keys and their default values. + +**Note:** The location of the application configuration information can also be specified by using the `--confdir` configuration option which can be passed in at client run-time. + +### The default configuration file is listed below: +```text +# Configuration for OneDrive Linux Client +# This file contains the list of supported configuration fields +# with their default values. +# All values need to be enclosed in quotes +# When changing a config option below, remove the '#' from the start of the line +# For explanations of all config options below see docs/usage.md or the man page. +# +# sync_dir = "~/OneDrive" +# skip_file = "~*|.~*|*.tmp" +# monitor_interval = "300" +# skip_dir = "" +# log_dir = "/var/log/onedrive/" +# drive_id = "" +# upload_only = "false" +# check_nomount = "false" +# check_nosync = "false" +# download_only = "false" +# disable_notifications = "false" +# disable_upload_validation = "false" +# enable_logging = "false" +# force_http_11 = "false" +# local_first = "false" +# no_remote_delete = "false" +# skip_symlinks = "false" +# debug_https = "false" +# skip_dotfiles = "false" +# skip_size = "1000" +# dry_run = "false" +# min_notify_changes = "5" +# monitor_log_frequency = "6" +# monitor_fullscan_frequency = "12" +# sync_root_files = "false" +# classify_as_big_delete = "1000" +# user_agent = "" +# remove_source_files = "false" +# skip_dir_strict_match = "false" +# application_id = "" +# resync = "false" +# resync_auth = "false" +# bypass_data_preservation = "false" +# azure_ad_endpoint = "" +# azure_tenant_id = "common" +# sync_business_shared_folders = "false" +# sync_dir_permissions = "700" +# sync_file_permissions = "600" +# rate_limit = "131072" +# webhook_enabled = "false" +# webhook_public_url = "" +# webhook_listening_host = "" +# webhook_listening_port = "8888" +# webhook_expiration_interval = "86400" +# webhook_renewal_interval = "43200" +# space_reservation = "50" +# display_running_config = "false" +# read_only_auth_scope = "false" +# cleanup_local_files = "false" +# operation_timeout = "3600" +# dns_timeout = "60" +# connect_timeout = "10" +# data_timeout = "600" +# ip_protocol_version = "0" +``` + +### 'config' file configuration examples: +The below are 'config' file examples to assist with configuration of the 'config' file: + +#### sync_dir +Configure your local sync directory location. + +Example: +```text +# When changing a config option below, remove the '#' from the start of the line +# For explanations of all config options below see docs/usage.md or the man page. +# +sync_dir="~/MyDirToSync" +# skip_file = "~*|.~*|*.tmp" +# monitor_interval = "300" +# skip_dir = "" +# log_dir = "/var/log/onedrive/" +``` +**Please Note:** +Proceed with caution here when changing the default sync dir from `~/OneDrive` to `~/MyDirToSync` + +The issue here is around how the client stores the sync_dir path in the database. If the config file is missing, or you don't use the `--syncdir` parameter - what will happen is the client will default back to `~/OneDrive` and 'think' that either all your data has been deleted - thus delete the content on OneDrive, or will start downloading all data from OneDrive into the default location. + +**Note:** After changing `sync_dir`, you must perform a full re-synchronization by adding `--resync` to your existing command line - for example: `onedrive --synchronize --resync` + +**Important Note:** If your `sync_dir` is pointing to a network mount point (a network share via NFS, Windows Network Share, Samba Network Share) these types of network mount points do not support 'inotify', thus tracking real-time changes via inotify of local files is not possible. Local filesystem changes will be replicated between the local filesystem and OneDrive based on the `monitor_interval` value. This is not something (inotify support for NFS, Samba) that this client can fix. + +#### sync_dir directory and file permissions +The following are directory and file default permissions for any new directory or file that is created: +* Directories: 700 - This provides the following permissions: `drwx------` +* Files: 600 - This provides the following permissions: `-rw-------` + +To change the default permissions, update the following 2 configuration options with the required permissions. Utilise the [Unix Permissions Calculator](https://chmod-calculator.com/) to assist in determining the required permissions. + +```text +# When changing a config option below, remove the '#' from the start of the line +# For explanations of all config options below see docs/usage.md or the man page. +# +... +# sync_business_shared_folders = "false" +sync_dir_permissions = "700" +sync_file_permissions = "600" + +``` + +**Important:** Special permission bits (setuid, setgid, sticky bit) are not supported. Valid permission values are from `000` to `777` only. + +#### skip_dir +This option is used to 'skip' certain directories and supports pattern matching. + +Patterns are case insensitive. `*` and `?` [wildcards characters](https://technet.microsoft.com/en-us/library/bb490639.aspx) are supported. Use `|` to separate multiple patterns. + +**Important:** Entries under `skip_dir` are relative to your `sync_dir` path. + +Example: +```text +# When changing a config option below, remove the '#' from the start of the line +# For explanations of all config options below see docs/usage.md or the man page. +# +# sync_dir = "~/OneDrive" +# skip_file = "~*|.~*|*.tmp" +# monitor_interval = "300" +skip_dir = "Desktop|Documents/IISExpress|Documents/SQL Server Management Studio|Documents/Visual Studio*|Documents/WindowsPowerShell" +# log_dir = "/var/log/onedrive/" +``` + +**Note:** The `skip_dir` can be specified multiple times, for example: +```text +skip_dir = "SomeDir|OtherDir|ThisDir|ThatDir" +skip_dir = "/Path/To/A/Directory" +skip_dir = "/Another/Path/To/Different/Directory" +``` +This will be interpreted the same as: +```text +skip_dir = "SomeDir|OtherDir|ThisDir|ThatDir|/Path/To/A/Directory|/Another/Path/To/Different/Directory" +``` + +**Note:** After changing `skip_dir`, you must perform a full re-synchronization by adding `--resync` to your existing command line - for example: `onedrive --synchronize --resync` + +#### skip_file +This option is used to 'skip' certain files and supports pattern matching. + +Patterns are case insensitive. `*` and `?` [wildcards characters](https://technet.microsoft.com/en-us/library/bb490639.aspx) are supported. Use `|` to separate multiple patterns. + +Files can be skipped in the following fashion: +* Specify a wildcard, eg: '*.txt' (skip all txt files) +* Explicitly specify the filename and it's full path relative to your sync_dir, eg: 'path/to/file/filename.ext' +* Explicitly specify the filename only and skip every instance of this filename, eg: 'filename.ext' + +By default, the following files will be skipped: +* Files that start with ~ +* Files that start with .~ (like .~lock.* files generated by LibreOffice) +* Files that end in .tmp + +**Important:** Do not use a skip_file entry of `.*` as this will prevent correct searching of local changes to process. + +Example: +```text +# When changing a config option below, remove the '#' from the start of the line +# For explanations of all config options below see docs/usage.md or the man page. +# +# sync_dir = "~/OneDrive" +skip_file = "~*|Documents/OneNote*|Documents/config.xlaunch|myfile.ext" +# monitor_interval = "300" +# skip_dir = "" +# log_dir = "/var/log/onedrive/" +``` + +**Note:** The `skip_file` can be specified multiple times, for example: +```text +skip_file = "~*|.~*|*.tmp|*.swp" +skip_file = "*.blah" +skip_file = "never_sync.file" +``` +This will be interpreted the same as: +```text +skip_file = "~*|.~*|*.tmp|*.swp|*.blah|never_sync.file" +``` + +**Note:** after changing `skip_file`, you must perform a full re-synchronization by adding `--resync` to your existing command line - for example: `onedrive --synchronize --resync` + +#### skip_dotfiles +Setting this to `"true"` will skip all .files and .folders while syncing. + +Example: +```text +# skip_symlinks = "false" +# debug_https = "false" +skip_dotfiles = "true" +# dry_run = "false" +# monitor_interval = "300" +``` + +#### monitor_interval +The monitor interval is defined as the wait time 'between' sync's when running in monitor mode. When this interval expires, the client will check OneDrive for changes online, performing data integrity checks and scanning the local 'sync_dir' for new content. + +By default without configuration, 'monitor_interval' is set to 300 seconds. Setting this value to 600 will run the sync process every 10 minutes. + +Example: +```text +# skip_dotfiles = "false" +# dry_run = "false" +monitor_interval = "600" +# min_notify_changes = "5" +# monitor_log_frequency = "6" +``` +**Note:** It is strongly advised you do not use a value of less than 300 seconds for 'monitor_interval'. Using a value less than 300 means your application will be constantly needlessly checking OneDrive online for changes. Future versions of the application may enforce the checking of this minimum value. + +#### monitor_fullscan_frequency +This configuration option controls the number of 'monitor_interval' iterations between when a full scan of your data is performed to ensure data integrity and consistency. + +By default without configuration, 'monitor_fullscan_frequency' is set to 12. In this default state, this means that a full scan is performed every 'monitor_interval' x 'monitor_fullscan_frequency' = 3600 seconds. This is only applicable when running in --monitor mode. + +Setting this value to 24 means that the full scan of OneDrive and checking the integrity of the data stored locally will occur every 2 hours (assuming 'monitor_interval' is set to 300 seconds): + +Example: +```text +# min_notify_changes = "5" +# monitor_log_frequency = "6" +monitor_fullscan_frequency = "24" +# sync_root_files = "false" +# classify_as_big_delete = "1000" +``` + +**Note:** When running in --monitor mode, at application start-up, a full scan will be performed to ensure data integrity. This option has zero effect when running the application in `--synchronize` mode and a full scan will always be performed. + +#### monitor_log_frequency +This configuration option controls the output of when logging is performed to detail that a sync is occuring with OneDrive when using `--monitor` mode. The frequency of syncing with OneDrive is controled via 'monitor_interval'. + +By default without configuration, 'monitor_log_frequency' is set to 6. + +By default, at application start-up when using `--monitor` mode, the following will be logged to indicate that the application has correctly started and performed all the initial processing steps: +``` +Configuring Global Azure AD Endpoints +Initializing the Synchronization Engine ... +Initializing monitor ... +OneDrive monitor interval (seconds): 300 +Starting a sync with OneDrive +Syncing changes from OneDrive ... +Performing a database consistency and integrity check on locally stored data ... +Sync with OneDrive is complete +``` +Then, based on 'monitor_log_frequency', the following will be logged when the value is reached: +``` +Starting a sync with OneDrive +Syncing changes from OneDrive ... +Sync with OneDrive is complete +``` +**Note:** The additional log output `Performing a database consistency and integrity check on locally stored data ...` will only be displayed when this activity is occuring which is triggered by 'monitor_fullscan_frequency'. + +#### min_notify_changes +This option defines the minimum number of pending incoming changes necessary to trigger a desktop notification. This allows controlling the frequency of notifications. + +Example: +```text +# dry_run = "false" +# monitor_interval = "300" +min_notify_changes = "50" +# monitor_log_frequency = "6" +# monitor_fullscan_frequency = "12" +``` + +#### operation_timeout +Operation Timeout is the maximum amount of time (seconds) a file operation is allowed to take. This includes DNS resolution, connecting, data transfer, etc. + +Example: +```text +# sync_file_permissions = "600" +# rate_limit = "131072" +operation_timeout = "3600" +``` + +#### ip_protocol_version +By default, the application will use IPv4 and IPv6 to resolve and communicate with Microsoft OneDrive. In some Linux distributions (most notably Ubuntu and those distributions based on Ubuntu) this will cause problems due to how DNS resolution is being performed. + +To configure the application to use a specific IP version, configure the following in your config file: +```text +# operation_timeout = "3600" +# dns_timeout = "60" +# connect_timeout = "10" +# data_timeout = "600" +ip_protocol_version = "1" + +``` +**Note:** +* A value of 0 will mean the client will use IPv4 and IPv6. This is the default. +* A value of 1 will mean the client will use IPv4 only. +* A value of 2 will mean the client will use IPv6 only. + +#### classify_as_big_delete +This configuration option will help prevent the online deletion of files and folders online, when the directory that has been deleted contains more items than the specified value. + +By default, this value is 1000 which will count files and folders as children of the directory that has been deleted. + +To change this value, configure the following in your config file: +```text +# monitor_fullscan_frequency = "12" +# sync_root_files = "false" +classify_as_big_delete = "3000" +# user_agent = "" +# remove_source_files = "false" +``` + +**Note:** +* This option only looks at Directories. It has zero effect on deleting files located in your 'sync_dir' root +* This option (in v2.4.x and below) only gets activated when using `--monitor`. In `--synchronize` mode it is ignored as it is assumed you performed that desired operation before you started your next manual sync with OneDrive. +* Be sensible with setting this value - do not use a low value such as '1' as this will prevent you from syncing your data each and every time you delete a single file. + + +#### Configuring the client for 'single tenant application' use +In some instances when using OneDrive Business Accounts, depending on the Azure organisational configuration, it will be necessary to configure the client as a 'single tenant application'. +To configure this, after creating the application on your Azure tenant, update the 'config' file with the tenant name (not the GUID) and the newly created Application ID, then this will be used for the authentication process. +```text +# skip_dir_strict_match = "false" +application_id = "your.application.id.guid" +# resync = "false" +# bypass_data_preservation = "false" +# azure_ad_endpoint = "xxxxxx" +azure_tenant_id = "your.azure.tenant.name" +# sync_business_shared_folders = "false" +``` + +#### Configuring the client to use older 'skilion' application identifier +In some instances it may be desirable to utilise the older 'skilion' application identifier to avoid authorising a new application ID within Microsoft Azure environments. +To configure this, update the 'config' file with the old Application ID, then this will be used for the authentication process. +```text +# skip_dir_strict_match = "false" +application_id = "22c49a0d-d21c-4792-aed1-8f163c982546" +# resync = "false" +# bypass_data_preservation = "false" +``` +**Note:** The application will now use the older 'skilion' client identifier, however this may increase your chances of getting a OneDrive 429 error. + +**Note:** After changing the 'application_id' you will need to restart any 'onedrive' process you have running, and potentially issue a `--reauth` to re-authenticate the client with this updated application ID. + +## Frequently Asked Configuration Questions + +### How to sync only specific or single directory? +There are two methods to achieve this: +* Utilise '--single-directory' option to only sync this specific path +* Utilise 'sync_list' to configure what files and directories to sync, and what should be exluded + +### How to 'skip' directories from syncing? +There are several mechanisms available to 'skip' a directory from the sync process: +* Utilise 'skip_dir' to configure what directories to skip. Refer to above for configuration advice. +* Utilise 'sync_list' to configure what files and directories to sync, and what should be exluded + +One further method is to add a '.nosync' empty file to any folder. When this file is present, adding `--check-for-nosync` to your command line will now make the sync process skip any folder where the '.nosync' file is present. + +To make this a permanent change to always skip folders when a '.nosync' empty file is present, add the following to your config file: + +Example: +```text +# upload_only = "false" +# check_nomount = "false" +check_nosync = "true" +# download_only = "false" +# disable_notifications = "false" +``` +**Default:** False + +### How to 'skip' files from syncing? +There are two methods to achieve this: +* Utilise 'skip_file' to configure what files to skip. Refer to above for configuration advice. +* Utilise 'sync_list' to configure what files and directories to sync, and what should be exluded + +### How to 'skip' dot files and folders from syncing? +There are three methods to achieve this: +* Utilise 'skip_file' or 'skip_dir' to configure what files or folders to skip. Refer to above for configuration advice. +* Utilise 'sync_list' to configure what files and directories to sync, and what should be exluded +* Utilise 'skip_dotfiles' to skip any dot file (for example: `.Trash-1000` or `.xdg-volume-info`) from syncing to OneDrive. + +Example: +```text +# skip_symlinks = "false" +# debug_https = "false" +skip_dotfiles = "true" +# skip_size = "1000" +# dry_run = "false" +``` +**Default:** False + +### How to 'skip' files larger than a certain size from syncing? +There are two methods to achieve this: +* Use `--skip-size ARG` as part of a CLI command to skip new files larger than this size (in MB) +* Use `skip_size = "value"` as part of your 'config' file where files larger than this size (in MB) will be skipped + +### How to 'rate limit' the application to control bandwidth consumed for upload & download operations? +To minimise the Internet bandwidth for upload and download operations, you can configure the 'rate_limit' option within the config file. + +Example valid values for this are as follows: +* 131072 = 128 KB/s - minimum for basic application operations to prevent timeouts +* 262144 = 256 KB/s +* 524288 = 512 KB/s +* 1048576 = 1 MB/s +* 10485760 = 10 MB/s +* 104857600 = 100 MB/s + +Example: +```text +# sync_business_shared_folders = "false" +# sync_dir_permissions = "700" +# sync_file_permissions = "600" +rate_limit = "131072" +``` + +**Note:** A number greater than '131072' is a valid value, with '104857600' being tested as an upper limit. + +### How to prevent your local disk from filling up? +By default, the application will reserve 50MB of disk space to prevent your filesystem to run out of disk space. This value can be modified by adding the following to your config file: + +Example: +```text +... +# webhook_expiration_interval = "86400" +# webhook_renewal_interval = "43200" +space_reservation = "10" +``` + +The value entered is in MB (Mega Bytes). In this example, a value of 10MB is being used, and will be converted to bytes by the application. The value being used can be reviewed when using `--display-config`: +``` +Config option 'sync_dir_permissions' = 700 +Config option 'sync_file_permissions' = 600 +Config option 'space_reservation' = 10485760 +Config option 'application_id' = +Config option 'azure_ad_endpoint' = +Config option 'azure_tenant_id' = common +``` + +Any value is valid here, however, if you use a value of '0' a value of '1' will actually be used, so that you actually do not run out of disk space. + +### How are symbolic links handled by the client? +Microsoft OneDrive has zero concept or understanding of symbolic links, and attempting to upload a symbolic link to Microsoft OneDrive generates a platform API error. All data (files and folders) that are uploaded to OneDrive must be whole files or actual directories. + +As such, there are only two methods to support symbolic links with this client: +1. Follow the Linux symbolic link and upload what ever the link is pointing at to OneDrive. This is the default behaviour. +2. Skip symbolic links by configuring the application to do so. In skipping, no data, no link, no reference is uploaded to OneDrive. + +To skip symbolic links, edit your configuration as per below: + +```text +# local_first = "false" +# no_remote_delete = "false" +skip_symlinks = "true" +# debug_https = "false" +# skip_dotfiles = "false" +``` +Setting this to `"true"` will configure the client to skip all symbolic links while syncing. + +The default setting is `"false"` which will sync the whole folder structure referenced by the symbolic link, duplicating the contents on OneDrive in the place where the symbolic link is. + +### How to sync shared folders (OneDrive Personal)? +Folders shared with you can be synced by adding them to your OneDrive. To do that open your Onedrive, go to the Shared files list, right click on the folder you want to sync and then click on "Add to my OneDrive". + +### How to sync shared folders (OneDrive Business or Office 365)? +Refer to [./business-shared-folders.md](business-shared-folders.md) for configuration assistance. + +Do not use the 'Add shortcut to My files' from the OneDrive web based interface to add a 'shortcut' to your shared folder. This shortcut is not supported by the OneDrive API, thus it cannot be used. + +### How to sync sharePoint / Office 365 Shared Libraries? +Refer to [./sharepoint-libraries.md](sharepoint-libraries.md) for configuration assistance. + +### How to run a user systemd service at boot without user login? +In some cases it may be desirable for the systemd service to start without having to login as your 'user' + +To avoid this issue, you need to reconfigure your 'user' account so that the systemd services you have created will startup without you having to login to your system: +```text +loginctl enable-linger +``` + +### How to create a shareable link? +In some cases it may be desirable to create a shareable file link and give this link to other users to access a specific file. + +To do this, use the following command: +```text +onedrive --create-share-link +``` +**Note:** By default this will be a read-only link. + +To make this a read-write link, use the following command: +```text +onedrive --create-share-link --with-editing-perms +``` +**Note:** The ordering of the option file path and option flag is important. + +### How to sync both Personal and Business accounts at the same time? +You must configure separate instances of the application configuration for each account. + +Refer to [./advanced-usage.md](advanced-usage.md) for configuration assistance. + +### How to sync multiple SharePoint Libraries at the same time? +You must configure a separate instances of the application configuration for each SharePoint Library. + +Refer to [./advanced-usage.md](advanced-usage.md) for configuration assistance. + +## Running 'onedrive' in 'monitor' mode +Monitor mode (`--monitor`) allows the onedrive process to continually monitor your local file system for changes to files. + +Two common errors can occur when using monitor mode: +* Intialisation failure +* Unable to add a new inotify watch + +Both of these errors are local environment issues, where the following system variables need to be increased as the current system values are potentially too low: +* `fs.file-max` +* `fs.inotify.max_user_watches` + +To determine what the existing values are on your system use the following commands: +```text +sysctl fs.file-max +sysctl fs.inotify.max_user_watches +``` + +To determine what value to change to, you need to count all the files and folders in your configured 'sync_dir': +```text +cd /path/to/your/sync/dir +ls -laR | wc -l +``` + +To make a change to these variables using your file and folder count: +``` +sudo sysctl fs.file-max= +sudo sysctl fs.inotify.max_user_watches= +``` + +To make these changes permanent, refer to your OS reference documentation. + +### Use webhook to subscribe to remote updates in 'monitor' mode + +A webhook can be optionally enabled in the monitor mode to allow the onedrive process to subscribe to remote updates. Remote changes can be synced to your local file system as soon as possible, without waiting for the next sync cycle. + +To enable this feature, you need to configure the following options in the config file: + +```text +webhook_enabled = "true" +webhook_public_url = "" +``` + +Setting `webhook_enabled` to `true` enables the webhook in 'monitor' mode. The onedrive process will listen for incoming updates at a configurable endpoint, which defaults to `0.0.0.0:8888`. The `webhook_public_url` must be set to an public-facing url for Microsoft to send updates to your webhook. If your host is directly exposed to the Internet, the `webhook_public_url` can be set to `http://:8888/` to match the default endpoint. However, the recommended approach is to configure a reverse proxy like nginx. + +**Note:** A valid HTTPS certificate is required for your public-facing URL if using nginx. + +For example, below is a nginx config snippet to proxy traffic into the webhook: + +```text +server { + listen 80; + location /webhooks/onedrive { + proxy_http_version 1.1; + proxy_pass http://127.0.0.1:8888; + } +} +``` + +With nginx running, you can configure `webhook_public_url` to `https:///webhooks/onedrive`. + +If you receive this application error: +```text +Subscription validation request failed. Response must exactly match validationToken query parameter. +``` +The most likely cause for this error will be your nginx configuration. To resolve, potentially investigate the following configuration for nginx: + +```text +server { + listen 80; + location /webhooks/onedrive { + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Original-Request-URI $request_uri; + proxy_read_timeout 300s; + proxy_connect_timeout 75s; + proxy_buffering off; + proxy_http_version 1.1; + proxy_pass http://127.0.0.1:8888; + } +} +``` + +For any further nginx configuration assistance, please refer to: https://docs.nginx.com/ + +### More webhook configuration options + +Below options can be optionally configured. The default is usually good enough. + +#### webhook_listening_host and webhook_listening_port + +Set `webhook_listening_host` and `webhook_listening_port` to change the webhook listening endpoint. If `webhook_listening_host` is left empty, which is the default, the webhook will bind to `0.0.0.0`. The default `webhook_listening_port` is `8888`. + +``` +webhook_listening_host = "" +webhook_listening_port = "8888" +``` + +#### webhook_expiration_interval and webhook_renewal_interval + +Set `webhook_expiration_interval` and `webhook_renewal_interval` to change the frequency of subscription renewal. By default, the webhook asks Microsoft to keep subscriptions alive for 24 hours, and it renews subscriptions when it is less than 12 hours before their expiration. + +``` +# Default expiration interval is 24 hours +webhook_expiration_interval = "86400" + +# Default renewal interval is 12 hours +webhook_renewal_interval = "43200" +``` + +## Running 'onedrive' as a system service +There are a few ways to use onedrive as a service +* via init.d +* via systemd +* via runit + +**Note:** If using the service files, you may need to increase the `fs.inotify.max_user_watches` value on your system to handle the number of files in the directory you are monitoring as the initial value may be too low. + +### OneDrive service running as root user via init.d +```text +chkconfig onedrive on +service onedrive start +``` +To see the logs run: +```text +tail -f /var/log/onedrive/.onedrive.log +``` +To change what 'user' the client runs under (by default root), manually edit the init.d service file and modify `daemon --user root onedrive_service.sh` for the correct user. + +### OneDrive service running as root user via systemd (Arch, Ubuntu, Debian, OpenSuSE, Fedora) +First, su to root using `su - root`, then enable the systemd service: +```text +systemctl --user enable onedrive +systemctl --user start onedrive +``` +**Note:** `systemctl --user` directive is not applicable for Red Hat Enterprise Linux (RHEL) or CentOS Linux platforms - see below. + +**Note:** This will run the 'onedrive' process with a UID/GID of '0', thus, any files or folders that are created will be owned by 'root' + +To view the status of the service running, use the following: +```text +systemctl --user status onedrive.service +``` + +To see the systemd application logs run: +```text +journalctl --user-unit=onedrive -f +``` + +**Note:** It is a 'systemd' requirement that the XDG environment variables exist for correct enablement and operation of systemd services. If you receive this error when enabling the systemd service: +``` +Failed to connect to bus: No such file or directory +``` +The most likely cause is that the XDG environment variables are missing. To fix this, you must add the following to `.bashrc` or any other file which is run on user login: +``` +export XDG_RUNTIME_DIR="/run/user/$UID" +export DBUS_SESSION_BUS_ADDRESS="unix:path=${XDG_RUNTIME_DIR}/bus" +``` + +To make this change effective, you must logout of all user accounts where this change has been made. + +**Note:** On some systems (for example - Raspbian / Ubuntu / Debian on Raspberry Pi) the above XDG fix may not be reliable after system reboots. The potential alternative to start the client via systemd as root, is to perform the following: +1. Create a symbolic link from `/home/root/.config/onedrive` pointing to `/root/.config/onedrive/` +2. Create a systemd service using the '@' service file: `systemctl enable onedrive@root.service` +3. Start the root@service: `systemctl start onedrive@root.service` + +This will ensure that the service will correctly restart on system reboot. + +To see the systemd application logs run: +```text +journalctl --unit=onedrive@ -f +``` + +### OneDrive service running as root user via systemd (Red Hat Enterprise Linux, CentOS Linux) +```text +systemctl enable onedrive +systemctl start onedrive +``` +**Note:** This will run the 'onedrive' process with a UID/GID of '0', thus, any files or folders that are created will be owned by 'root' + +To see the systemd application logs run: +```text +journalctl --unit=onedrive -f +``` + +### OneDrive service running as a non-root user via systemd (All Linux Distributions) +In some cases it is desirable to run the OneDrive client as a service, but not running as the 'root' user. In this case, follow the directions below to configure the service for your normal user login. + +1. As the user, who will be running the service, run the application in standalone mode, authorize the application for use & validate that the synchronization is working as expected: +```text +onedrive --synchronize --verbose +``` +2. Once the application is validated and working for your user, as the 'root' user, where is your username from step 1 above. +```text +systemctl enable onedrive@.service +systemctl start onedrive@.service +``` +3. To view the status of the service running for the user, use the following: +```text +systemctl status onedrive@.service +``` + +To see the systemd application logs run: +```text +journalctl --unit=onedrive@ -f +``` + +### OneDrive service running as a non-root user via systemd (with notifications enabled) (Arch, Ubuntu, Debian, OpenSuSE, Fedora) +In some cases you may wish to receive GUI notifications when using the client when logged in as a non-root user. In this case, follow the directions below: + +1. Login via graphical UI as user you wish to enable the service for +2. Disable any `onedrive@` service files for your username - eg: +```text +sudo systemctl stop onedrive@alex.service +sudo systemctl disable onedrive@alex.service +``` +3. Enable service as per the following: +```text +systemctl --user enable onedrive +systemctl --user start onedrive +``` + +To view the status of the service running for the user, use the following: +```text +systemctl --user status onedrive.service +``` + +To see the systemd application logs run: +```text +journalctl --user-unit=onedrive -f +``` + +**Note:** `systemctl --user` directive is not applicable for Red Hat Enterprise Linux (RHEL) or CentOS Linux platforms + +### OneDrive service running as a non-root user via runit (antiX, Devuan, Artix, Void) + +1. Create the following folder if not present already `/etc/sv/runsvdir-` + + - where `` is the `USER` targeted for the service + - _e.g_ `# mkdir /etc/sv/runsvdir-nolan` + +2. Create a file called `run` under the previously created folder with + executable permissions + + - `# touch /etc/sv/runsvdir-/run` + - `# chmod 0755 /etc/sv/runsvdir-/run` + +3. Edit the `run` file with the following contents (priviledges needed) + + ```sh + #!/bin/sh + export USER="" + export HOME="/home/" + + groups="$(id -Gn "${USER}" | tr ' ' ':')" + svdir="${HOME}/service" + + exec chpst -u "${USER}:${groups}" runsvdir "${svdir}" + ``` + + - do not forget to correct the `` according to the `USER` set on + step #1 + +4. Enable the previously created folder as a service + + - `# ln -fs /etc/sv/runsvdir- /var/service/` + +5. Create a subfolder on the `USER`'s `HOME` directory to store the services + (or symlinks) + + - `$ mkdir ~/service` + +6. Create a subfolder for OneDrive specifically + + - `$ mkdir ~/service/onedrive/` + +7. Create a file called `run` under the previously created folder with + executable permissions + + - `$ touch ~/service/onedrive/run` + - `$ chmod 0755 ~/service/onedrive/run` + +8. Append the following contents to the `run` file + + ```sh + #!/usr/bin/env sh + exec /usr/bin/onedrive --monitor + ``` + + - in some scenario the path for the `onedrive` binary might differ, you can + obtain it regardless by running `$ command -v onedrive` + +9. Reboot to apply changes + +10. Check status of user-defined services + + - `$ sv status ~/service/*` + +You may refer to Void's documentation regarding +[Per-User Services](https://docs.voidlinux.org/config/services/user-services.html) +for extra details. + +## Additional Configuration +### Advanced Configuration of the OneDrive Free Client +* Configuring the client to use mulitple OneDrive accounts / configurations, for example: + * Setup to use onedrive with both Personal and Business accounts + * Setup to use onedrive with multiple SharePoint Libraries +* Configuring the client for use in dual-boot (Windows / Linux) situations +* Configuring the client for use when 'sync_dir' is a mounted directory +* Upload data from the local ~/OneDrive folder to a specific location on OneDrive + +Refer to [./advanced-usage.md](advanced-usage.md) for configuration assistance. + +### Access OneDrive service through a proxy +If you have a requirement to run the client through a proxy, there are a couple of ways to achieve this: +1. Set proxy configuration in `~/.bashrc` to allow the authorization process and when utilizing `--synchronize` +2. If running as a systemd service, edit the applicable systemd service file to include the proxy configuration information: +```text +[Unit] +Description=OneDrive Free Client +Documentation=https://github.com/abraunegg/onedrive +After=network-online.target +Wants=network-online.target + +[Service] +Environment="HTTP_PROXY=http://ip.address:port" +Environment="HTTPS_PROXY=http://ip.address:port" +ExecStart=/usr/local/bin/onedrive --monitor +Restart=on-failure +RestartSec=3 + +[Install] +WantedBy=default.target +``` + +**Note:** After modifying the service files, you will need to run `sudo systemctl daemon-reload` to ensure the service file changes are picked up. A restart of the OneDrive service will also be required to pick up the change to send the traffic via the proxy server + +### Setup selinux for a sync folder outside of the home folder +If selinux is enforced and the sync folder is outside of the home folder, as long as there is no policy for cloud fileservice providers, label the file system folder to user_home_t. +```text +sudo semanage fcontext -a -t user_home_t /path/to/onedriveSyncFolder +sudo restorecon -R -v /path/to/onedriveSyncFolder +``` +To remove this change from selinux and restore the default behaivor: +```text +sudo semanage fcontext -d /path/to/onedriveSyncFolder +sudo restorecon -R -v /path/to/onedriveSyncFolder +``` + +## All available commands +Output of `onedrive --help` +```text +OneDrive - a client for OneDrive Cloud Services + +Usage: + onedrive [options] --synchronize + Do a one time synchronization + onedrive [options] --monitor + Monitor filesystem and sync regularly + onedrive [options] --display-config + Display the currently used configuration + onedrive [options] --display-sync-status + Query OneDrive service and report on pending changes + onedrive -h | --help + Show this help screen + onedrive --version + Show version + +Options: + + --auth-files ARG + Perform authorization via two files passed in as ARG in the format `authUrl:responseUrl` + The authorization URL is written to the `authUrl`, then onedrive waits for the file `responseUrl` + to be present, and reads the response from that file. + --auth-response ARG + Perform authentication not via interactive dialog but via providing the response url directly. + --check-for-nomount + Check for the presence of .nosync in the syncdir root. If found, do not perform sync. + --check-for-nosync + Check for the presence of .nosync in each directory. If found, skip directory from sync. + --classify-as-big-delete + Number of children in a path that is locally removed which will be classified as a 'big data delete' + --cleanup-local-files + Cleanup additional local files when using --download-only. This will remove local data. + --confdir ARG + Set the directory used to store the configuration files + --create-directory ARG + Create a directory on OneDrive - no sync will be performed. + --create-share-link ARG + Create a shareable link for an existing file on OneDrive + --debug-https + Debug OneDrive HTTPS communication. + --destination-directory ARG + Destination directory for renamed or move on OneDrive - no sync will be performed. + --disable-download-validation + Disable download validation when downloading from OneDrive + --disable-notifications + Do not use desktop notifications in monitor mode. + --disable-upload-validation + Disable upload validation when uploading to OneDrive + --display-config + Display what options the client will use as currently configured - no sync will be performed. + --display-running-config + Display what options the client has been configured to use on application startup. + --display-sync-status + Display the sync status of the client - no sync will be performed. + --download-only + Replicate the OneDrive online state locally, by only downloading changes from OneDrive. Do not upload local changes to OneDrive. + --dry-run + Perform a trial sync with no changes made + --enable-logging + Enable client activity to a separate log file + --force + Force the deletion of data when a 'big delete' is detected + --force-http-11 + Force the use of HTTP 1.1 for all operations + --force-sync + Force a synchronization of a specific folder, only when using --single-directory and ignoring all non-default skip_dir and skip_file rules + --get-O365-drive-id ARG + Query and return the Office 365 Drive ID for a given Office 365 SharePoint Shared Library + --get-file-link ARG + Display the file link of a synced file + --help -h + This help information. + --list-shared-folders + List OneDrive Business Shared Folders + --local-first + Synchronize from the local directory source first, before downloading changes from OneDrive. + --log-dir ARG + Directory where logging output is saved to, needs to end with a slash. + --logout + Logout the current user + --min-notify-changes ARG + Minimum number of pending incoming changes necessary to trigger a desktop notification + --modified-by ARG + Display the last modified by details of a given path + --monitor -m + Keep monitoring for local and remote changes + --monitor-fullscan-frequency ARG + Number of sync runs before performing a full local scan of the synced directory + --monitor-interval ARG + Number of seconds by which each sync operation is undertaken when idle under monitor mode. + --monitor-log-frequency ARG + Frequency of logging in monitor mode + --no-remote-delete + Do not delete local file 'deletes' from OneDrive when using --upload-only + --operation-timeout ARG + Maximum amount of time (in seconds) an operation is allowed to take + --print-token + Print the access token, useful for debugging + --reauth + Reauthenticate the client with OneDrive + --remove-directory ARG + Remove a directory on OneDrive - no sync will be performed. + --remove-source-files + Remove source file after successful transfer to OneDrive when using --upload-only + --resync + Forget the last saved state, perform a full sync + --resync-auth + Approve the use of performing a --resync action + --single-directory ARG + Specify a single local directory within the OneDrive root to sync. + --skip-dir ARG + Skip any directories that match this pattern from syncing + --skip-dir-strict-match + When matching skip_dir directories, only match explicit matches + --skip-dot-files + Skip dot files and folders from syncing + --skip-file ARG + Skip any files that match this pattern from syncing + --skip-size ARG + Skip new files larger than this size (in MB) + --skip-symlinks + Skip syncing of symlinks + --source-directory ARG + Source directory to rename or move on OneDrive - no sync will be performed. + --space-reservation ARG + The amount of disk space to reserve (in MB) to avoid 100% disk space utilisation + --sync-root-files + Sync all files in sync_dir root when using sync_list. + --sync-shared-folders + Sync OneDrive Business Shared Folders + --syncdir ARG + Specify the local directory used for synchronization to OneDrive + --synchronize + Perform a synchronization + --upload-only + Replicate the locally configured sync_dir state to OneDrive, by only uploading local changes to OneDrive. Do not download changes from OneDrive. + --user-agent ARG + Specify a User Agent string to the http client + --verbose -v+ + Print more details, useful for debugging (repeat for extra debugging) + --version + Print the version and exit + --with-editing-perms + Create a read-write shareable link for an existing file on OneDrive when used with --create-share-link +``` diff --git a/readme.md b/readme.md new file mode 100644 index 00000000..6c3e2fcf --- /dev/null +++ b/readme.md @@ -0,0 +1,81 @@ +# OneDrive Client for Linux +[![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) +[![Release Date](https://img.shields.io/github/release-date/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) +[![Test Build](https://github.com/abraunegg/onedrive/actions/workflows/testbuild.yaml/badge.svg)](https://github.com/abraunegg/onedrive/actions/workflows/testbuild.yaml) +[![Build Docker Images](https://github.com/abraunegg/onedrive/actions/workflows/docker.yaml/badge.svg)](https://github.com/abraunegg/onedrive/actions/workflows/docker.yaml) +[![Docker Pulls](https://img.shields.io/docker/pulls/driveone/onedrive)](https://hub.docker.com/r/driveone/onedrive) + +A free Microsoft OneDrive Client which supports OneDrive Personal, OneDrive for Business, OneDrive for Office365 and SharePoint. + +This powerful and highly configurable client can run on all major Linux distributions, FreeBSD, or as a Docker container. It supports one-way and two-way sync capabilities and securely connects to Microsoft OneDrive services. + +This client is a 'fork' of the [skilion](https://github.com/skilion/onedrive) client, which the developer has confirmed he has no desire to maintain or support the client ([reference](https://github.com/skilion/onedrive/issues/518#issuecomment-717604726)). This fork has been in active development since mid 2018. + +## Features +* State caching +* Real-Time local file monitoring with inotify +* Real-Time syncing of remote updates via webhooks +* File upload / download validation to ensure data integrity +* Resumable uploads +* Support OneDrive for Business (part of Office 365) +* Shared Folder support for OneDrive Personal and OneDrive Business accounts +* SharePoint / Office365 Shared Libraries +* Desktop notifications via libnotify +* Dry-run capability to test configuration changes +* Prevent major OneDrive accidental data deletion after configuration change +* Support for National cloud deployments (Microsoft Cloud for US Government, Microsoft Cloud Germany, Azure and Office 365 operated by 21Vianet in China) +* Supports single & multi-tenanted applications +* Supports rate limiting of traffic + +## What's missing +* Ability to encrypt/decrypt files on-the-fly when uploading/downloading files from OneDrive +* Support for Windows 'On-Demand' functionality so file is only downloaded when accessed locally + +## External Enhancements +* A GUI for configuration management: [OneDrive Client for Linux GUI](https://github.com/bpozdena/OneDriveGUI) +* Colorful log output terminal modification: [OneDrive Client for Linux Colorful log Output](https://github.com/zzzdeb/dotfiles/blob/master/scripts/tools/onedrive_log) +* System Tray Icon: [OneDrive Client for Linux System Tray Icon](https://github.com/DanielBorgesOliveira/onedrive_tray) + +## Frequently Asked Questions +Refer to [Frequently Asked Questions](https://github.com/abraunegg/onedrive/wiki/Frequently-Asked-Questions) + +## Have a question +If you have a question or need something clarified, please raise a new disscussion post [here](https://github.com/abraunegg/onedrive/discussions) + +## Reporting an Issue or Bug +If you encounter any bugs you can report them here on Github. Before filing an issue be sure to: + +1. Check the version of the application you are using `onedrive --version` and ensure that you are running either the latest [release](https://github.com/abraunegg/onedrive/releases) or built from master. +2. Fill in a new bug report using the [issue template](https://github.com/abraunegg/onedrive/issues/new?template=bug_report.md) +3. Generate a debug log for support using the following [process](https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support) + * If you are in *any* way concerned regarding the sensitivity of the data contained with in the verbose debug log file, create a new OneDrive account, configure the client to use that, use *dummy* data to simulate your environment and then replicate your original issue + * If you are still concerned, provide an NDA or confidentiality document to sign +4. Upload the debug log to [pastebin](https://pastebin.com/) or archive and email to support@mynas.com.au + * If you are concerned regarding the sensitivity of your debug data, encrypt + password protect the archive file and provide the decryption password via an out-of-band (OOB) mechanism. Email support@mynas.com.au for an OOB method for the password to be sent. + * If you are still concerned, provide an NDA or confidentiality document to sign + +## Known issues +Refer to [docs/known-issues.md](https://github.com/abraunegg/onedrive/blob/master/docs/known-issues.md) + +## Documentation and Configuration Assistance +### Installing from Distribution Packages or Building the OneDrive Client for Linux from source +Refer to [docs/install.md](https://github.com/abraunegg/onedrive/blob/master/docs/install.md) + +### Configuration and Usage +Refer to [docs/usage.md](https://github.com/abraunegg/onedrive/blob/master/docs/usage.md) + +### Configure OneDrive Business Shared Folders +Refer to [docs/business-shared-folders.md](https://github.com/abraunegg/onedrive/blob/master/docs/business-shared-folders.md) + +### Configure SharePoint / Office 365 Shared Libraries (Business or Education) +Refer to [docs/sharepoint-libraries.md](https://github.com/abraunegg/onedrive/blob/master/docs/sharepoint-libraries.md) + +### Configure National Cloud support +Refer to [docs/national-cloud-deployments.md](https://github.com/abraunegg/onedrive/blob/master/docs/national-cloud-deployments.md) + +### Docker support +Refer to [docs/docker.md](https://github.com/abraunegg/onedrive/blob/master/docs/docker.md) + +### Podman support +Refer to [docs/podman.md](https://github.com/abraunegg/onedrive/blob/master/docs/podman.md) +