Implement Feature Request #1300 - Support the syncing of individual business shared files (#2646)

* Implement Feature Request #1300 - Support the syncing of individual business shared files
This commit is contained in:
abraunegg 2024-03-09 07:15:11 +11:00 committed by GitHub
parent e895a1174c
commit 75c3d0d939
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 1256 additions and 482 deletions

View file

@ -1,40 +0,0 @@
# How to configure OneDrive Business Shared Folder Sync
## Application Version
Before reading this document, please ensure you are running application version [![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) or greater. Use `onedrive --version` to determine what application version you are using and upgrade your client if required.
## Important Note
This feature has been 100% re-written from v2.5.0 onwards. A pre-requesite before using this capability in v2.5.0 and above is for you to revert any Shared Business Folder configuration you may be currently using, including, but not limited to:
* Removing `sync_business_shared_folders = "true|false"` from your 'config' file
* Removing the 'business_shared_folders' file
* Removing any local data | shared folder data from your configured 'sync_dir' to ensure that there are no conflicts or issues.
## Process Overview
Syncing OneDrive Business Shared Folders requires additional configuration for your 'onedrive' client:
1. From the OneDrive web interface, review the 'Shared' objects that have been shared with you.
2. Select the applicable folder, and click the 'Add shortcut to My files', which will then add this to your 'My files' folder
3. Update your OneDrive Client for Linux 'config' file to enable the feature by adding `sync_business_shared_items = "true"`. Adding this option will trigger a `--resync` requirement.
4. Test the configuration using '--dry-run'
5. Remove the use of '--dry-run' and sync the OneDrive Business Shared folders as required
**NOTE:** This documentation will be updated as this feature progresses.
### Enable syncing of OneDrive Business Shared Folders via config file
```text
sync_business_shared_items = "true"
```
### Disable syncing of OneDrive Business Shared Folders via config file
```text
sync_business_shared_items = "false"
```
## Known Issues
Shared folders, shared with you from people outside of your 'organisation' are unable to be synced. This is due to the Microsoft Graph API not presenting these folders.
Shared folders that match this scenario, when you view 'Shared' via OneDrive online, will have a 'world' symbol as per below:
![shared_with_me](./images/shared_with_me.JPG)
This issue is being tracked by: [#966](https://github.com/abraunegg/onedrive/issues/966)

View file

@ -0,0 +1,251 @@
# How to sync OneDrive Business Shared Items
## Application Version
Before reading this document, please ensure you are running application version [![Version](https://img.shields.io/github/v/release/abraunegg/onedrive)](https://github.com/abraunegg/onedrive/releases) or greater. Use `onedrive --version` to determine what application version you are using and upgrade your client if required.
## Important Note
This feature has been 100% re-written from v2.5.0 onwards. A pre-requesite before using this capability in v2.5.0 and above is for you to revert any Shared Business Folder configuration you may be currently using, including, but not limited to:
* Removing `sync_business_shared_folders = "true|false"` from your 'config' file
* Removing the 'business_shared_folders' file
* Removing any local data | shared folder data from your configured 'sync_dir' to ensure that there are no conflicts or issues.
* Removing any configuration online that might be related to using this feature prior to v2.5.0
## Process Overview
Syncing OneDrive Business Shared Folders requires additional configuration for your 'onedrive' client:
1. From the OneDrive web interface, review the 'Shared' objects that have been shared with you.
2. Select the applicable folder, and click the 'Add shortcut to My files', which will then add this to your 'My files' folder
3. Update your OneDrive Client for Linux 'config' file to enable the feature by adding `sync_business_shared_items = "true"`. Adding this option will trigger a `--resync` requirement.
4. Test the configuration using '--dry-run'
5. Remove the use of '--dry-run' and sync the OneDrive Business Shared folders as required
**NOTE:** This documentation will be updated as this feature progresses.
### Enable syncing of OneDrive Business Shared Items via config file
```text
sync_business_shared_items = "true"
```
### Disable syncing of OneDrive Business Shared Items via config file
```text
sync_business_shared_items = "false"
```
## Syncing OneDrive Business Shared Folders
Use the following steps to add a OneDrive Business Shared Folder to your account:
1. Login to Microsoft OneDrive online, and navigate to 'Shared' from the left hand side pane
![objects_shared_with_me](./images/objects_shared_with_me.png)
2. Select the respective folder you wish to sync, and click the 'Add shortcut to My files' at the top of the page
![add_shared_folder](./images/add_shared_folder.png)
3. The final result online will look like this:
![shared_folder_added](./images/shared_folder_added.png)
When using Microsoft Windows, this shared folder will appear as the following:
![windows_view_shared_folders](./images/windows_view_shared_folders.png)
4. Sync your data using `onedrive --sync --verbose`. If you have just enabled the `sync_business_shared_items = "true"` configuration option, you will be required to perform a resync. During the sync, the selected shared folder will be downloaded:
```
...
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 4
Finished processing /delta JSON response from the OneDrive API
Processing 3 applicable changes and items received from Microsoft OneDrive
Processing OneDrive JSON item batch [1/1] to ensure consistent local state
Creating local directory: ./my_shared_folder
Quota information is restricted or not available for this drive.
Syncing this OneDrive Business Shared Folder: my_shared_folder
Fetching /delta response from the OneDrive API for Drive ID: b!BhWyqa7K_kqXqHtSIlsqjR5iJogxpWxDradnpVGTU2VxBOJh82Y6S4he4rdnGPBT
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 6
Finished processing /delta JSON response from the OneDrive API
Processing 6 applicable changes and items received from Microsoft OneDrive
Processing OneDrive JSON item batch [1/1] to ensure consistent local state
Creating local directory: ./my_shared_folder/asdf
Creating local directory: ./my_shared_folder/original_data
Number of items to download from OneDrive: 3
Downloading file: my_shared_folder/asdf/asdfasdfhashdkfasdf.txt ... done
Downloading file: my_shared_folder/asdf/asdfasdf.txt ... done
Downloading file: my_shared_folder/original_data/file1.data ... done
Performing a database consistency and integrity check on locally stored data
...
```
When this is viewed locally, on Linux, this shared folder is seen as the following:
![linux_shared_folder_view](./images/linux_shared_folder_view.png)
Any shared folder you add can utilise any 'client side filtering' rules that you have created.
## Syncing OneDrive Business Shared Files
There are two methods to support the syncing OneDrive Business Shared Files with the OneDrive Application
1. Add a 'shortcut' to your 'My Files' for the file, which creates a URL shortcut to the file which can be followed when using a Linux Window Manager (Gnome, KDE etc) and the link will open up in a browser. Microsoft Windows only supports this option.
2. Use `--sync-shared-files` option to sync all files shared with you to your local disk. If you use this method, you can utilise any 'client side filtering' rules that you have created to filter out files you do not want locally. This option will create a new folder locally, with sub-folders named after the person who shared the data with you.
### Syncing OneDrive Business Shared Files using Option 1
1. As per the above method for adding folders, select the shared file, then select to 'Add shorcut' to the file
![add_shared_file_shortcut](./images/add_shared_file_shortcut.png)
2. The final result online will look like this:
![add_shared_file_shortcut_added](./images/online_shared_file_link.png)
When using Microsoft Windows, this shared file will appear as the following:
![windows_view_shared_file_link](./images/windows_view_shared_file_link.png)
3. Sync your data using `onedrive --sync --verbose`. If you have just enabled the `sync_business_shared_items = "true"` configuration option, you will be required to perform a resync.
```
...
All application operations will be performed in the configured local 'sync_dir' directory: /home/alex/OneDrive
Fetching /delta response from the OneDrive API for Drive ID: b!bO8V7s9SSk6r7mWHpIjURotN33W1W2tEv3OXV_oFIdQimEdOHR-1So7CqeT1MfHA
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 2
Finished processing /delta JSON response from the OneDrive API
Processing 1 applicable changes and items received from Microsoft OneDrive
Processing OneDrive JSON item batch [1/1] to ensure consistent local state
Number of items to download from OneDrive: 1
Downloading file: ./file to share.docx.url ... done
Syncing this OneDrive Business Shared Folder: my_shared_folder
Fetching /delta response from the OneDrive API for Drive ID: b!BhWyqa7K_kqXqHtSIlsqjR5iJogxpWxDradnpVGTU2VxBOJh82Y6S4he4rdnGPBT
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
Finished processing /delta JSON response from the OneDrive API
No additional changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
Quota information is restricted or not available for this drive.
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: b!BhWyqa7K_kqXqHtSIlsqjR5iJogxpWxDradnpVGTU2VxBOJh82Y6S4he4rdnGPBT
Quota information is restricted or not available for this drive.
...
```
When this is viewed locally, on Linux, this shared folder is seen as the following:
![linux_view_shared_file_link](./images/linux_view_shared_file_link.png)
Any shared file link you add can utilise any 'client side filtering' rules that you have created.
### Syncing OneDrive Business Shared Files using Option 2
**NOTE:** When using option 2, all files that have been shared with you will be downloaded by default. To reduce this, first use `--list-shared-items` to list all shared items with your account, then use 'client side filtering' rules such as 'sync_list' configuration to selectivly sync all the files to your local system.
1. Review all items that have been shared with you by using `onedrive --list-shared-items`. This should display output similar to the following:
```
...
Listing available OneDrive Business Shared Items:
-----------------------------------------------------------------------------------
Shared File: large_document_shared.docx
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared File: no_download_access.docx
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared File: online_access_only.txt
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared File: read_only.txt
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared File: qewrqwerwqer.txt
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared File: dummy_file_to_share.docx
Shared By: testuser2 testuser2 (testuser2@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared Folder: Sub Folder 2
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared File: file to share.docx
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared Folder: Top Folder
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared Folder: my_shared_folder
Shared By: testuser2 testuser2 (testuser2@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
Shared Folder: Jenkins
Shared By: test user (testuser@mynasau3.onmicrosoft.com)
-----------------------------------------------------------------------------------
...
```
2. If applicable, add entries to a 'sync_list' file, to only sync the shared files that are of importance to you.
3. Run the command `onedrive --sync --verbose --sync-shared-files` to sync the shared files to your local file system. This will create a new local folder called 'Files Shared With Me', and will contain sub-directories named after the entity account that has shared the file with you. In that folder will reside the shared file:
```
...
Finished processing /delta JSON response from the OneDrive API
No additional changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
Syncing this OneDrive Business Shared Folder: my_shared_folder
Fetching /delta response from the OneDrive API for Drive ID: b!BhWyqa7K_kqXqHtSIlsqjR5iJogxpWxDradnpVGTU2VxBOJh82Y6S4he4rdnGPBT
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 0
Finished processing /delta JSON response from the OneDrive API
No additional changes or items that can be applied were discovered while processing the data received from Microsoft OneDrive
Quota information is restricted or not available for this drive.
Creating the OneDrive Business Shared Files Local Directory: /home/alex/OneDrive/Files Shared With Me
Checking for any applicable OneDrive Business Shared Files which need to be synced locally
Creating the OneDrive Business Shared File Users Local Directory: /home/alex/OneDrive/Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)
Creating the OneDrive Business Shared File Users Local Directory: /home/alex/OneDrive/Files Shared With Me/testuser2 testuser2 (testuser2@mynasau3.onmicrosoft.com)
Number of items to download from OneDrive: 7
Downloading file: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/file to share.docx ... done
OneDrive returned a 'HTTP 403 - Forbidden' - gracefully handling error
Unable to download this file as this was shared as read-only without download permission: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/no_download_access.docx
ERROR: File failed to download. Increase logging verbosity to determine why.
Downloading file: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/no_download_access.docx ... failed!
Downloading file: Files Shared With Me/testuser2 testuser2 (testuser2@mynasau3.onmicrosoft.com)/dummy_file_to_share.docx ... done
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 0% | ETA --:--:--
Downloading file: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/online_access_only.txt ... done
Downloading file: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/read_only.txt ... done
Downloading file: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/qewrqwerwqer.txt ... done
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 5% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 10% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 15% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 20% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 25% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 30% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 35% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 40% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 45% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 50% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 55% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 60% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 65% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 70% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 75% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 80% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 85% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 90% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 95% | ETA 00:00:00
Downloading: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... 100% | DONE in 00:00:00
Quota information is restricted or not available for this drive.
Downloading file: Files Shared With Me/test user (testuser@mynasau3.onmicrosoft.com)/large_document_shared.docx ... done
Quota information is restricted or not available for this drive.
Quota information is restricted or not available for this drive.
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: b!BhWyqa7K_kqXqHtSIlsqjR5iJogxpWxDradnpVGTU2VxBOJh82Y6S4he4rdnGPBT
Quota information is restricted or not available for this drive.
...
```
When this is viewed locally, on Linux, this 'Files Shared With Me' and content is seen as the following:
![files_shared_with_me_folder](./images/files_shared_with_me_folder.png)
Unfortunatly there is no Microsoft Windows equivalent for this capability.
## Known Issues
Shared folders, shared with you from people outside of your 'organisation' are unable to be synced. This is due to the Microsoft Graph API not presenting these folders.
Shared folders that match this scenario, when you view 'Shared' via OneDrive online, will have a 'world' symbol as per below:
![shared_with_me](./images/shared_with_me.JPG)
This issue is being tracked by: [#966](https://github.com/abraunegg/onedrive/issues/966)

View file

@ -1,12 +1,27 @@
# OneDrive Client for Linux Application Architecture
## How does the client work at a high level?
The client utilises the 'libcurl' library to communicate with the Microsoft Authentication Service and the Microsoft Graph API. The diagram below shows this high level interaction with the Microsoft services online:
The diagram below outlines at a high level the operational workflow of the OneDrive Client for Linux, demonstrating how it interacts with the OneDrive API to maintain synchronisation, manage local and cloud data integrity, and ensure that user data is accurately mirrored between the local filesystem and OneDrive cloud storage.
![client_use_of_libcurl](./puml/client_use_of_libcurl.png)
Depending on your operational environment, it is possible to 'tweak' the following options which will modify how libcurl operates with it's interaction with Microsoft OneDrive services:
* Downgrade all HTTPS operations to use HTTP1.1 ('force_http_11')
* Control how long a specific transfer should take before it is considered too slow and aborted ('operation_timeout')
* Control libcurl handling of DNS Cache Timeout ('dns_timeout')
* Control the maximum time allowed for the connection to be established ('connect_timeout')
* Control the timeout for activity on an established HTTPS connection ('data_timeout')
* Control what IP protocol version should be used when communicating with OneDrive ('ip_protocol_version')
* Control what User Agent is presented to Microsoft services ('user_agent')
**Note:** The default 'user_agent' value conforms to specific Microsoft requirements to identify as an ISV that complies with OneDrive traffic decoration requirements. Changing this value potentially will impact how Microsoft see's your client, thus your traffic may get throttled. For further information please read: https://learn.microsoft.com/en-us/sharepoint/dev/general-development/how-to-avoid-getting-throttled-or-blocked-in-sharepoint-online
Diving a little deeper into how the client operates, the diagram below outlines at a high level the operational workflow of the OneDrive Client for Linux, demonstrating how it interacts with the OneDrive API to maintain synchronisation, manage local and cloud data integrity, and ensure that user data is accurately mirrored between the local filesystem and OneDrive cloud storage.
![High Level Application Sequence](./puml/high_level_operational_process.png)
The above process involves several high level key stages:
The application operational processes have several high level key stages:
1. **Access Token Validation:** Initially, the client validates its access and the existing access token, refreshing it if necessary. This step ensures that the client has the required permissions to interact with the OneDrive API.

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 184 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 98 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 223 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 226 KiB

View file

@ -14,23 +14,23 @@ Distribution packages may be of an older release when compared to the latest rel
| Arch Linux<br><br>Manjaro Linux | [onedrive-abraunegg](https://aur.archlinux.org/packages/onedrive-abraunegg/) |<a href="https://aur.archlinux.org/packages/onedrive-abraunegg"><img src="https://repology.org/badge/version-for-repo/aur/onedrive-abraunegg.svg?header=" alt="AUR package" width="46" height="20"></a>|✔|✔|✔|✔ | Install via: `pamac build onedrive-abraunegg` from the Arch Linux User Repository (AUR)<br><br>**Note:** You must first install 'base-devel' as this is a pre-requisite for using the AUR<br><br>**Note:** If asked regarding a provider for 'd-runtime' and 'd-compiler', select 'liblphobos' and 'ldc'<br><br>**Note:** System must have at least 1GB of memory & 1GB swap space
| CentOS 8 | [onedrive](https://koji.fedoraproject.org/koji/packageinfo?packageID=26044) |<a href="https://koji.fedoraproject.org/koji/packageinfo?packageID=26044"><img src="https://repology.org/badge/version-for-repo/epel_8/onedrive.svg?header=" alt="CentOS 8 package" width="46" height="20"></a>|❌|✔|❌|✔| **Note:** You must install the EPEL Repository first |
| CentOS 9 | [onedrive](https://koji.fedoraproject.org/koji/packageinfo?packageID=26044) |<a href="https://koji.fedoraproject.org/koji/packageinfo?packageID=26044"><img src="https://repology.org/badge/version-for-repo/epel_9/onedrive.svg?header=" alt="CentOS 9 package" width="46" height="20"></a>|❌|✔|❌|✔| **Note:** You must install the EPEL Repository first |
| Debian 11 | [onedrive](https://packages.debian.org/bullseye/source/onedrive) |<a href="https://packages.debian.org/bullseye/source/onedrive"><img src="https://repology.org/badge/version-for-repo/debian_11/onedrive.svg?header=" alt="Debian 11 package" width="46" height="20"></a>|✔|✔|✔|✔| **Note:** Do not install from Debian Package Repositories<br><br>It is recommended that for Debian 11 that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) |
| Debian 12 | [onedrive](https://packages.debian.org/bookworm/source/onedrive) |<a href="https://packages.debian.org/bookworm/source/onedrive"><img src="https://repology.org/badge/version-for-repo/debian_12/onedrive.svg?header=" alt="Debian 12 package" width="46" height="20"></a>|✔|✔|✔|✔| **Note:** Do not install from Debian Package Repositories<br><br>It is recommended that for Debian 12 that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) |
| Debian 11 | [onedrive](https://packages.debian.org/bullseye/source/onedrive) |<a href="https://packages.debian.org/bullseye/source/onedrive"><img src="https://repology.org/badge/version-for-repo/debian_11/onedrive.svg?header=" alt="Debian 11 package" width="46" height="20"></a>|✔|✔|✔|✔| **Note:** Do not install from Debian Package Repositories as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Debian 11 that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) |
| Debian 12 | [onedrive](https://packages.debian.org/bookworm/source/onedrive) |<a href="https://packages.debian.org/bookworm/source/onedrive"><img src="https://repology.org/badge/version-for-repo/debian_12/onedrive.svg?header=" alt="Debian 12 package" width="46" height="20"></a>|✔|✔|✔|✔| **Note:** Do not install from Debian Package Repositories as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Debian 12 that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) |
| Debian Sid | [onedrive](https://packages.debian.org/sid/onedrive) |<a href="https://packages.debian.org/sid/onedrive"><img src="https://repology.org/badge/version-for-repo/debian_unstable/onedrive.svg?header=" alt="Debian Sid package" width="46" height="20"></a>|✔|✔|✔|✔| |
| Fedora | [onedrive](https://koji.fedoraproject.org/koji/packageinfo?packageID=26044) |<a href="https://koji.fedoraproject.org/koji/packageinfo?packageID=26044"><img src="https://repology.org/badge/version-for-repo/fedora_rawhide/onedrive.svg?header=" alt="Fedora Rawhide package" width="46" height="20"></a>|✔|✔|✔|✔| |
| Gentoo | [onedrive](https://gpo.zugaina.org/net-misc/onedrive) | No API Available |✔|✔|❌|❌| |
| Homebrew | [onedrive](https://formulae.brew.sh/formula/onedrive) | <a href="https://formulae.brew.sh/formula/onedrive"><img src="https://repology.org/badge/version-for-repo/homebrew/onedrive.svg?header=" alt="Homebrew package" width="46" height="20"></a> |❌|✔|❌|❌| |
| Linux Mint 20.x | [onedrive](https://community.linuxmint.com/software/view/onedrive) |<a href="https://community.linuxmint.com/software/view/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_20_04/onedrive.svg?header=" alt="Ubuntu 20.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Linux Mint Repositories<br><br>It is recommended that for Linux Mint that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Linux Mint 21.x | [onedrive](https://community.linuxmint.com/software/view/onedrive) |<a href="https://community.linuxmint.com/software/view/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_22_04/onedrive.svg?header=" alt="Ubuntu 22.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Linux Mint Repositories<br><br>It is recommended that for Linux Mint that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Linux Mint 20.x | [onedrive](https://community.linuxmint.com/software/view/onedrive) |<a href="https://community.linuxmint.com/software/view/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_20_04/onedrive.svg?header=" alt="Ubuntu 20.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Linux Mint Repositories as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Linux Mint that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Linux Mint 21.x | [onedrive](https://community.linuxmint.com/software/view/onedrive) |<a href="https://community.linuxmint.com/software/view/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_22_04/onedrive.svg?header=" alt="Ubuntu 22.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Linux Mint Repositories as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Linux Mint that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| NixOS | [onedrive](https://search.nixos.org/packages?channel=20.09&from=0&size=50&sort=relevance&query=onedrive)|<a href="https://search.nixos.org/packages?channel=20.09&from=0&size=50&sort=relevance&query=onedrive"><img src="https://repology.org/badge/version-for-repo/nix_unstable/onedrive.svg?header=" alt="nixpkgs unstable package" width="46" height="20"></a>|❌|✔|❌|❌| Use package `onedrive` either by adding it to `configuration.nix` or by using the command `nix-env -iA <channel name>.onedrive`. This does not install a service. To install a service, use unstable channel (will stabilize in 20.09) and add `services.onedrive.enable=true` in `configuration.nix`. You can also add a custom package using the `services.onedrive.package` option (recommended since package lags upstream). Enabling the service installs a default package too (based on the channel). You can also add multiple onedrive accounts trivially, see [documentation](https://github.com/NixOS/nixpkgs/pull/77734#issuecomment-575874225). |
| OpenSuSE | [onedrive](https://software.opensuse.org/package/onedrive) |<a href="https://software.opensuse.org/package/onedrive"><img src="https://repology.org/badge/version-for-repo/opensuse_network_tumbleweed/onedrive.svg?header=" alt="openSUSE Tumbleweed package" width="46" height="20"></a>|✔|✔|❌|❌| |
| OpenSuSE Build Service | [onedrive](https://build.opensuse.org/package/show/home:npreining:debian-ubuntu-onedrive/onedrive) | No API Available |✔|✔|✔|✔| Package Build Service for Debian and Ubuntu |
| Raspbian | [onedrive](https://archive.raspbian.org/raspbian/pool/main/o/onedrive/) |<a href="https://archive.raspbian.org/raspbian/pool/main/o/onedrive/"><img src="https://repology.org/badge/version-for-repo/raspbian_stable/onedrive.svg?header=" alt="Raspbian Stable package" width="46" height="20"></a> |❌|❌|✔|✔| **Note:** Do not install from Raspbian Package Repositories<br><br>It is recommended that for Raspbian that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) |
| Raspbian | [onedrive](https://archive.raspbian.org/raspbian/pool/main/o/onedrive/) |<a href="https://archive.raspbian.org/raspbian/pool/main/o/onedrive/"><img src="https://repology.org/badge/version-for-repo/raspbian_stable/onedrive.svg?header=" alt="Raspbian Stable package" width="46" height="20"></a> |❌|❌|✔|✔| **Note:** Do not install from Raspbian Package Repositories as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Raspbian that you install from OpenSuSE Build Service using the Debian Package Install [Instructions](ubuntu-package-install.md) |
| Slackware | [onedrive](https://slackbuilds.org/result/?search=onedrive&sv=) |<a href="https://slackbuilds.org/result/?search=onedrive&sv="><img src="https://repology.org/badge/version-for-repo/slackbuilds/onedrive.svg?header=" alt="SlackBuilds package" width="46" height="20"></a>|✔|✔|❌|❌| |
| Solus | [onedrive](https://dev.getsol.us/search/query/FB7PIf1jG9Z9/#R) |<a href="https://dev.getsol.us/search/query/FB7PIf1jG9Z9/#R"><img src="https://repology.org/badge/version-for-repo/solus/onedrive.svg?header=" alt="Solus package" width="46" height="20"></a>|✔|✔|❌|❌| |
| Ubuntu 20.04 | [onedrive](https://packages.ubuntu.com/focal/onedrive) |<a href="https://packages.ubuntu.com/focal/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_20_04/onedrive.svg?header=" alt="Ubuntu 20.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe<br><br>It is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Ubuntu 22.04 | [onedrive](https://packages.ubuntu.com/jammy/onedrive) |<a href="https://packages.ubuntu.com/jammy/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_22_04/onedrive.svg?header=" alt="Ubuntu 22.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe<br><br>It is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Ubuntu 23.04 | [onedrive](https://packages.ubuntu.com/lunar/onedrive) |<a href="https://packages.ubuntu.com/lunar/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_23_04/onedrive.svg?header=" alt="Ubuntu 23.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe<br><br>It is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Ubuntu 20.04 | [onedrive](https://packages.ubuntu.com/focal/onedrive) |<a href="https://packages.ubuntu.com/focal/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_20_04/onedrive.svg?header=" alt="Ubuntu 20.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Ubuntu 22.04 | [onedrive](https://packages.ubuntu.com/jammy/onedrive) |<a href="https://packages.ubuntu.com/jammy/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_22_04/onedrive.svg?header=" alt="Ubuntu 22.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Ubuntu 23.04 | [onedrive](https://packages.ubuntu.com/lunar/onedrive) |<a href="https://packages.ubuntu.com/lunar/onedrive"><img src="https://repology.org/badge/version-for-repo/ubuntu_23_04/onedrive.svg?header=" alt="Ubuntu 23.04 package" width="46" height="20"></a> |❌|✔|✔|✔| **Note:** Do not install from Ubuntu Universe as the package is obsolete and is not supported<br><br>For a supported application version, it is recommended that for Ubuntu that you install from OpenSuSE Build Service using the Ubuntu Package Install [Instructions](ubuntu-package-install.md) |
| Void Linux | [onedrive](https://voidlinux.org/packages/?arch=x86_64&q=onedrive) |<a href="https://voidlinux.org/packages/?arch=x86_64&q=onedrive"><img src="https://repology.org/badge/version-for-repo/void_x86_64/onedrive.svg?header=" alt="Void Linux x86_64 package" width="46" height="20"></a>|✔|✔|❌|❌| |
#### Important information for all Ubuntu and Ubuntu based distribution users:

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

View file

@ -0,0 +1,25 @@
@startuml
participant "OneDrive Client\nfor Linux" as od
participant "libcurl" as lc
participant "Microsoft Authentication Service\n(OAuth 2.0 Endpoint)" as oauth
participant "Microsoft Graph API" as graph
activate od
activate lc
od->oauth: Request access token
activate oauth
oauth-->od: Access token
deactivate oauth
loop API Communication
od->lc: Construct HTTPS request (with token)
activate lc
lc->graph: API Request
activate graph
graph-->lc: API Response
deactivate graph
lc-->od: Process response
deactivate lc
end
@enduml

Binary file not shown.

Before

Width:  |  Height:  |  Size: 36 KiB

After

Width:  |  Height:  |  Size: 39 KiB

View file

@ -13,7 +13,9 @@ class item {
quickXorHash: TEXT
sha256Hash: TEXT
remoteDriveId: TEXT
remoteParentId: TEXT
remoteId: TEXT
remoteType: TEXT
deltaLink: TEXT
syncStatus: TEXT
size: TEXT
@ -34,4 +36,4 @@ note left of N1
selectByPath_idx ON item (name, driveId, parentId)
end note
@enduml
@enduml

View file

@ -14,22 +14,21 @@ Originally derived as a 'fork' from the [skilion](https://github.com/skilion/one
This client represents a 100% re-imagining of the original work, addressing numerous notable bugs and issues while incorporating a significant array of new features. This client has been under active development since mid-2018.
## Features
* Supports 'Client Side Filtering' rules to determine what should be synced with Microsoft OneDrive
* Sync State Caching
* Real-Time local file monitoring with inotify
* Real-Time syncing of remote updates via webhooks
* File upload / download validation to ensure data integrity
* Resumable uploads
* Support OneDrive for Business (part of Office 365)
* Shared Folder support for OneDrive Personal and OneDrive Business accounts
* SharePoint / Office365 Shared Libraries
* Desktop notifications via libnotify
* Dry-run capability to test configuration changes
* Prevent major OneDrive accidental data deletion after configuration change
* Support for National cloud deployments (Microsoft Cloud for US Government, Microsoft Cloud Germany, Azure and Office 365 operated by 21Vianet in China)
* Supports single & multi-tenanted applications
* Supports rate limiting of traffic
* Supports multi-threaded uploads and downloads
* Compatible with OneDrive Personal, OneDrive for Business including accessing Microsoft SharePoint Libraries
* Provides rules for client-side filtering to select data for syncing with Microsoft OneDrive accounts
* Caches sync state for efficiency
* Supports a dry-run option for safe configuration testing
* Validates file transfers to ensure data integrity
* Monitors local files in real-time using inotify
* Supports interrupted uploads for completion at a later time
* Capability to sync remote updates immediately via webhooks
* Enhanced syncronisation speed with multi-threaded file transfers
* Manages traffic bandwidth use with rate limiting
* Supports seamless access to shared folders and files across both OneDrive Personal and OneDrive for Business accounts
* Supports national cloud deployments including Microsoft Cloud for US Government, Microsoft Cloud Germany and Azure and Office 365 operated by 21Vianet in China
* Supports sending desktop alerts using libnotify
* Protects against significant data loss on OneDrive after configuration changes
* Works with both single and multi-tenant applications
## What's missing
* Ability to encrypt/decrypt files on-the-fly when uploading/downloading files from OneDrive
@ -68,8 +67,8 @@ Refer to [docs/install.md](https://github.com/abraunegg/onedrive/blob/master/doc
### Configuration and Usage
Refer to [docs/usage.md](https://github.com/abraunegg/onedrive/blob/master/docs/usage.md)
### Configure OneDrive Business Shared Folders
Refer to [docs/business-shared-folders.md](https://github.com/abraunegg/onedrive/blob/master/docs/business-shared-folders.md)
### Configure OneDrive Business Shared Items
Refer to [docs/business-shared-items.md](https://github.com/abraunegg/onedrive/blob/master/docs/business-shared-items.md)
### Configure SharePoint / Office 365 Shared Libraries (Business or Education)
Refer to [docs/sharepoint-libraries.md](https://github.com/abraunegg/onedrive/blob/master/docs/sharepoint-libraries.md)

View file

@ -20,7 +20,6 @@ class ClientSideFiltering {
// Class variables
ApplicationConfig appConfig;
string[] paths;
string[] businessSharedItemsList;
Regex!char fileMask;
Regex!char directoryMask;
bool skipDirStrictMatch = false;
@ -41,11 +40,6 @@ class ClientSideFiltering {
loadSyncList(appConfig.syncListFilePath);
}
// Load the Business Shared Items file if it exists
if (exists(appConfig.businessSharedItemsFilePath)){
loadBusinessSharedItems(appConfig.businessSharedItemsFilePath);
}
// Configure skip_dir, skip_file, skip-dir-strict-match & skip_dotfiles from config entries
// Handle skip_dir configuration in config file
addLogEntry("Configuring skip_dir ...", ["debug"]);
@ -91,7 +85,6 @@ class ClientSideFiltering {
void shutdown() {
object.destroy(appConfig);
object.destroy(paths);
object.destroy(businessSharedItemsList);
object.destroy(fileMask);
object.destroy(directoryMask);
}
@ -109,19 +102,6 @@ class ClientSideFiltering {
file.close();
}
// load business_shared_folders file
void loadBusinessSharedItems(string filepath) {
// open file as read only
auto file = File(filepath, "r");
auto range = file.byLine();
foreach (line; range) {
// Skip comments in file
if (line.length == 0 || line[0] == ';' || line[0] == '#') continue;
businessSharedItemsList ~= buildNormalizedPath(line);
}
file.close();
}
// Configure the regex that will be used for 'skip_file'
void setFileMask(const(char)[] mask) {
fileMask = wild2regex(mask);

View file

@ -41,6 +41,8 @@ class ApplicationConfig {
immutable string defaultLogFileDir = "/var/log/onedrive";
// - Default configuration directory
immutable string defaultConfigDirName = "~/.config/onedrive";
// - Default 'OneDrive Business Shared Files' Folder Name
immutable string defaultBusinessSharedFilesDirectoryName = "Files Shared With Me";
// Microsoft Requirements
// - Default Application ID (abraunegg)
@ -106,7 +108,6 @@ class ApplicationConfig {
bool debugLogging = false;
long verbosityCount = 0;
// Was the application just authorised - paste of response uri
bool applicationAuthorizeResponseUri = false;
@ -121,6 +122,7 @@ class ApplicationConfig {
// Store the 'session_upload.CRC32-HASH' file path
string uploadSessionFilePath = "";
// API initialisation flags
bool apiWasInitialised = false;
bool syncEngineWasInitialised = false;
@ -161,25 +163,23 @@ class ApplicationConfig {
private string applicableConfigFilePath = "";
// - Store the 'sync_list' file path
string syncListFilePath = "";
// - Store the 'business_shared_items' file path
string businessSharedItemsFilePath = "";
// OneDrive Business Shared File handling - what directory will be used?
string configuredBusinessSharedFilesDirectoryName = "";
// Hash files so that we can detect when the configuration has changed, in items that will require a --resync
private string configHashFile = "";
private string configBackupFile = "";
private string syncListHashFile = "";
private string businessSharedItemsHashFile = "";
// Store the actual 'runtime' hash
private string currentConfigHash = "";
private string currentSyncListHash = "";
private string currentBusinessSharedItemsHash = "";
// Store the previous config files hash values (file contents)
private string previousConfigHash = "";
private string previousSyncListHash = "";
private string previousBusinessSharedItemsHash = "";
// Store items that come in from the 'config' file, otherwise these need to be set the the defaults
private string configFileSyncDir = defaultSyncDir;
private string configFileSkipFile = defaultSkipFile;
@ -197,7 +197,6 @@ class ApplicationConfig {
string[string] stringValues;
long[string] longValues;
bool[string] boolValues;
bool shellEnvironmentSet = false;
// Initialise the application configuration
@ -275,7 +274,7 @@ class ApplicationConfig {
longValues["ip_protocol_version"] = defaultIpProtocol; // 0 = IPv4 + IPv6, 1 = IPv4 Only, 2 = IPv6 Only
// Number of concurrent threads
longValues["threads"] = defaultConcurrentThreads; // Default is 8, user can increase or decrease
longValues["threads"] = defaultConcurrentThreads; // Default is 8, user can increase to max of 16 or decrease
// - Do we wish to upload only?
boolValues["upload_only"] = false;
@ -469,9 +468,6 @@ class ApplicationConfig {
// - What is the full path for the system 'config' file if it is required
systemConfigFilePath = buildNormalizedPath(buildPath(systemConfigDirName, "config"));
// - What is the full path for the 'business_shared_items'
businessSharedItemsFilePath = buildNormalizedPath(buildPath(configDirName, "business_shared_items"));
// To determine if any configuration items has changed, where a --resync would be required, we need to have a hash file for the following items
// - 'config.backup' file
// - applicable 'config' file
@ -480,8 +476,7 @@ class ApplicationConfig {
configBackupFile = buildNormalizedPath(buildPath(configDirName, ".config.backup"));
configHashFile = buildNormalizedPath(buildPath(configDirName, ".config.hash"));
syncListHashFile = buildNormalizedPath(buildPath(configDirName, ".sync_list.hash"));
businessSharedItemsHashFile = buildNormalizedPath(buildPath(configDirName, ".business_shared_items.hash"));
// Debug Output for application set variables based on configDirName
addLogEntry("refreshTokenFilePath = " ~ refreshTokenFilePath, ["debug"]);
addLogEntry("deltaLinkFilePath = " ~ deltaLinkFilePath, ["debug"]);
@ -494,8 +489,6 @@ class ApplicationConfig {
addLogEntry("configBackupFile = " ~ configBackupFile, ["debug"]);
addLogEntry("configHashFile = " ~ configHashFile, ["debug"]);
addLogEntry("syncListHashFile = " ~ syncListHashFile, ["debug"]);
addLogEntry("businessSharedItemsFilePath = " ~ businessSharedItemsFilePath, ["debug"]);
addLogEntry("businessSharedItemsHashFile = " ~ businessSharedItemsHashFile, ["debug"]);
// Configure the Hash and Backup File Permission Value
string valueToConvert = to!string(defaultFilePermissionMode);
@ -900,6 +893,7 @@ class ApplicationConfig {
boolValues["synchronize"] = false;
boolValues["force"] = false;
boolValues["list_business_shared_items"] = false;
boolValues["sync_business_shared_files"] = false;
boolValues["force_sync"] = false;
boolValues["with_editing_perms"] = false;
@ -995,6 +989,12 @@ class ApplicationConfig {
"get-O365-drive-id",
"Query and return the Office 365 Drive ID for a given Office 365 SharePoint Shared Library (DEPRECIATED)",
&stringValues["sharepoint_library_name"],
"list-shared-items",
"List OneDrive Business Shared Items",
&boolValues["list_business_shared_items"],
"sync-shared-files",
"Sync OneDrive Business Shared Files to the local filesystem",
&boolValues["sync_business_shared_files"],
"local-first",
"Synchronize from the local directory source first, before downloading changes from OneDrive.",
&boolValues["local_first"],
@ -1365,20 +1365,7 @@ class ApplicationConfig {
// Is sync_business_shared_items enabled and configured ?
addLogEntry(); // used instead of an empty 'writeln();' to ensure the line break is correct in the buffered console output ordering
addLogEntry("Config option 'sync_business_shared_items' = " ~ to!string(getValueBool("sync_business_shared_items")));
if (exists(businessSharedItemsFilePath)){
addLogEntry("Selective Business Shared Items configured = true");
addLogEntry("sync_business_shared_items contents:");
// Output the sync_business_shared_items contents
auto businessSharedItemsFileList = File(businessSharedItemsFilePath, "r");
auto range = businessSharedItemsFileList.byLine();
foreach (line; range)
{
addLogEntry(to!string(line));
}
} else {
addLogEntry("Selective Business Shared Items configured = false");
}
addLogEntry("Config option 'Shared Files Directory' = " ~ configuredBusinessSharedFilesDirectoryName);
// Are webhooks enabled?
addLogEntry(); // used instead of an empty 'writeln();' to ensure the line break is correct in the buffered console output ordering
@ -1518,9 +1505,6 @@ class ApplicationConfig {
if (currentSyncListHash != previousSyncListHash)
logAndSetDifference("sync_list file has been updated, --resync needed", 0);
if (currentBusinessSharedItemsHash != previousBusinessSharedItemsHash)
logAndSetDifference("business_shared_folders file has been updated, --resync needed", 1);
// Check for updates in the config file
if (currentConfigHash != previousConfigHash) {
addLogEntry("Application configuration file has been updated, checking if --resync needed");
@ -1665,7 +1649,14 @@ class ApplicationConfig {
break;
}
}
// Final override
// In certain situations, regardless of config 'resync' needed status, ignore this so that the application can display 'non-syncable' information
// Options that should now be looked at are:
// --list-shared-items
if (getValueBool("list_business_shared_items")) resyncRequired = false;
// Return the calculated boolean
return resyncRequired;
}
@ -1676,7 +1667,6 @@ class ApplicationConfig {
addLogEntry("Cleaning up configuration hash files", ["debug"]);
safeRemove(configHashFile);
safeRemove(syncListHashFile);
safeRemove(businessSharedItemsHashFile);
} else {
// --dry-run scenario ... technically we should not be making any local file changes .......
addLogEntry("DRY RUN: Not removing hash files as --dry-run has been used");
@ -1704,17 +1694,6 @@ class ApplicationConfig {
// Hash file should only be readable by the user who created it - 0600 permissions needed
syncListHashFile.setAttributes(convertedPermissionValue);
}
// Update 'update business_shared_items' files
if (exists(businessSharedItemsFilePath)) {
// update business_shared_folders hash
addLogEntry("Updating business_shared_items hash", ["debug"]);
std.file.write(businessSharedItemsHashFile, computeQuickXorHash(businessSharedItemsFilePath));
// Hash file should only be readable by the user who created it - 0600 permissions needed
businessSharedItemsHashFile.setAttributes(convertedPermissionValue);
}
} else {
// --dry-run scenario ... technically we should not be making any local file changes .......
addLogEntry("DRY RUN: Not updating hash files as --dry-run has been used");
@ -1746,18 +1725,6 @@ class ApplicationConfig {
// Generate the runtime hash for the 'sync_list' file
currentSyncListHash = computeQuickXorHash(syncListFilePath);
}
// Does a 'business_shared_items' file exist with a valid hash file
if (exists(businessSharedItemsFilePath)) {
if (!exists(businessSharedItemsHashFile)) {
// no existing hash file exists
std.file.write(businessSharedItemsHashFile, "initial-hash");
// Hash file should only be readable by the user who created it - 0600 permissions needed
businessSharedItemsHashFile.setAttributes(convertedPermissionValue);
}
// Generate the runtime hash for the 'sync_list' file
currentBusinessSharedItemsHash = computeQuickXorHash(businessSharedItemsFilePath);
}
}
// Read in the text values of the previous configurations
@ -1783,16 +1750,7 @@ class ApplicationConfig {
return EXIT_FAILURE;
}
}
if (exists(businessSharedItemsHashFile)) {
try {
previousBusinessSharedItemsHash = readText(businessSharedItemsHashFile);
} catch (std.file.FileException e) {
// Unable to access required hash file
addLogEntry("ERROR: Unable to access " ~ e.msg);
// Use exit scopes to shutdown API
return EXIT_FAILURE;
}
}
return 0;
}
@ -1850,10 +1808,22 @@ class ApplicationConfig {
// --list-shared-folders cannot be used with --resync and/or --resync-auth
if ((getValueBool("list_business_shared_items")) && ((getValueBool("resync")) || (getValueBool("resync_auth")))) {
addLogEntry("ERROR: --list-shared-folders cannot be used with --resync or --resync-auth");
addLogEntry("ERROR: --list-shared-items cannot be used with --resync or --resync-auth");
operationalConflictDetected = true;
}
// --list-shared-folders cannot be used with --sync or --monitor
if ((getValueBool("list_business_shared_items")) && ((getValueBool("synchronize")) || (getValueBool("monitor")))) {
addLogEntry("ERROR: --list-shared-items cannot be used with --sync or --monitor");
operationalConflictDetected = true;
}
// --sync-shared-files can ONLY be used with sync_business_shared_items
if ((getValueBool("sync_business_shared_files")) && (!getValueBool("sync_business_shared_items"))) {
addLogEntry("ERROR: The --sync-shared-files option can only be utilised if the 'sync_business_shared_items' configuration setting is enabled.");
operationalConflictDetected = true;
}
// --display-sync-status cannot be used with --resync and/or --resync-auth
if ((getValueBool("display_sync_status")) && ((getValueBool("resync")) || (getValueBool("resync_auth")))) {
addLogEntry("ERROR: --display-sync-status cannot be used with --resync or --resync-auth");
@ -2057,6 +2027,9 @@ class ApplicationConfig {
// What will runtimeSyncDirectory be actually set to?
addLogEntry("sync_dir: runtimeSyncDirectory set to: " ~ runtimeSyncDirectory, ["debug"]);
// Configure configuredBusinessSharedFilesDirectoryName
configuredBusinessSharedFilesDirectoryName = buildNormalizedPath(buildPath(runtimeSyncDirectory, defaultBusinessSharedFilesDirectoryName));
return runtimeSyncDirectory;
}

View file

@ -18,6 +18,7 @@ import util;
import log;
enum ItemType {
none,
file,
dir,
remote,
@ -37,7 +38,9 @@ struct Item {
string quickXorHash;
string sha256Hash;
string remoteDriveId;
string remoteParentId;
string remoteId;
ItemType remoteType;
string syncStatus;
string size;
}
@ -144,8 +147,27 @@ Item makeDatabaseItem(JSONValue driveItem) {
// Is the object a remote drive item - living on another driveId ?
if (isItemRemote(driveItem)) {
item.remoteDriveId = driveItem["remoteItem"]["parentReference"]["driveId"].str;
item.remoteId = driveItem["remoteItem"]["id"].str;
// Check and assign remoteDriveId
if ("parentReference" in driveItem["remoteItem"] && "driveId" in driveItem["remoteItem"]["parentReference"]) {
item.remoteDriveId = driveItem["remoteItem"]["parentReference"]["driveId"].str;
}
// Check and assign remoteParentId
if ("parentReference" in driveItem["remoteItem"] && "id" in driveItem["remoteItem"]["parentReference"]) {
item.remoteParentId = driveItem["remoteItem"]["parentReference"]["id"].str;
}
// Check and assign remoteId
if ("id" in driveItem["remoteItem"]) {
item.remoteId = driveItem["remoteItem"]["id"].str;
}
// Check and assign remoteType
if ("file" in driveItem["remoteItem"].object) {
item.remoteType = ItemType.file;
} else {
item.remoteType = ItemType.dir;
}
}
// We have 3 different operational modes where 'item.syncStatus' is used to flag if an item is synced or not:
@ -165,7 +187,7 @@ Item makeDatabaseItem(JSONValue driveItem) {
final class ItemDatabase {
// increment this for every change in the db schema
immutable int itemDatabaseVersion = 12;
immutable int itemDatabaseVersion = 13;
Database db;
string insertItemStmt;
@ -236,12 +258,12 @@ final class ItemDatabase {
db.exec("PRAGMA locking_mode = EXCLUSIVE");
insertItemStmt = "
INSERT OR REPLACE INTO item (driveId, id, name, remoteName, type, eTag, cTag, mtime, parentId, quickXorHash, sha256Hash, remoteDriveId, remoteId, syncStatus, size)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15)
INSERT OR REPLACE INTO item (driveId, id, name, remoteName, type, eTag, cTag, mtime, parentId, quickXorHash, sha256Hash, remoteDriveId, remoteParentId, remoteId, remoteType, syncStatus, size)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12, ?13, ?14, ?15, ?16, ?17)
";
updateItemStmt = "
UPDATE item
SET name = ?3, remoteName = ?4, type = ?5, eTag = ?6, cTag = ?7, mtime = ?8, parentId = ?9, quickXorHash = ?10, sha256Hash = ?11, remoteDriveId = ?12, remoteId = ?13, syncStatus = ?14, size = ?15
SET name = ?3, remoteName = ?4, type = ?5, eTag = ?6, cTag = ?7, mtime = ?8, parentId = ?9, quickXorHash = ?10, sha256Hash = ?11, remoteDriveId = ?12, remoteParentId = ?13, remoteId = ?14, remoteType = ?15, syncStatus = ?16, size = ?17
WHERE driveId = ?1 AND id = ?2
";
selectItemByIdStmt = "
@ -279,7 +301,9 @@ final class ItemDatabase {
quickXorHash TEXT,
sha256Hash TEXT,
remoteDriveId TEXT,
remoteParentId TEXT,
remoteId TEXT,
remoteType TEXT,
deltaLink TEXT,
syncStatus TEXT,
size TEXT,
@ -447,12 +471,14 @@ final class ItemDatabase {
bind(2, id);
bind(3, name);
bind(4, remoteName);
// type handling
string typeStr = null;
final switch (type) with (ItemType) {
case file: typeStr = "file"; break;
case dir: typeStr = "dir"; break;
case remote: typeStr = "remote"; break;
case unknown: typeStr = "unknown"; break;
case none: typeStr = null; break;
}
bind(5, typeStr);
bind(6, eTag);
@ -462,15 +488,26 @@ final class ItemDatabase {
bind(10, quickXorHash);
bind(11, sha256Hash);
bind(12, remoteDriveId);
bind(13, remoteId);
bind(14, syncStatus);
bind(15, size);
bind(13, remoteParentId);
bind(14, remoteId);
// remoteType handling
string remoteTypeStr = null;
final switch (remoteType) with (ItemType) {
case file: remoteTypeStr = "file"; break;
case dir: remoteTypeStr = "dir"; break;
case remote: remoteTypeStr = "remote"; break;
case unknown: remoteTypeStr = "unknown"; break;
case none: remoteTypeStr = null; break;
}
bind(15, remoteTypeStr);
bind(16, syncStatus);
bind(17, size);
}
}
private Item buildItem(Statement.Result result) {
assert(!result.empty, "The result must not be empty");
assert(result.front.length == 16, "The result must have 16 columns");
assert(result.front.length == 18, "The result must have 18 columns");
Item item = {
// column 0: driveId
@ -485,10 +522,12 @@ final class ItemDatabase {
// column 9: quickXorHash
// column 10: sha256Hash
// column 11: remoteDriveId
// column 12: remoteId
// column 13: deltaLink
// column 14: syncStatus
// column 15: size
// column 12: remoteParentId
// column 13: remoteId
// column 14: remoteType
// column 15: deltaLink
// column 16: syncStatus
// column 17: size
driveId: result.front[0].dup,
id: result.front[1].dup,
@ -502,17 +541,30 @@ final class ItemDatabase {
quickXorHash: result.front[9].dup,
sha256Hash: result.front[10].dup,
remoteDriveId: result.front[11].dup,
remoteId: result.front[12].dup,
// Column 13 is deltaLink - not set here
syncStatus: result.front[14].dup,
size: result.front[15].dup
remoteParentId: result.front[12].dup,
remoteId: result.front[13].dup,
// Column 14 is remoteType - not set here
// Column 15 is deltaLink - not set here
syncStatus: result.front[16].dup,
size: result.front[17].dup
};
// Configure item.type
switch (result.front[4]) {
case "file": item.type = ItemType.file; break;
case "dir": item.type = ItemType.dir; break;
case "remote": item.type = ItemType.remote; break;
default: assert(0, "Invalid item type");
}
// Configure item.remoteType
switch (result.front[14]) {
// We only care about 'dir' and 'file' for 'remote' items
case "file": item.remoteType = ItemType.file; break;
case "dir": item.remoteType = ItemType.dir; break;
default: item.remoteType = ItemType.none; break; // Default to ItemType.none
}
// Return item
return item;
}

View file

@ -337,19 +337,25 @@ int main(string[] cliArgs) {
processResyncDatabaseRemoval(runtimeDatabaseFile);
}
} else {
// Has any of our application configuration that would require a --resync been changed?
if (appConfig.applicationChangeWhereResyncRequired()) {
// Application configuration has changed however --resync not issued, fail fast
addLogEntry();
addLogEntry("An application configuration change has been detected where a --resync is required");
addLogEntry();
return EXIT_RESYNC_REQUIRED;
} else {
// No configuration change that requires a --resync to be issued
// Make a backup of the applicable configuration file
appConfig.createBackupConfigFile();
// Update hash files and generate a new config backup
appConfig.updateHashContentsForConfigFiles();
// Is the application currently authenticated? If not, it is pointless checking if a --resync is required until the application is authenticated
if (exists(appConfig.refreshTokenFilePath)) {
// Has any of our application configuration that would require a --resync been changed?
if (appConfig.applicationChangeWhereResyncRequired()) {
// Application configuration has changed however --resync not issued, fail fast
addLogEntry();
addLogEntry("An application configuration change has been detected where a --resync is required");
addLogEntry();
return EXIT_RESYNC_REQUIRED;
} else {
// No configuration change that requires a --resync to be issued
// Special cases need to be checked - if these options were enabled, it creates a false 'Resync Required' flag, so do not create a backup
if ((!appConfig.getValueBool("list_business_shared_items"))) {
// Make a backup of the applicable configuration file
appConfig.createBackupConfigFile();
// Update hash files and generate a new config backup
appConfig.updateHashContentsForConfigFiles();
}
}
}
}
@ -446,8 +452,9 @@ int main(string[] cliArgs) {
// Are we performing some sort of 'no-sync' task?
// - Are we obtaining the Office 365 Drive ID for a given Office 365 SharePoint Shared Library?
// - Are we displaying the sync satus?
// - Are we getting the URL for a file online
// - Are we listing who modified a file last online
// - Are we getting the URL for a file online?
// - Are we listing who modified a file last online?
// - Are we listing OneDrive Business Shared Items?
// - Are we createing a shareable link for an existing file on OneDrive?
// - Are we just creating a directory online, without any sync being performed?
// - Are we just deleting a directory online, without any sync being performed?
@ -499,6 +506,20 @@ int main(string[] cliArgs) {
return EXIT_SUCCESS;
}
// --list-shared-items - Are we listing OneDrive Business Shared Items
if (appConfig.getValueBool("list_business_shared_items")) {
// Is this a business account type?
if (appConfig.accountType == "business") {
// List OneDrive Business Shared Items
syncEngineInstance.listBusinessSharedObjects();
} else {
addLogEntry("ERROR: Unsupported account type for listing OneDrive Business Shared Items");
}
// Exit application
// Use exit scopes to shutdown API
return EXIT_SUCCESS;
}
// --create-share-link - Are we createing a shareable link for an existing file on OneDrive?
if (appConfig.getValueString("create_share_link") != "") {
// Query OneDrive for the file, and if valid, create a shareable link for the file

View file

@ -512,6 +512,13 @@ class OneDriveApi {
return get(url);
}
// Return all the items that are shared with the user
// https://docs.microsoft.com/en-us/graph/api/drive-sharedwithme
JSONValue getSharedWithMe() {
checkAccessTokenExpired();
return get(sharedWithMeUrl);
}
// Create a shareable link for an existing file on OneDrive based on the accessScope JSON permissions
// https://docs.microsoft.com/en-us/onedrive/developer/rest-api/api/driveitem_createlink
JSONValue createShareableLink(string driveId, string id, JSONValue accessScope) {
@ -1584,7 +1591,7 @@ class OneDriveApi {
case 403:
// OneDrive responded that the user is forbidden
addLogEntry("OneDrive returned a 'HTTP 403 - Forbidden' - gracefully handling error", ["verbose"]);
break;
throw new OneDriveException(curlEngine.http.statusLine.code, curlEngine.http.statusLine.reason);
// 404 - Item not found
case 404:

1069
src/sync.d

File diff suppressed because it is too large Load diff