mirror of
https://github.com/abraunegg/onedrive
synced 2026-03-14 14:35:46 +01:00
* Add a new configuration option 'force_session_upload' so that the local timestamp is uploaded and used by the API directly, rather than the API generating its own timestamp * Update setLocalPathTimestamp() function to only set the timestamp of the local file if this is determined to be actually required * Update allow.txt - add fracsec, howto * Add config option 'force_session_upload' to 'config' file * Add 'Compatibility with Editors and Applications Using Atomic Save Operations' * Update application-config-options.md to document 'force_session_upload' option * Add usage note for 'force_session_upload' option as a recommendation for applications that use atomic saves * Update usage.md - add link(s) to other docs for easier reference / visibility
This commit is contained in:
abraunegg
GitHub
parent
a51fabc22c
commit
bad8e7bb20
7 changed files with 320 additions and 56 deletions
2
.github/actions/spelling/allow.txt
vendored
2
.github/actions/spelling/allow.txt
vendored
|
|
@ -127,6 +127,7 @@ ffat
|
|||
FFFD
|
||||
fhandler
|
||||
flto
|
||||
fracsec
|
||||
fstack
|
||||
FState
|
||||
fullchain
|
||||
|
|
@ -151,6 +152,7 @@ gshared
|
|||
GVariant
|
||||
hideonindex
|
||||
hnsecs
|
||||
howto
|
||||
hskrieg
|
||||
htons
|
||||
idk
|
||||
|
|
|
|||
3
config
3
config
|
|
@ -76,6 +76,9 @@
|
|||
## This setting controls the application HTTP protocol version, downgrading to HTTP/1.1 when enabled.
|
||||
#force_http_11 = "false"
|
||||
|
||||
## This option, when enabled, forces the client to use a 'session' upload, which, when the 'file' is uploaded by the session, this includes the local timestamp of the file
|
||||
#force_session_upload = "false"
|
||||
|
||||
## This setting controls the application IP protocol used when communicating with Microsoft OneDrive.
|
||||
#ip_protocol_version = "0"
|
||||
|
||||
|
|
|
|||
|
|
@ -29,6 +29,7 @@ Before reading this document, please ensure you are running application version
|
|||
- [dry_run](#dry_run)
|
||||
- [enable_logging](#enable_logging)
|
||||
- [force_http_11](#force_http_11)
|
||||
- [force_session_upload](#force_session_upload)
|
||||
- [ip_protocol_version](#ip_protocol_version)
|
||||
- [local_first](#local_first)
|
||||
- [log_dir](#log_dir)
|
||||
|
|
@ -404,6 +405,18 @@ _**Config Example:**_ `force_http_11 = "false"` or `force_http_11 = "true"`
|
|||
|
||||
_**CLI Option Use:**_ `--force-http-11`
|
||||
|
||||
|
||||
### force_session_upload
|
||||
_**Description:**_ This option, when enabled, forces the client to use a 'session' upload, which, when the 'file' is uploaded by the session, this includes the local timestamp of the file.
|
||||
|
||||
_**Value Type:**_ Boolean
|
||||
|
||||
_**Default Value:**_ False
|
||||
|
||||
_**Config Example:**_ `force_session_upload = "false"` or `force_session_upload = "true"`
|
||||
|
||||
_**CLI Option Use:**_ *None - this is a config file option only*
|
||||
|
||||
### ip_protocol_version
|
||||
_**Description:**_ This setting controls the application IP protocol that should be used when communicating with Microsoft OneDrive. The default is to use IPv4 and IPv6 networks for communicating to Microsoft OneDrive.
|
||||
|
||||
|
|
|
|||
115
docs/usage.md
115
docs/usage.md
|
|
@ -9,6 +9,7 @@ Before reading this document, please ensure you are running application version
|
|||
- [Upgrading from the 'skilion' Client](#upgrading-from-the-skilion-client)
|
||||
- [Guidelines for Local File and Folder Naming in the Synchronisation Directory](#guidelines-for-local-file-and-folder-naming-in-the-synchronisation-directory)
|
||||
- [Support for Microsoft Azure Information Protected Files](#support-for-microsoft-azure-information-protected-files)
|
||||
- [Compatibility with Editors and Applications Using Atomic Save Operations](#compatibility-with-editors-and-applications-using-atomic-save-operations)
|
||||
- [Compatibility with curl](#compatibility-with-curl)
|
||||
- [First Steps](#first-steps)
|
||||
- [Authorise the Application with Your Microsoft OneDrive Account](#authorise-the-application-with-your-microsoft-onedrive-account)
|
||||
|
|
@ -129,6 +130,76 @@ The above guidelines are essential for maintaining synchronisation integrity wit
|
|||
>
|
||||
> Please use the `--disable-download-validation` option with extreme caution and understand the risk if you enable it.
|
||||
|
||||
### Compatibility with Editors and Applications Using Atomic Save Operations
|
||||
|
||||
Many modern editors and applications—including `vi`, `vim`, `nvim`, `emacs`, `LibreOffice`, and others—use *atomic save* strategies to preserve data integrity when writing files. This section outlines how such operations interact with the `onedrive` client, what users can expect, and why certain side effects (such as editor warnings or perceived timestamp discrepancies) may occur.
|
||||
|
||||
#### How Atomic Save Operations Work
|
||||
|
||||
When these applications save a file, they typically follow this sequence:
|
||||
|
||||
1. **Create a Temporary File**
|
||||
A new file is written with the updated content, often in the same directory as the original.
|
||||
|
||||
2. **Flush to Disk**
|
||||
The temporary file is flushed to disk using `fsync()` or an equivalent method to ensure data safety.
|
||||
|
||||
3. **Atomic Rename**
|
||||
The temporary file is renamed to the original filename using the `rename()` syscall.
|
||||
This is an atomic operation on Linux, meaning the original file is *replaced*, not modified.
|
||||
|
||||
4. **Remove Lock or Swap Files**
|
||||
Auxiliary files used during editing (e.g., `.swp`, `.#filename`) are deleted.
|
||||
|
||||
As a result, the saved file is **technically a new file** with a new inode and a new timestamp, even if the filename remains unchanged.
|
||||
|
||||
#### How This Affects the OneDrive Client
|
||||
|
||||
When the `onedrive` client observes such an atomic save operation via `inotify`, it detects:
|
||||
|
||||
- The original file as *deleted*.
|
||||
- A new file (with the same name) as *created*.
|
||||
|
||||
The client responds accordingly:
|
||||
|
||||
- The "new" file is uploaded to Microsoft OneDrive.
|
||||
- After upload, Microsoft assigns its own *modification timestamp* to the file.
|
||||
- To ensure consistency between local and remote states, the client updates the local file’s timestamp to match the **exact time** stored in OneDrive.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Microsoft OneDrive does **not support fractional-second precision** in file timestamps—only whole seconds. As a result, small discrepancies may occur if the local file system supports higher-resolution timestamps.
|
||||
|
||||
This behaviour ensures accurate syncing and content integrity, but may lead to subtle side effects in timestamp-sensitive applications.
|
||||
|
||||
#### Expected Side Effects
|
||||
|
||||
- **Timestamp Alignment for Atomic Saves**
|
||||
Editors that rely on local file timestamps (rather than content checksums) previously issued warnings that a file had changed unexpectedly—typically because the `onedrive` client updated the modification time after upload.
|
||||
This client preserves the original modification timestamp if only fractional seconds differ, preventing unnecessary timestamp changes. As a result, editors such as `vi`, `vim`, `nvim`, `emacs`, and `LibreOffice` should not trigger warnings when saving files using atomic operations.
|
||||
|
||||
- **False Conflict Prompts (Collaborative Editing)**
|
||||
In collaborative editing scenarios—such as with LibreOffice or shared OneDrive folders—conflict prompts may still occur if another user or device modifies a file, resulting in a meaningful timestamp or content change.
|
||||
However, for local edits using atomic save methods, the client now avoids unnecessary timestamp updates, effectively eliminating false conflicts in those cases.
|
||||
|
||||
#### Recommendation
|
||||
|
||||
If you are using editors that rely on strict timestamp semantics and wish to minimise interference from the `onedrive` client:
|
||||
|
||||
- Save your work, then pause or temporarily stop sync (`onedrive --monitor`).
|
||||
- Resume syncing when finished.
|
||||
- Configure the client to ignore such files via the `skip_file` setting if they do not need to be synced.
|
||||
- Configure the client to use 'session uploads' for all files via the `force_session_upload` setting. This option, when enabled, forces the client to use a 'session' upload, which, when the 'file' is uploaded by the session, this includes the actual local timestamp (without fractional seconds) of the file that Microsoft OneDrive should store.
|
||||
|
||||
#### Summary
|
||||
|
||||
The `onedrive` client is fully compatible with applications that use atomic save operations. Users should be aware that:
|
||||
|
||||
- Atomic saves result in the file being treated as a new item.
|
||||
- Timestamps may be adjusted post-upload to match OneDrive's stored value.
|
||||
- In rare cases, timestamp-sensitive applications may display warnings or prompts.
|
||||
|
||||
This behaviour is by design and ensures consistency and data integrity between your local filesystem and the OneDrive cloud.
|
||||
|
||||
### Compatibility with curl
|
||||
If your system uses curl < 7.47.0, curl will default to HTTP/1.1 for HTTPS operations, and the client will follow suit, using HTTP/1.1.
|
||||
|
||||
|
|
@ -338,6 +409,50 @@ To make these changes permanent on your system, refer to your OS reference docum
|
|||
|
||||
## Using the OneDrive Client for Linux to synchronise your data
|
||||
|
||||
### Client Documentation
|
||||
|
||||
The following documents provide detailed guidance on installing, configuring, and using the OneDrive Client for Linux:
|
||||
|
||||
* **[advanced-usage.md](https://github.com/abraunegg/onedrive/blob/master/docs/advanced-usage.md)**
|
||||
Instructions for advanced configurations, including multiple account setups, Docker usage, dual-boot scenarios, and syncing to mounted directories.
|
||||
|
||||
* **[application-config-options.md](https://github.com/abraunegg/onedrive/blob/master/docs/application-config-options.md)**
|
||||
Comprehensive list and explanation of all configuration file and command-line options available in the client.
|
||||
|
||||
* **[application-security.md](https://github.com/abraunegg/onedrive/blob/master/docs/application-security.md)**
|
||||
Details on security considerations and practices related to the OneDrive client.
|
||||
|
||||
* **[business-shared-items.md](https://github.com/abraunegg/onedrive/blob/master/docs/business-shared-items.md)**
|
||||
Instructions on syncing shared items in OneDrive for Business accounts.
|
||||
|
||||
* **[client-architecture.md](https://github.com/abraunegg/onedrive/blob/master/docs/client-architecture.md)**
|
||||
Overview of the client's architecture and design principles.
|
||||
|
||||
* **[docker.md](https://github.com/abraunegg/onedrive/blob/master/docs/docker.md)**
|
||||
Instructions for running the OneDrive client within Docker containers.
|
||||
|
||||
* **[known-issues.md](https://github.com/abraunegg/onedrive/blob/master/docs/known-issues.md)**
|
||||
List of known issues and limitations of the OneDrive client.
|
||||
|
||||
* **[national-cloud-deployments.md](https://github.com/abraunegg/onedrive/blob/master/docs/national-cloud-deployments.md)**
|
||||
Information on deploying the client in national cloud environments.
|
||||
|
||||
* **[podman.md](https://github.com/abraunegg/onedrive/blob/master/docs/podman.md)**
|
||||
Guide for running the OneDrive client using Podman containers.
|
||||
|
||||
* **[sharepoint-libraries.md](https://github.com/abraunegg/onedrive/blob/master/docs/sharepoint-libraries.md)**
|
||||
Instructions for syncing SharePoint document libraries.
|
||||
|
||||
* **[ubuntu-package-install.md](https://github.com/abraunegg/onedrive/blob/master/docs/ubuntu-package-install.md)**
|
||||
Specific instructions for installing the client on Ubuntu systems.
|
||||
|
||||
* **[webhooks.md](https://github.com/abraunegg/onedrive/blob/master/docs/webhooks.md)**
|
||||
Information on configuring and using webhooks with the OneDrive client.
|
||||
|
||||
Further documentation not listed above can be found here: https://github.com/abraunegg/onedrive/blob/master/docs/
|
||||
|
||||
Please read these additional references to assist you with installing, configuring, and using the OneDrive Client for Linux.
|
||||
|
||||
### Increasing application logging level
|
||||
When running a sync (`--sync`) or using monitor mode (`--monitor`), it may be desirable to see additional information regarding the progress and operation of the client. For example, for a `--sync` command, this would be:
|
||||
```text
|
||||
|
|
|
|||
82
src/config.d
82
src/config.d
|
|
@ -300,68 +300,82 @@ class ApplicationConfig {
|
|||
// Number of concurrent threads
|
||||
longValues["threads"] = defaultConcurrentThreads; // Default is 8, user can increase to max of 16 or decrease
|
||||
|
||||
// - Do we wish to upload only?
|
||||
// Do we wish to upload only?
|
||||
boolValues["upload_only"] = false;
|
||||
// - Do we need to check for the .nomount file on the mount point?
|
||||
// Do we need to check for the .nomount file on the mount point?
|
||||
boolValues["check_nomount"] = false;
|
||||
// - Do we need to check for the .nosync file anywhere?
|
||||
// Do we need to check for the .nosync file anywhere?
|
||||
boolValues["check_nosync"] = false;
|
||||
// - Do we wish to download only?
|
||||
// Do we wish to download only?
|
||||
boolValues["download_only"] = false;
|
||||
// - Do we disable notifications?
|
||||
// Do we disable notifications?
|
||||
boolValues["disable_notifications"] = false;
|
||||
// - Do we bypass all the download validation?
|
||||
// This is critically important not to disable, but because of SharePoint 'feature' can be highly desirable to enable
|
||||
// Do we bypass all the download validation?
|
||||
// - This is critically important not to disable, but because of SharePoint 'feature' can be highly desirable to enable
|
||||
boolValues["disable_download_validation"] = false;
|
||||
// - Do we bypass all the upload validation?
|
||||
// This is critically important not to disable, but because of SharePoint 'feature' can be highly desirable to enable
|
||||
// Do we bypass all the upload validation?
|
||||
// - This is critically important not to disable, but because of SharePoint 'feature' can be highly desirable to enable
|
||||
boolValues["disable_upload_validation"] = false;
|
||||
// - Do we enable logging?
|
||||
// Do we enable logging?
|
||||
boolValues["enable_logging"] = false;
|
||||
// - Do we force HTTP 1.1 for connections to the OneDrive API
|
||||
// By default we use the curl library default, which should be HTTP2 for most operations governed by the OneDrive API
|
||||
// Do we force HTTP 1.1 for connections to the OneDrive API
|
||||
// - By default we use the curl library default, which should be HTTP2 for most operations governed by the OneDrive API
|
||||
boolValues["force_http_11"] = false;
|
||||
// - Do we treat the local file system as the source of truth for our data?
|
||||
// Do we treat the local file system as the source of truth for our data?
|
||||
boolValues["local_first"] = false;
|
||||
// - Do we ignore local file deletes, so that all files are retained online?
|
||||
// Do we ignore local file deletes, so that all files are retained online?
|
||||
boolValues["no_remote_delete"] = false;
|
||||
// - Do we skip symbolic links?
|
||||
// Do we skip symbolic links?
|
||||
boolValues["skip_symlinks"] = false;
|
||||
// - Do we enable debugging for all HTTPS flows. Critically important for debugging API issues.
|
||||
// Do we enable debugging for all HTTPS flows. Critically important for debugging API issues.
|
||||
boolValues["debug_https"] = false;
|
||||
// - Do we skip .files and .folders?
|
||||
// Do we skip .files and .folders?
|
||||
boolValues["skip_dotfiles"] = false;
|
||||
// - Do we perform a 'dry-run' with no local or remote changes actually being performed?
|
||||
// Do we perform a 'dry-run' with no local or remote changes actually being performed?
|
||||
boolValues["dry_run"] = false;
|
||||
// - Do we sync all the files in the 'sync_dir' root?
|
||||
// Do we sync all the files in the 'sync_dir' root?
|
||||
boolValues["sync_root_files"] = false;
|
||||
// - Do we delete source after successful transfer?
|
||||
// Do we delete source after successful transfer?
|
||||
boolValues["remove_source_files"] = false;
|
||||
// - Do we perform strict matching for skip_dir?
|
||||
// Do we perform strict matching for skip_dir?
|
||||
boolValues["skip_dir_strict_match"] = false;
|
||||
// - Do we perform a --resync?
|
||||
// Do we perform a --resync?
|
||||
boolValues["resync"] = false;
|
||||
// - resync now needs to be acknowledged based on the 'risk' of using it
|
||||
// 'resync' now needs to be acknowledged based on the 'risk' of using it
|
||||
boolValues["resync_auth"] = false;
|
||||
// - Ignore data safety checks and overwrite local data rather than preserve & rename
|
||||
// This is a config file option ONLY
|
||||
// Ignore data safety checks and overwrite local data rather than preserve & rename
|
||||
// - This is a config file option ONLY
|
||||
boolValues["bypass_data_preservation"] = false;
|
||||
// - Allow enable / disable of the syncing of OneDrive Business Shared items (files & folders) via configuration file
|
||||
// Allow enable / disable of the syncing of OneDrive Business Shared items (files & folders) via configuration file
|
||||
boolValues["sync_business_shared_items"] = false;
|
||||
// - Log to application output running configuration values
|
||||
// Log to application output running configuration values
|
||||
boolValues["display_running_config"] = false;
|
||||
// - Configure read-only authentication scope
|
||||
// Configure read-only authentication scope
|
||||
boolValues["read_only_auth_scope"] = false;
|
||||
// - Flag to cleanup local files when using --download-only
|
||||
// Flag to cleanup local files when using --download-only
|
||||
boolValues["cleanup_local_files"] = false;
|
||||
// - Perform a permanentDelete on deletion activities
|
||||
// Perform a permanentDelete on deletion activities
|
||||
boolValues["permanent_delete"] = false;
|
||||
// - Controls how the application handles the Microsoft SharePoint 'feature' of modifying all PDF, MS Office & HTML files with added XML content post upload
|
||||
// There are 2 ways to solve this:
|
||||
// 1. Download the modified file immediately after upload as per v2.4.x (default)
|
||||
// 2. Create a new online version of the file, which then contributes to the users 'quota'
|
||||
|
||||
// Controls how the application handles the Microsoft SharePoint 'feature' of modifying all PDF, MS Office & HTML files with added XML content post upload
|
||||
// - There are 2 ways to solve this:
|
||||
// 1. Download the modified file immediately after upload as per v2.4.x (default)
|
||||
// 2. Create a new online version of the file, which then contributes to the users 'quota'
|
||||
boolValues["create_new_file_version"] = false;
|
||||
|
||||
// Some Linux editors (vi|vim|nvim|emacs|LibreOffice) use use a safe file-save strategy designed to avoid data corruption. As such, as part of this Process
|
||||
// they 'track' the last modified timestamp of the 'new' file that they create on file save (regardless of new file, modified file)
|
||||
// If *any* other application in the background then 'updates' this timestamp, these Linux editors complain saying that the file has changed:
|
||||
//
|
||||
// WARNING: The file has been changed since reading it!!!
|
||||
// Do you really want to write to it (y/n)?
|
||||
//
|
||||
// This is simply because they are looking at the timestamp and *not* if the content has actually changed .... a poor design on those editors
|
||||
//
|
||||
// This option, when enabled, forces the client to use a 'session' upload, which, when the 'file' is uploaded by the session, this includes the local timestamp of the file
|
||||
// and Microsoft OneDrive should be respecting this timestamp as the timestamp to use|set when storing that file online
|
||||
boolValues["force_session_upload"] = false;
|
||||
|
||||
// Webhook Feature Options
|
||||
boolValues["webhook_enabled"] = false;
|
||||
stringValues["webhook_public_url"] = "";
|
||||
|
|
|
|||
89
src/sync.d
89
src/sync.d
|
|
@ -2965,7 +2965,7 @@ class SyncEngine {
|
|||
// updated by the local Operating System with the latest timestamp - as this is normal operation
|
||||
// as the directory has been modified
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, newItemPath, newDatabaseItem.mtime);
|
||||
setLocalPathTimestamp(dryRun, newItemPath, newDatabaseItem.mtime);
|
||||
|
||||
// Save the newDatabaseItem to the database
|
||||
saveDatabaseItem(newDatabaseItem);
|
||||
|
|
@ -3277,7 +3277,7 @@ class SyncEngine {
|
|||
// which is 'correct' .. but we need to report locally the online timestamp here as the move was made online
|
||||
if (changedOneDriveItem.type == ItemType.file) {
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, changedItemPath, changedOneDriveItem.mtime);
|
||||
setLocalPathTimestamp(dryRun, changedItemPath, changedOneDriveItem.mtime);
|
||||
}
|
||||
} else {
|
||||
// --dry-run situation - the actual rename did not occur - but we need to track like it did
|
||||
|
|
@ -3699,7 +3699,7 @@ class SyncEngine {
|
|||
if (debugLogging) {addLogEntry("Downloaded file matches reported size and reported file hash", ["debug"]);}
|
||||
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, newItemPath, itemModifiedTime);
|
||||
setLocalPathTimestamp(dryRun, newItemPath, itemModifiedTime);
|
||||
} else {
|
||||
// Downloaded file does not match size or hash .. which is it?
|
||||
bool downloadValueMismatch = false;
|
||||
|
|
@ -3777,7 +3777,7 @@ class SyncEngine {
|
|||
|
||||
// Whilst the download integrity checks were disabled, we still have to set the correct timestamp on the file
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, newItemPath, itemModifiedTime);
|
||||
setLocalPathTimestamp(dryRun, newItemPath, itemModifiedTime);
|
||||
|
||||
// Azure Information Protection (AIP) protected files potentially have missing data and/or inconsistent data
|
||||
if (appConfig.accountType != "personal") {
|
||||
|
|
@ -4004,7 +4004,7 @@ class SyncEngine {
|
|||
// The source of the out-of-date timestamp was the local item and needs to be corrected ... but why is it newer - indexing application potentially changing the timestamp ?
|
||||
if (verboseLogging) {addLogEntry("The source of the incorrect timestamp was the local file - correcting timestamp locally due to --resync", ["verbose"]);}
|
||||
// Fix the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, path, item.mtime);
|
||||
setLocalPathTimestamp(dryRun, path, item.mtime);
|
||||
} else {
|
||||
// The source of the out-of-date timestamp was OneDrive and this needs to be corrected to avoid always generating a hash test if timestamp is different
|
||||
if (verboseLogging) {addLogEntry("The source of the incorrect timestamp was OneDrive online - correcting timestamp online", ["verbose"]);}
|
||||
|
|
@ -4022,14 +4022,14 @@ class SyncEngine {
|
|||
// --download-only is being used ... local file needs to be corrected ... but why is it newer - indexing application potentially changing the timestamp ?
|
||||
if (verboseLogging) {addLogEntry("The source of the incorrect timestamp was the local file - correcting timestamp locally due to --download-only", ["verbose"]);}
|
||||
// Fix the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, path, item.mtime);
|
||||
setLocalPathTimestamp(dryRun, path, item.mtime);
|
||||
}
|
||||
} else if (!dryRun) {
|
||||
// The source of the out-of-date timestamp was the local file and this needs to be corrected to avoid always generating a hash test if timestamp is different
|
||||
if (verboseLogging) {addLogEntry("The source of the incorrect timestamp was the local file - correcting timestamp locally", ["verbose"]);}
|
||||
|
||||
// Fix the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, path, item.mtime);
|
||||
setLocalPathTimestamp(dryRun, path, item.mtime);
|
||||
}
|
||||
|
||||
// Display function processing time if configured to do so
|
||||
|
|
@ -5042,14 +5042,14 @@ class SyncEngine {
|
|||
if (verboseLogging) {addLogEntry("The local item has the same hash value as the item online, however file is a OneDrive Business Shared File - correcting local timestamp", ["verbose"]);}
|
||||
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, localFilePath, dbItem.mtime);
|
||||
setLocalPathTimestamp(dryRun, localFilePath, dbItem.mtime);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// --download-only being used
|
||||
if (verboseLogging) {addLogEntry("The local item has the same hash value as the item online - correcting local timestamp due to --download-only being used to ensure local file matches timestamp online", ["verbose"]);}
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, localFilePath, dbItem.mtime);
|
||||
setLocalPathTimestamp(dryRun, localFilePath, dbItem.mtime);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
|
|
@ -6340,21 +6340,56 @@ class SyncEngine {
|
|||
// Update the date / time of the file online to match the local item
|
||||
// Get the local file last modified time
|
||||
SysTime localModifiedTime = timeLastModified(localFilePath).toUTC();
|
||||
// Drop fractional seconds for upload timestamp modification as Microsoft OneDrive does not support fractional seconds
|
||||
localModifiedTime.fracSecs = Duration.zero;
|
||||
|
||||
// Get the latest eTag, and use that
|
||||
string etagFromUploadResponse = uploadResponse["eTag"].str;
|
||||
// Attempt to update the online date time stamp based on our local data
|
||||
|
||||
// Attempt to update the online lastModifiedDateTime value based on our local timestamp data
|
||||
if (appConfig.accountType == "personal") {
|
||||
// Business | SharePoint we used a session to upload the data, thus, local timestamps are given when the session is created
|
||||
uploadLastModifiedTime(dbItem, targetDriveId, targetItemId, localModifiedTime, etagFromUploadResponse);
|
||||
// Personal Account Handling for Modified File Upload
|
||||
//
|
||||
// Did the upload integrity check pass or fail?
|
||||
if (!uploadIntegrityPassed) {
|
||||
// upload integrity check failed for the modified file
|
||||
if (!appConfig.getValueBool("create_new_file_version")) {
|
||||
// warn that file differences will exist online
|
||||
// as this is a 'personal' account .. we have no idea / reason potentially, so do not download the 'online' file
|
||||
addLogEntry("WARNING: The file uploaded to Microsoft OneDrive does not match your local version. Data loss may occur.");
|
||||
} else {
|
||||
// Create a new online version of the file by updating the online metadata
|
||||
uploadLastModifiedTime(dbItem, targetDriveId, targetItemId, localModifiedTime, etagFromUploadResponse);
|
||||
}
|
||||
} else {
|
||||
// Upload of the modified file passed integrity checks
|
||||
// We need to make sure that the local file on disk has this timestamp from this JSON, otherwise on the next application run:
|
||||
// The last modified timestamp has changed however the file content has not changed
|
||||
// The local item has the same hash value as the item online - correcting timestamp online
|
||||
// This then creates another version online which we do not want to do .. unless configured to do so
|
||||
if (!appConfig.getValueBool("create_new_file_version")) {
|
||||
// Create an applicable DB item from the upload JSON response
|
||||
Item onlineItem;
|
||||
onlineItem = makeItem(uploadResponse);
|
||||
// Correct the local file timestamp to avoid creating a new version online
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setLocalPathTimestamp(dryRun, localFilePath, onlineItem.mtime);
|
||||
} else {
|
||||
// Create a new online version of the file by updating the metadata, which negates the need to download the file
|
||||
uploadLastModifiedTime(dbItem, targetDriveId, targetItemId, localModifiedTime, etagFromUploadResponse);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
// Business | SharePoint Account Handling for Modified File Upload
|
||||
//
|
||||
// Due to https://github.com/OneDrive/onedrive-api-docs/issues/935 Microsoft modifies all PDF, MS Office & HTML files with added XML content. It is a 'feature' of SharePoint.
|
||||
// This means that the file which was uploaded, is potentially no longer the file we have locally
|
||||
// There are 2 ways to solve this:
|
||||
// 1. Download the modified file immediately after upload as per v2.4.x (default)
|
||||
// 2. Create a new online version of the file, which then contributes to the users 'quota'
|
||||
// Did the upload integrity check pass or fail?
|
||||
if (!uploadIntegrityPassed) {
|
||||
// upload integrity check failed
|
||||
// upload integrity check failed for the modified file
|
||||
if (!appConfig.getValueBool("create_new_file_version")) {
|
||||
// are we in an --upload-only scenario
|
||||
if(!uploadOnly){
|
||||
|
|
@ -6380,12 +6415,12 @@ class SyncEngine {
|
|||
// The local item has the same hash value as the item online - correcting timestamp online
|
||||
// This then creates another version online which we do not want to do .. unless configured to do so
|
||||
if (!appConfig.getValueBool("create_new_file_version")) {
|
||||
// create an applicable item
|
||||
// Create an applicable DB item from the upload JSON response
|
||||
Item onlineItem;
|
||||
onlineItem = makeItem(uploadResponse);
|
||||
// Correct the local file timestamp to avoid creating a new version online
|
||||
// Set the timestamp, logging and error handling done within function
|
||||
setPathTimestamp(dryRun, localFilePath, onlineItem.mtime);
|
||||
setLocalPathTimestamp(dryRun, localFilePath, onlineItem.mtime);
|
||||
} else {
|
||||
// Create a new online version of the file by updating the metadata, which negates the need to download the file
|
||||
uploadLastModifiedTime(dbItem, targetDriveId, targetItemId, localModifiedTime, etagFromUploadResponse);
|
||||
|
|
@ -6511,8 +6546,20 @@ class SyncEngine {
|
|||
|
||||
// What upload method should be used?
|
||||
if (thisFileSizeLocal <= sessionThresholdFileSize) {
|
||||
// file size is below session threshold
|
||||
useSimpleUpload = true;
|
||||
}
|
||||
|
||||
// Use Session Upload regardless
|
||||
if (appConfig.getValueBool("force_session_upload")) {
|
||||
|
||||
// forcing session upload
|
||||
addLogEntry("FORCING SESSION UPLOAD (MODIFIED)");
|
||||
|
||||
useSimpleUpload = false;
|
||||
|
||||
}
|
||||
|
||||
|
||||
// If the filesize is greater than zero , and we have valid 'latest' online data is the online file matching what we think is in the database?
|
||||
if ((thisFileSizeLocal > 0) && (currentOnlineJSONData.type() == JSONType.object)) {
|
||||
|
|
@ -8539,10 +8586,22 @@ class SyncEngine {
|
|||
// Not a dry-run situation
|
||||
// Do we use simpleUpload or create an upload session?
|
||||
bool useSimpleUpload = false;
|
||||
|
||||
// What upload method should be used?
|
||||
if (thisFileSize <= sessionThresholdFileSize) {
|
||||
useSimpleUpload = true;
|
||||
}
|
||||
|
||||
// Use Session Upload regardless
|
||||
if (appConfig.getValueBool("force_session_upload")) {
|
||||
|
||||
// forcing session upload
|
||||
addLogEntry("FORCING SESSION UPLOAD (NEWFILE)");
|
||||
|
||||
useSimpleUpload = false;
|
||||
|
||||
}
|
||||
|
||||
// We can only upload zero size files via simpleFileUpload regardless of account type
|
||||
// Reference: https://github.com/OneDrive/onedrive-api-docs/issues/53
|
||||
// Additionally, only where file size is < 4MB should be uploaded by simpleUpload - everything else should use a session to upload
|
||||
|
|
|
|||
72
src/util.d
72
src/util.d
|
|
@ -1657,28 +1657,86 @@ void checkOpenSSLVersion() {
|
|||
}
|
||||
|
||||
// Set the timestamp of the provided path to ensure this is done in a consistent manner
|
||||
void setPathTimestamp(bool dryRun, string inputPath, SysTime newTimeStamp) {
|
||||
void setLocalPathTimestamp(bool dryRun, string inputPath, SysTime newTimeStamp) {
|
||||
|
||||
SysTime updatedModificationTime;
|
||||
bool makeTimestampChange = false;
|
||||
|
||||
// Try and set the local path timestamp, catch filesystem error
|
||||
try {
|
||||
// Set the correct time on the requested inputPath
|
||||
if (!dryRun) {
|
||||
if (debugLogging) {
|
||||
addLogEntry("Setting 'lastAccessTime' and 'lastModificationTime' properties for: " ~ inputPath ~ " to " ~ to!string(newTimeStamp), ["debug"]);
|
||||
// Generate the initial log message
|
||||
string logMessage = format("Setting 'lastAccessTime' and 'lastModificationTime' properties for: %s to %s if required", inputPath, to!string(newTimeStamp));
|
||||
addLogEntry(logMessage, ["debug"]);
|
||||
}
|
||||
|
||||
// Obtain the existing timestamp values
|
||||
SysTime existingAccessTime;
|
||||
SysTime existingModificationTime;
|
||||
getTimes(inputPath, existingAccessTime, existingModificationTime);
|
||||
|
||||
if (debugLogging) {
|
||||
addLogEntry("Existing timestamp values:", ["debug"]);
|
||||
addLogEntry(" Access Time: " ~ to!string(existingAccessTime), ["debug"]);
|
||||
addLogEntry(" Modification Time:" ~ to!string(existingModificationTime), ["debug"]);
|
||||
}
|
||||
|
||||
// Compare the requested new modified timestamp to existing local modified timestamp
|
||||
SysTime newTimeStampZeroFracSec = newTimeStamp;
|
||||
SysTime existingTimeStampZeroFracSec = existingModificationTime;
|
||||
newTimeStampZeroFracSec = newTimeStampZeroFracSec.toUTC();
|
||||
existingTimeStampZeroFracSec = newTimeStampZeroFracSec.toUTC();
|
||||
newTimeStampZeroFracSec.fracSecs = Duration.zero;
|
||||
existingTimeStampZeroFracSec.fracSecs = Duration.zero;
|
||||
|
||||
if (debugLogging) {
|
||||
addLogEntry("Comparison timestamp values:", ["debug"]);
|
||||
addLogEntry(" newTimeStampZeroFracSec = " ~ to!string(newTimeStampZeroFracSec), ["debug"]);
|
||||
addLogEntry(" existingTimeStampZeroFracSec = " ~ to!string(existingTimeStampZeroFracSec), ["debug"]);
|
||||
}
|
||||
|
||||
// Perform the comparison of the fracsec truncated timestamps
|
||||
if (newTimeStampZeroFracSec == existingTimeStampZeroFracSec) {
|
||||
if (debugLogging) {addLogEntry("Fractional seconds only difference in modification time; preserving existing modification time", ["debug"]);}
|
||||
updatedModificationTime = existingModificationTime;
|
||||
} else {
|
||||
if (debugLogging) {addLogEntry("New timestamp is different to existing timestamp; using new modification time", ["debug"]);}
|
||||
updatedModificationTime = newTimeStamp;
|
||||
makeTimestampChange = true;
|
||||
}
|
||||
|
||||
// Make the timestamp change for the path provided
|
||||
try {
|
||||
// Function detailed here: https://dlang.org/library/std/file/set_times.html
|
||||
// setTimes(path, accessTime, modificationTime)
|
||||
// We use the provided 'newTimeStamp' to set both:
|
||||
// accessTime Time the file/folder was last accessed.
|
||||
// We use the provided 'updatedModificationTime' to set modificationTime:
|
||||
// modificationTime Time the file/folder was last modified.
|
||||
if (debugLogging) {addLogEntry("Calling setTimes() for the given path", ["debug"]);}
|
||||
setTimes(inputPath, newTimeStamp, newTimeStamp);
|
||||
if (debugLogging) {addLogEntry("Timestamp updated for this path: " ~ inputPath, ["debug"]);}
|
||||
// We use the existing 'existingAccessTime' to set accessTime:
|
||||
// accessTime Time the file/folder was last accessed.
|
||||
if (makeTimestampChange) {
|
||||
// new timestamp is different
|
||||
if (debugLogging) {addLogEntry("Calling setTimes() for the given path", ["debug"]);}
|
||||
setTimes(inputPath, existingAccessTime, updatedModificationTime);
|
||||
if (debugLogging) {addLogEntry("Timestamp updated for this path: " ~ inputPath, ["debug"]);}
|
||||
} else {
|
||||
if (debugLogging) {addLogEntry("No local timestamp change required", ["debug"]);}
|
||||
}
|
||||
} catch (FileException e) {
|
||||
// display the error message
|
||||
displayFileSystemErrorMessage(e.msg, getFunctionName!({}));
|
||||
}
|
||||
|
||||
SysTime newAccessTime;
|
||||
SysTime newModificationTime;
|
||||
getTimes(inputPath, newAccessTime, newModificationTime);
|
||||
|
||||
if (debugLogging) {
|
||||
addLogEntry("Current timestamp values post any change (if required):", ["debug"]);
|
||||
addLogEntry(" Access Time: " ~ to!string(newAccessTime), ["debug"]);
|
||||
addLogEntry(" Modification Time:" ~ to!string(newModificationTime), ["debug"]);
|
||||
}
|
||||
}
|
||||
} catch (FileException e) {
|
||||
// display the error message
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue