Commit graph

78 commits

Author SHA1 Message Date
Nicola Murino c8158e14e0
move SFTPGo package to the internal folder
SFTPGo is a daemon and command line tool, not a library.

The public API are provided by the SDK

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-07-24 16:18:54 +02:00
Nicola Murino 21682d1c1d
add license header to source files
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-07-17 20:16:00 +02:00
Nicola Murino 55b47cf741
sftp realpath: resolve symlinks
Fixes #890

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-07-17 16:02:45 +02:00
Nicola Murino a5e41c9336
S3: allow empty region
the region may be embedded within the endpoint for some S3 compatible
object storage

Fixes #884

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-06-20 19:55:01 +02:00
Nicola Murino 93ce593ed0
squash database migrations and remove the credentials_path setting
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-06-13 20:08:49 +02:00
Nicola Murino 7ab30099dd
add httpfs
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-06-11 10:41:34 +02:00
Nicola Murino 796ea1dde9
allow to store temporary sessions within the data provider
so we can persist password reset codes, OIDC auth sessions and tokens.
These features will also work in multi-node setups without sicky
sessions now

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-05-19 19:49:51 +02:00
Nicola Murino ecd488a840
data provider: remove prefer_database_credentials
Google Cloud Storage credentials are now always stored within the data
provider.

Added a migration to read credentials from disk and store them inside the
data provider.

After v2.3 we can also remove credentials_path

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-04-28 12:55:01 +02:00
Nicola Murino 77f3400161
allow to mount virtual folders on root (/) path
Fixes #783

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-04-02 18:32:46 +02:00
Nicola Murino ca32cd5e0e
allow placeholders for add/update users and folders
remove session token for S3, a temporary token is useless for our usage

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-03-27 16:32:21 +02:00
Nicola Murino 5c2fd8d52a
add support for a start directory
Fixes #705

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-03-03 12:44:56 +01:00
Nicola Murino 4519bffa39
S3: add support for assume role
Fixes #736

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-28 20:19:13 +01:00
Nicola Murino e18ad55067
S3: add support for session tokens
Fixes #736

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-25 15:30:04 +01:00
Nicola Murino f5a0559be6
don't execute fs check if the user has recent activity
The check could be expensive with some backends and is generally
only required the first time that a user logs in

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-24 16:11:35 +01:00
Nicola Murino 8bbf54d2b6
azure blobs: add support for multipart downloads
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-21 19:01:31 +01:00
Nicola Murino cd35636939
S3: add a timeout for single part uploads
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-02-01 12:15:56 +01:00
Nicola Murino 1f619d5ea6
make the sdk a separate module
The SFTPGo SDK now is at the following URL

https://github.com/sftpgo/sdk

Fixes #657

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-06 11:54:43 +01:00
Nicola Murino 6d3d94a01f
move kms implementation outside the sdk package
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-06 10:11:47 +01:00
Nicola Murino 7c68b03d07
move plugin handling outside the sdk package
Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-05 11:37:45 +01:00
Nicola Murino a6fe802370
move kms definitions to the sdk package
This is the first step to make the sdk a separate module

Signed-off-by: Nicola Murino <nicola.murino@gmail.com>
2022-01-04 12:49:30 +01:00
Nicola Murino a587228cf0
add support for metadata plugins 2021-12-16 18:18:36 +01:00
Nicola Murino 24b0352eb6
GCS: add ACL support 2021-11-15 21:57:41 +01:00
Nicola Murino 52f3a98cc8
preserve GCS credentials on update if not set
credentials were not preserved if "prefer_database_credentials" was
set to true

Fixes #613
2021-11-15 19:12:58 +01:00
Nicola Murino ee5c5e033d
S3: add ACL support
Fixes #610
2021-11-13 16:05:40 +01:00
Nicola Murino 5c34d814d6
fix a possible nil pointer dereference
it can happen by upgrading from very old versions
2021-09-11 14:19:17 +02:00
Nicola Murino 85a47810ff
S3: expose more properties, possible backward incompatible change
Before these changes we implictly set S3ForcePathStyle if an endpoint
was provided.

This can cause issues with some S3 compatible object storages and must
be explicitly set now.

AWS is also deprecating this setting

https://aws.amazon.com/it/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/
2021-07-23 16:56:48 +02:00
Nicola Murino bd5191dfc5
add experimental plugin system 2021-07-11 15:26:51 +02:00
Nicola Murino 076b2f0ee0
modules: add v2 support 2021-06-26 07:31:41 +02:00
Nicola Murino 9d3d7db29c
azblob: store SAS URL as kms.Secret 2021-06-11 22:27:36 +02:00
Nicola Murino 3b46e6a6fb
add support for a global temp path
Fixes #436
2021-05-27 15:38:27 +02:00
Nicola Murino ea26d7786c
sftpfs: add buffering support
this way we improve performance over high latency networks
2021-04-03 16:00:55 +02:00
Nicola Murino 2f56375121
improve SFTP loop detection 2021-04-01 18:53:48 +02:00
Nicola Murino 9ad750da54
WebDAV: try to preserve the lock fs as much as possible 2021-03-27 19:10:27 +01:00
Nicola Murino d6dc3a507e
extend virtual folders support to all storage backends
Fixes #241
2021-03-21 19:15:47 +01:00
Nicola Murino 46176a54b4
minor doc fixes 2021-02-14 22:08:08 +01:00
Nicola Murino 51f110bc7b
sftpd: add statvfs@openssh.com support 2021-02-11 19:45:52 +01:00
Nicola Murino 2a95d031ea
FTP: add support for AVBL command 2020-12-25 11:14:08 +01:00
Nicola Murino a6985075b9
add sftpfs storage backend
Fixes #224
2020-12-12 10:31:09 +01:00
Nicola Murino 4a88ea5c03
add Data At Rest Encryption support 2020-12-05 13:48:13 +01:00
Nicola Murino 634b723b5d
add KMS support
Fixes #226
2020-11-30 21:46:34 +01:00
Nicola Murino dccc583b5d
add a dedicated struct to store encrypted credentials
also gcs credentials are now encrypted, both on disk and inside the
provider.

Data provider is automatically migrated and load data will accept
old format too but you should upgrade to the new format to avoid future
issues
2020-11-22 21:53:04 +01:00
Nicola Murino ca0ff0d630
add a File interface so we can avoid to use os.File directly 2020-11-17 19:36:39 +01:00
Nicola Murino 1a6863f4b1
GCS uploads: check Close() error
some code simplification too
2020-11-13 18:40:18 +01:00
Nicola Murino 5720d40fee
add setstat_mode 2
in this mode chmod/chtimes/chown can be silently ignored only for cloud
based file systems

Fixes #223
2020-11-12 10:39:46 +01:00
Nicola Murino 0a14297b48
webdav: performance improvements and bug fixes
we need my custom golang/x/net/webdav fork for now

https://github.com/drakkan/net/tree/sftpgo
2020-11-04 19:11:40 +01:00
Nicola Murino fcfdd633f6
Azure Blob: update SDK and add access tier support 2020-10-30 22:17:17 +01:00
Nicola Murino 9b49f63a97
azure: implement multipart uploads using low level API
The high level wrapper seems to hang if there are network issues
2020-10-25 17:41:04 +01:00
Nicola Murino 14bcc6f2fc
s3, azblob: check upper limit for part size 2020-10-25 12:10:11 +01:00
Nicola Murino 5ff8f75917
add Azure Blob support 2020-10-25 08:18:48 +01:00
Sean Hildebrand db7e81e9d0
add prefer_database_credentials configuration parameter
When true, users' Google Cloud Storage credentials will be written to
the data provider instead of disk.
Pre-existing credentials on disk will be used as a fallback

Fixes #201
2020-10-22 10:42:40 +02:00