Commit graph

92 commits

Author SHA1 Message Date
Fabian d5c9b29942 Upgrade jshint 2020-12-31 19:14:29 -06:00
Fabian b0a34e7a86 fs: assert that sha256 is present 2020-12-31 19:14:29 -06:00
Ernest Wong 3dde97f14a Filesystem: Work on entire buffer when writing to file
The Uint8Array that stores the file data is usually larger than the file
size itself, to improve the amortized complexity of resizing the array.

When we previously switched over to using FileStorages, the method
fs#Write() was modified to retrieve only the subarray of the Uint8Array
that contains the file (i.e. excluding the unused padding at the end),
so appending to a file always tries to resize the array even though the
ArrayBuffer that backs it has enough space for the new data.
2020-08-30 19:37:15 -05:00
Ernest Wong 4651e28177 Fix type annotations for async functions 2020-08-30 19:37:15 -05:00
Ernest Wong ad16fb0656 Filesystem: Add missing awaits to async function calls 2020-08-30 19:37:15 -05:00
Ernest Wong 840415a052 Filestorage: (minor) Improve naming can_uncache() -> uncache() 2020-08-30 19:37:15 -05:00
Ernest Wong 2f4520619a Filesystem: Tidy naming of STATUS_ON_SERVER to ON_STORAGE 2020-08-30 19:37:15 -05:00
Ernest Wong c625bba498 Filestorage: Implement cache cleanup - avoid memory growth 2020-08-30 19:37:15 -05:00
Ernest Wong 62c880b415 Filestorage: Integrate chunking and revert unnecessary chunking
- Integrate new FileStorage interface with lib/filesystem.js
- Simplify the FileStorage interface with the intention that it only
serves read-only files.
2020-08-30 19:37:15 -05:00
Ernest Wong c029944965 Filesystem: Distinguish server file via inode status.
Also make file_storage a mandatory argument to FS.
2020-08-30 19:37:15 -05:00
Ernest Wong 6a06a7108a Tidy: file_storage snake case and interface naming convention 2020-08-30 19:37:15 -05:00
Fabian d2f86799e9 Simplify load_from_json: Accept json object directly, don't run asynchronously 2020-08-30 19:37:14 -05:00
Fabian eb93fa5db7 Filesystem: Temporarily cache inode data until storage is chunked properly 2020-08-30 19:37:14 -05:00
Fabian fe5a1f5322 Fix error case in read_dir/read_file 2020-08-30 19:37:14 -05:00
Fabian 028e131ddb Rename OnJSONLoaded & small refactor 2020-08-30 19:37:14 -05:00
Fabian 3f79814e05 Detect initrd/bzimage in 9p filesystem and load before boot 2020-08-30 19:37:14 -05:00
Fabian fbdb4a28ea Remove all uses of "for in" 2020-08-30 19:29:54 -05:00
Ernest Wong 44d54facb7 Filesystem: Remove remnants of file loader 2020-08-30 19:29:54 -05:00
Ernest Wong c1729b7daf Filesystem: Don't modify local copy of filedata after set_data
While the original code would've worked for MemoryFileStorage, it will
not work for FileStorages that clone the data before saving (e.g.
IndexedDBFileStorage) as local changes won't be reflected onto the
storage class.
2020-08-30 19:29:54 -05:00
Ernest Wong 0a67e3532f Filesystem: Introduce FileStorage class for server-loaded files 2020-08-30 19:29:54 -05:00
Ernest Wong 595d6dca55 Filesystem: Refactor inodedata accesses to async methods
The "// jshint ignore:line" comments are pretty messy, so squint your eyes.
They're systematically placed, so we can regex it out when jshint's
new version finally arrives.

Using async/await instead of callbacks due to callback hell, and it also
helps minimising the diff)
2020-08-30 19:29:54 -05:00
Ernest Wong 401b814f93 Filesystem: Store infinite lock lengths
Having both lock.length and lock.get_length() giving different values is
confusing.
2020-08-30 19:29:54 -05:00
Ernest Wong 2bac2198e8 Filesystem: Don't store lock length of Infinity
Infinity is not JSON.stringifiable and will turn into null during state
save and restore.
2020-08-30 19:29:54 -05:00
Ernest Wong 267083e897 Filesystem: Implement POSIX record locks 2020-08-30 19:29:54 -05:00
Ernest Wong 40ed4f8dde Filesystem: Simplify nlinks management across filesystems
Now that hard links aren't allowed across filesystems, it's ok to store
the relevant nlinks information within local inode.

For mountpoints (forwarder inodes that point to a root inode), it is
possible to bump the nlinks of the forwarder during link_under_dir and
unlink_from_dir so that this nlinks counter is modified independently
from the real inode's nlinks counter.

For divert(), it is actually ok to divert forwarder inodes that point to
hardlinked files, so I've relaxed the assertion predicate.
2020-08-30 19:29:54 -05:00
Ernest Wong 66cf97e0a7 Filesystem: Tidy comments, clarify nlinks special case. 2020-08-30 19:29:54 -05:00
Ernest Wong 6c61d48c8f Filesystem: Fix typo with path concatenation during Rename 2020-08-30 19:29:54 -05:00
Ernest Wong c7dda46de1 Filesystem: Disallow hardlinks across filesystems
Although hardlinks across filesystems worked quite well in the current
implementation, we will run into problems when we try to implement
different backends for each sub-filesystem.

Note: EPERM is used instead of EXDEV since the mv command will silently
try to use copy-and-unlink when rename(2) fails with EXDEV.

The rules for linking has been reverted back:
 - Before commit: Any inode, including forwarders, could be linked as
long as the parent is not a forwarder and is a directory.
 - After commit: Only non-forwarders and root-forwarders are allowed to
be linked, and must be linked under a directory and not a forwarder.
2020-08-30 19:29:54 -05:00
Ernest Wong 2560394855 Filesystem: set unlinked status on the real inode
The real_inode's status is supposedly set to STATUS_UNLINKED when there
are no more references to the inode from other directories. Previously,
this status was incorrectly applied to the forwarder and not the actual
real_inode.
2020-08-30 19:29:54 -05:00
Ernest Wong 0595c11740 Filesystem: fix nlinks counting for mountpoints 2020-08-30 19:29:54 -05:00
Ernest Wong 5aa5c88bf8 Filesystem state: restore onto existing configuration
Don't create new filesystems based off from state file.
Don't overwrite fs.baseurl
2020-08-30 19:29:54 -05:00
Ernest Wong d1519d4509 Filesystem: hard links, replace linked lists with Map.
Unfortunately, I didn't split this into multiple commits:

- Keeping track of nlinks is now all done by link_under_dir and
unlink_from_dir methods.

- Inodes are no longer associated with a name.

- Directory structure is now represented using a Map<name to idx>, so
the directory inode fully owns the directory entries instead a linked
list scattered across many inodes.

- The parameter order for FS#Link and FS#link_under_dir has been
modified to be consistent with 9p's: (dirid, fid, name)

- Allowed the movement of hardlinked files between filesystems
"vertically", as long as the target inode is reachable within the
filesystem and subfilesystems without ever having to traverse to the
parent filesystem.

- The methods FS#copy_inode and FS#divert does not put the filesystem in
an invalid state: FS#copy_inode does not modify inode.nlinks nor
inode.direntries, and FS#divert makes sure to delete the original nlinks
and direntries after it has been copied over to a new inode.

- Added extra assertions in FS#link_under_dir and FS#unlink_from_dir

- Forwarders should not be deleted upon unlink: the files are still
accessible.

- Non-root non-directory forwarders can now be linked under
non-forwarder directories for extra freedom for hardlinks. Now, the rule
for linking is: never link under a forwarder, and never link a directory
more than once.

Some tests and some 9p debug code is broken because they rely on
inode.name which no longer exists nor makes sense to exist.

Will fix in next commit.
2020-08-30 19:29:54 -05:00
Fabian 28ac891dc3 Filesystem: Load files via their sha256 2020-08-30 19:29:53 -05:00
Ernest Wong 6c55912a59 Fail early if unlink of destination failed for fs.Rename 2020-08-30 19:29:53 -05:00
Ernest Wong 7aa42c3418 Identify type of failure for fs.Unlink using -errno convention 2020-08-30 19:29:53 -05:00
Ernest Wong 3a2ff7c47e Fix: fs.Rename should configure the correct filename
Specifically when moving a file from a mounted filesystem to the local
filesystem.
2020-08-30 19:29:53 -05:00
Ernest Wong 53edf989ea Fix: fs.divert should update children parentids
Otherwise, calling fs.divert leaves the filesystem in an invalid state.
2020-08-30 19:29:53 -05:00
Ernest Wong e683e812fd Mount only onto non-existent paths, existent parent
If mountpoint already exists, then we're silently making its children
inaccessible which may not be what we expected/intended.

Create a new forwarder inode upon mounting.
2020-08-30 19:29:53 -05:00
Ernest Wong 3fe75a9130 Identify type of failure for fs.Rename using -errno convention 2020-08-30 19:29:53 -05:00
Ernest Wong c66731d55d Properly forward fs.Open/CloseInode 2020-08-30 19:29:53 -05:00
Ernest Wong 23080e2731 Isolate filesystem qid counting to each v86 instance 2020-08-30 19:29:53 -05:00
Ernest Wong 8cb6f23248 Fix filesystem mount state saving 2020-08-30 19:29:53 -05:00
Ernest Wong ff11e9c00f Minor: better rename-failure logs, comments, & unnecessary return 2020-08-30 19:29:53 -05:00
Ernest Wong a063962311 Handle forwarding and root inode in other api methods
Root inode dosn't have a parent. Methods should work fine with root
inodes too.
2020-08-30 19:29:53 -05:00
Ernest Wong 121b9c3b96 Incorperate mounted filesystems in statistics 2020-08-30 19:29:53 -05:00
Ernest Wong b6b6b8cada Implement Rename - moving files between filesystems 2020-08-30 19:29:53 -05:00
Ernest Wong d98ec5f37b Handle forwarding for our dummy fs.Link method 2020-08-30 19:29:53 -05:00
Ernest Wong 71cb654502 Partially handle forwarding for fs.FillDirectory
Unresolved bug:
Calling fs.FillDirectory on a mountpoint gives incorrect information.
".." entry should describe parent of mountpoint, but currently describes
itself.
2020-08-30 19:29:53 -05:00
Ernest Wong 687607569b Handle forwarding for open, close, read, and write 2020-08-30 19:29:53 -05:00
Ernest Wong 004182adc8 Handle forwarding for fs.Create* methods 2020-08-30 19:29:53 -05:00