Reusing the naming scheme of
sha256sum
sha256sum-0
sha256sum-1
sha256sum-2
etc
but the extra metadata such as total file size and block count are no
longer needed in the first block.
IndexedDB uses the StructuredSerializeForStorage algorithm to store
records into the database.
It appears that the html5 spec defines the StructuredSerializeForStorage
algorithm in a way that, when we serialize a TypedArray, the associated
ArrayBuffer is also serialized. Meaning even if the TypedArray is a view
of a single byte in the ArrayBuffer, the potentially-megabytes of data
in the ArrayBuffer are also copied over.
This makes sense. After deserialization, you'd expect the TypedArray to
function the same way as before meaning it should have the same
`.byteOffset` and `.buffer` properties.
However, it is easy to overlook this and think that a TypedArray viewing
2 bytes will only have the 2 bytes stored in the database.
This commit fixes a source of crashes. When calling
IndexedDBFileStorage#set() with a 3Mb file, the file is to be stored as
approx 800 lots of 4k blocks. However, in reality, 800 lots of 3 Mb array
buffers are being stored into the database. As the transaction is not
committed until the last request has been placed, I suspect that Chrome was
trying to store those 2.4 Gb worth of data in memory and did not handle
this case properly, leading to out-of-memory.
This commit also fixes a source of poor performance. When calling
IndexedDBFileStorage#read() to read 4k of a 48 Mb file, the browser seems
to be deserializing the entire 48 Mb into memory and garbage collecting
it even though only 4k is needed.
- More like passing the same reference to it when calling db_get/set.
- In reality, it was still the same store object being accessed, but
this commit improves the coherence of the db_get/set helper methods.
- Integrate new FileStorage interface with lib/filesystem.js
- Simplify the FileStorage interface with the intention that it only
serves read-only files.
Simplifies both the filestorage.js code as well as the starter.js code
that uses these FileStorage classes.
Now, the ServerFileStorageWrapper decorates the other FileStorage
classes by loading from the server when the file is not available.
Moreover, the previous starter.js was incorrectly passing the `baseurl`
parameter (it was passing it when baseurl was not defined nor needed).
Fallback logic is moved to the caller's responsibility.
Empty FileStorages and FileStorages that load from the server are
separated into different classes. This is to avoid faults where the
caller of the constructor forgets to pass in a `baseurl` parameter and
leads to some confusing bug.
Uses MemoryFileStorage as a fallback when:
- v86 is used in a browserless environment, e.g. NodeJS
- browser doesn't support indexedDB
- opening indexedDB fails
- existing database is a newer version
- existing database is an old version and is under use - blocking
current v86 instance to connect to it.
