Uses MemoryFileStorage as a fallback when:
- v86 is used in a browserless environment, e.g. NodeJS
- browser doesn't support indexedDB
- opening indexedDB fails
- existing database is a newer version
- existing database is an old version and is under use - blocking
current v86 instance to connect to it.
The "// jshint ignore:line" comments are pretty messy, so squint your eyes.
They're systematically placed, so we can regex it out when jshint's
new version finally arrives.
Using async/await instead of callbacks due to callback hell, and it also
helps minimising the diff)
This commit contains the final changes requires for porting all C code
to Rust and from emscripten to llvm:
- tools/wasm-patch-indirect-function-table.js: A script that rewrites
the wasm generated by llvm to remove the table limit
- tools/rust-lld-wrapper: A wrapper around rust-lld that removes
arguments forced by rustc that break compilation for us
- src/rust/cpu2/Makefile: A monstrosity to postprocess c2rust's output
- gen/generate_interpreter.js: Ported to produce Rust instead of C
- src/rust/*: A few functions and macros to connect the old Rust code
and the new Rust code
- src/*.js: Removes the loading of the old emscripten wasm module and
adapts imports and exports from emscripten to llvm
If mountpoint already exists, then we're silently making its children
inaccessible which may not be what we expected/intended.
Create a new forwarder inode upon mounting.
The testing "framework" code is slowly turning into spaghetti due to the
asynchronous nature of the triggers. Using async functions will help
clarify the program flow if we think we should address this issue.
The following files and functions were ported:
- jit.c
- codegen.c
- _jit functions in instructions*.c and misc_instr.c
- generate_{analyzer,jit}.js (produces Rust code)
- jit_* from cpu.c
And the following data structures:
- hot_code_addresses
- wasm_table_index_free_list
- entry_points
- jit_cache_array
- page_first_jit_cache_entry
Other miscellaneous changes:
- Page is an abstract type
- Addresses, locals and bitflags are unsigned
- Make the number of entry points a growable type
- Avoid use of global state wherever possible
- Delete string packing
- Make CachedStateFlags abstract
- Make AnalysisType product type
- Make BasicBlockType product type
- Restore opcode assertion
- Set opt-level=2 in debug mode (for test performance)
- Delete JIT_ALWAYS instrumentation (now possible via api)
- Refactor generate_analyzer.js
- Refactor generate_jit.js
- Use this data structure to delete cached code immediately when page is
written, not later when wasm index is reused
- Remove "dirty page" data structure
- Simplify cycle_internal, as no entries can be found dirty, they are
removed immediately after being overwritten
This commit consists of three components:
1. A new generated x86-parser that analyses instructions. For now, it
only detects the control flow of an instruction: Whether it is a
(conditional) jump, a normal instruction or a basic block boundary
2. A new function, jit_find_basic_blocks, that finds and connects basic
blocks using 1. It loosely finds all basic blocks making up a function,
i.e. it doesn't follow call or return instructions (but it does follow
all near jumps). Different from our previous analysis, it also finds
basic blocks in the strict sense that no basic block contains a jump
into the middle of another basic block
3. A new code-generating function, jit_generate, that takes the output
of 2 as input. It generates a state machine:
- Each basic block becomes a case block in a switch-table
- Each basic block ends with setting a state variable for the following basic block
- The switch-table is inside a while(true) loop, which is terminated
by return statements in basic blocks which are leaves
Additionally:
- Block linking has been removed as it is (mostly) obsoleted by these
changes. It may later be reactived for call instructions
- The code generator API has been extended to generate the code for the state machine
- The iterations of the state machine are limited in order to avoid
infinite loops that can't be interrupted
According to Fabian: safe_write128 is called suprisingly often as it is used by
Linux to fill the frame buffer. We can do two optimisations here: Add write128
to avoid one in_mapped_range check. Add mmap_write128 (taking 4 32-bit integers)
to avoid several switches from wasm to js and lookups of
memory_map_write32[aligned_addr]
(we can assume that writes don't cross a 16-byte boundary)
- Don't pass the old entry to jit_generate, instead look it up in
create_cache_entry (negligible performance overhead)
- Don't call the generated code immediate, this is a pre-requisite for
asynchronous compilation. This also required disabling the
self-modifying code assertion
- Don't pass codegen_finalize the start address to codegen_finalize,
instead set it in jit_generate
- Moved all helper functions to coverage.js
- Refactor individual cov_*[func_id] objects to coverage[func_id].*
- Write coverage data to its own directory (./build/coverage/coverage_data*)
- Enable/disable coverage logging in do_many_cycles to account for exceptions
- Better naming
- Minor stylistic refactoring
Notes:
- The coverage_dump_loop doesn't really help much, but it helps keep the memory
usage a bit down for longer running tests (eg. booting Linux) - Linux still
slows down too much and can't callibrate its time to boot
- Dumping the incoming data without structuring by file results in very large
files (~150-300 mb for the nasm tests). Structuring by fn_id/fn_name slows
execution down, but allows for more manageable coverage data
- Buffering data in memory and writing to disk synchronously is faster / about
the same speed as async or buffering + async IO.
restore memcpy comment
delete all the things!
fix jshint issues
restore memcpy comment
remove duplicate fxsave assignment
Count cache drops
Use already available physical address instead of calling read_imm8
Remove useless assertion
Just move around to reduce later diff
Run jit paging test with assertions enabled
Run jit-paging test on CI
Extend jit-paging test
Fix deleting invalidated code across memory pages
Add jit-paging test to gitlab ci
Remove jit_in_progress
Clean up old comments, use bool for jit_jump
Fix state image not begin garbage collected
Add ENABLE_PROFILER_TIMES to configure slow profiling times
Move to jit_generate and jit_run_interpreted to separate function
Add missing struct field
Fix: Don't write jit cache entry until no more faults can happen
Download image for jit paging test
Add missing initialiser
Mark jit_{generate,run_interpreted} as static
Specify full path to profiler.h
Clean up duplicate/missing declaration after rebase
mmap error handling, line length and fix some warnings
remove further unused code
move js imports to single header file
2e469796 Minor
fab422ef Improve generation of 0f instructions
08ad7fe9 Improved if-else generation
3f81014d Minor: Align test output
4a3a84ef Generate modrm tests
61aa1875 Simplify
a6e47954 Generate decoding of immediate operands
435b2c10 Fix warnings
e4933042 Add missing immediate operand
3f3810c7 Generate immediate operands for instructions with modrm byte
a0aa7b1f Make memory layout in nasm tests clearer
6b8ef212 Remove 'g' property from instruction table (implied by 'e')
bf15c58c Remove unused declarations
1e543035 Remove useless `| 0` and `>>> 0` javascriptisms
1ccc5d53 Fix headers
8b40c532 Update qemu tests with changes from qemu
ec9b0fb5 Port xchg instructions to C
c73613e7 Port virt_boundary_* to C
d61d1241 Add headers
fd19f22c Make written value in write8 and write16 int32_t
497dcaec Generate read_imm for instructions with a modrm byte
8b7003d6 Generate read_imm8s
0cc75498 Remove read_op
9d716086 Trigger unimplemented_sse for partial sse instructions with prefix
8d5edd03 Remove unimplmented sse c-to-js hack
585d3565 Remove | 0
308124b2 Use int32_t as return value
f193f8e1 Use JS version of cvttsd2si for now
12747b97 Generate trigger_ud for missing modrm branches
770f674e Split 0f00 and 0f01 into multiple instructions depending on modrm bits
1cb372a3 Generate decoder for some 0f-prefix instructions
cec7bc63 Disable unused parameter warnings in instruction functions
807665b1 Generate read_imm for 0f/jmpcc
cdf6eccc Generate modrm decoding for shld
04528429 Create temporary files in /tmp/, not cwd
d8f3fbd8 Generate modrm/imm decoding for shld
00ef0942 Generate modrm decoding for bts
f531984b Generate modrm decoding for shrd and imul
07569c53 Generate modrm decoding cmpxchg
535ff190 Generate modrm decoding for lfs/lgs/lss
2f8ced8d Generate modrm decoding for btr and btc
95de6c66 Generate modrm decoding for movzx
c4d07e7e Generate modrm decoding for bsf and bsr
f0985d26 Generate modrm decoding for movsx
4b30937a Generate modrm decoding for xadd
a422eb27 Generate modrm decoding for movnti
e5501d3c Generate modrm decoding for mov to/from control/debug registers
bce11ec5 Generate modrm decoding for lar/lsl
5729a23c Fix access to DR4 and DR5 when cr4.DE is clear
44269a81 Specify immediate size explicitly instead of inferring it
82b2867a Fix STR instruction
98a9cc89 Log failing assertion
6d2f9964 Fix rdtsc
00260694 Log GP exceptions
7916883d Port trigger_ud and trigger_nm to C
36fedae9 Remove unused code
e08fabd0 Generate modrm decoding for 0f00 and 0f01
8ae8174d Generate modrm decoding for 0fae and 0fc7 (fxsave, cmpxchg8, etc.)
26168164 Generate modrm+immediate decoding for 0fba (bit test with immediate operand)
6adf7fa7 Simplify create_tests.js (unused prefix call)
c77cbdd8 Add comments about the implementation of pop [addr]
4640b4fe Simplify prefix call
a81a5497 Don't use var
3ca5d13d Separate call name and arguments in code generator
3191a543 Simplify other prefix call (8D/lea)
5185080e Update generated code (stylistic changes and #ud generation)
93b51d41 Remove unused wasm externals
e4af0a7f Avoid hardcoding special cases in code generator (lea, pop r/m)
654a2b49 Avoid hardcoding special cases in code generator (enter/jumpf/callf)
fd1a1e86 Commit generated code (only stylistic changes)
7310fd1a Simplify code generator by merging code for with and without 0f prefix
e7eae4af Simplify code generator by merging code for immediate operands
00fafd8a Improve assertions
db084e49 Simplify code generator (modrm if-else)
0a0e4c9e Improve code generation of switch-case
ce292795 Clarify some comments
37cf33fa Generate code in if/else blocks
cbcc33fc Document naming scheme
e30b97eb Generate modrm decoding for 0f12 (sse) instruction
24b72c2f movlpd cannot be used for register-to-register moves
72d72995 Generate modrm decoding for 0f13 (sse) instruction and disable register-to-register moving
75d76fbb Generate modrm decoding for 0f14 (sse) instruction
ac8965a7 Generate modrm decoding for 0f28-0f2b (movap, movntp)
e919d33e Generate modrm decoding for cvttsd2si
5f2ca2b4 Generate modrm decoding for andp and xorp
c8d1c6de Generate modrm decoding for 0f60-0f70 (sse instructions)
ae4ed46d Add multi-byte nop and prefetch to nasm test, generate modrm decoding
718a1acf Print qemu test error message more useful
d1ecc37e Generate modrm decoding for 0f70-0f80 (sse instructions)
6a7219a5 Generate modrm decoding for popcnt
25278217 Generate modrm decoding for 0f71-0f73 (sse shift with immediate byte)
ed1ec81b Generate modrm decoding for the remaining sse instructions (0fc0-0fff)
42bc5a6f Use 64-bit multiplication for native code
dda3fb39 Remove old modrm-decoding functions
717975ef Move register access functions to cpu.c
aee8138f Remove read_op, read_sib, read_op0F, read_disp
f31317f2 Rename xmm/mmx register access functions
a525e70b Remove 32-bit access to reg_xmm and reg_mmx
c803eabc Rename s8/s16/s32 to i8/i16/i32
9fbd2ddf Don't use uninitialised structs
942eb4f7 Use 64-bit load for mmx registers and assert reg64 and reg128 size
f94ec612 Use 64-bit writes for write_xmm64
08022de9 Use more efficient method for some 128-bit stores
9d5b084c Make timestamp counter unsigned
2ef388b3 Pass 64-bit value to safe_write64
4cb2b1be Optimise safe_write64 and safe_write128
b0ab09fb Implement psllq (660ff3)
9935e5d4 Optimise safe_read64s and safe_read128s
af9ea1cc Log cl in cpuid only if relevant
be5fe23e Add multi-op cache (disabled by default through ENABLE_JIT macro) and JIT paging test (similar to QEMU test).
aa2f286e Don't initialise group_dirtiness with 1 as it increases the binary size significantly
b8e14ed9 Remove unused reg_xmm32s
bc726e03 Implement dbg_log for native code with format characters 'd' and 'x'
454039d6 Fix store task register
63a4b349 Remove unnecessary parens and clean up some log statements
4cc96814 Add logop and dbg_trace imports
7940655d Only inhibit interrupts if the interrupt flag was 0 in STI
876c68a7 Split create_tests into create_tests and generate_interpreter
aa82499f Move detection of string instructions to x86_table
f3840ec2 Move C ast to separate file
90400703 Skip tests for lfence/mfence/sfence, clarify their encoding
4a9d8204 elf: Hide log messages when log level is zero
a601c526 Allow setting log level via settings
8a624453 Add cpu_exception_hook to debug builds
f9e335bf Nasm: Test exceptions
599ad088 logop: Format instruction pointer as unsigned
f95cf22b Don't skip zero dividing tests
2a655a0e Remove get_seg_prefix_ds from read_moffs (preparation for calling read_moffs from the code generator)
bc580b71 Remove obsolete comment
e556cee0 Fix nasmtest dependencies in makefile and clean
dcb1e72b Use all cores on travis
86efa737 Replace all instances of u32 & 0xFFFF with the respective u16 accesses
98b9f439 Use u8 instead of bit-shifts and masks from u32
b43f6569 Replace all instances of u32 >> 16 with the respective u16 accesses
9bfa72c7 Remove unnecessary parens
9cf93734 Clean up remaining instance of u32 with a mask instead of u16
22d4117f Correct order of writes in virt_boundary_write32
6734c7c1 Fix keyboard on ios, fixes#105
858a4506 Add missing file, c_ast.js
1d62e39e Move instruction naming scheme into function
f4816852 Reorder some code
69d49788 Minor improvements
0493e05f Add util.js
af9000c1 Improve full test
e5feba31 Add missing export
c7c42065 Replace prefix_call with custom_resolve_modrm
3186e6ad Add support for "%%" format string to dbg_log_wasm for printf import
efe54fad Add barebones instrumentation profiler (disabled by default).
c9f0d462 Implement movlps m64, xmm and enable its test
42869a12 Add tests for cross-page reads/writes confirmed with byte reads/writes
d68976ea Mask word values in port byte reads
9758d51e Add PS2_LOG_VERBOSE
5f52f037 Update NASM Makefile to include all dependencies to prevent unnecessary recompilation
2c71f927 Have NASM test generator use a seedable PRNG to allow for faster incremental tests
e4aa45bb Add chunk{16,32}_rw paging tests; instructions that read and write to memory
bdf538a2 add codegen to cpu constructor
aa76ce8e add resolve_modrm16
14d7ecf1 refactor codegen
b710319f [rebased] Merge branch codegen
0565ea42 minor refactoring
071dff3f temporary fix for automatic cast warnings
57c504f2 fix modrm16 issue
c2db5d9e jit modrm32
85c04245 reinstate modrm_fn0 and modrm_fn1
be65dafd add ip and previous ip manipulating functions
ae00ef89 update codegen js interface
530a74fa squashed commit for refactor
2c692199 add codegen-test to build
c15afe68 prefix gen to codegen api
c9611533 codegen tests fixes
