Makes the following a block boundary:
- push
- Any non-custom instruction that uses modrm encoding
- Any sse/fpu instruction
This commit affects performance negatively. In order to fix this, the
above instructions need to be implemented using custom code generators
for the memory access.
This commit makes the return type of most basic memory access primitives
Result, where the Err(()) case means a page fault happened, the
instruction should be aborted and execution should continue at the page
fault handler.
The following primites have a Result return type: safe_{read,write}*,
translate_address_*, read_imm*, writable_or_pagefault, get_phys_eip,
modrm_resolve, push*, pop*.
Any instruction needs to handle the page fault cases and abort
execution appropriately. The return_on_pagefault! macro has been
provided to get the same behaviour as the previously used JS exceptions
(local to the function).
Calls from JavaScript abort on a pagefault, except for
writable_or_pagefault, which returns a boolean. JS needs to check
before calling any function that may pagefault.
This commit does not yet pervasively apply return_on_pagefault!, this
will be added in the next commit.
Jitted code does not yet properly handle the new form of page faults,
this will be added in a later commit.
The generated rust code doesn't call read_imm* functions for custom
instructions now for the memory variant branches when both immediate
values and modrm byte is used
The following files and functions were ported:
- jit.c
- codegen.c
- _jit functions in instructions*.c and misc_instr.c
- generate_{analyzer,jit}.js (produces Rust code)
- jit_* from cpu.c
And the following data structures:
- hot_code_addresses
- wasm_table_index_free_list
- entry_points
- jit_cache_array
- page_first_jit_cache_entry
Other miscellaneous changes:
- Page is an abstract type
- Addresses, locals and bitflags are unsigned
- Make the number of entry points a growable type
- Avoid use of global state wherever possible
- Delete string packing
- Make CachedStateFlags abstract
- Make AnalysisType product type
- Make BasicBlockType product type
- Restore opcode assertion
- Set opt-level=2 in debug mode (for test performance)
- Delete JIT_ALWAYS instrumentation (now possible via api)
- Refactor generate_analyzer.js
- Refactor generate_jit.js
- introduce multiple entry points per compiled wasm module, by passing
the initial state to the generated function.
- continue analysing and compiling after instructions that change eip, but
will eventually return to the next instruction, in particular CALLs
(and generate an entry point for the following instruction)
This commit is incomplete in the sense that the container will crash
after some time of execution, as wasm table indices are never freed
This commit consists of three components:
1. A new generated x86-parser that analyses instructions. For now, it
only detects the control flow of an instruction: Whether it is a
(conditional) jump, a normal instruction or a basic block boundary
2. A new function, jit_find_basic_blocks, that finds and connects basic
blocks using 1. It loosely finds all basic blocks making up a function,
i.e. it doesn't follow call or return instructions (but it does follow
all near jumps). Different from our previous analysis, it also finds
basic blocks in the strict sense that no basic block contains a jump
into the middle of another basic block
3. A new code-generating function, jit_generate, that takes the output
of 2 as input. It generates a state machine:
- Each basic block becomes a case block in a switch-table
- Each basic block ends with setting a state variable for the following basic block
- The switch-table is inside a while(true) loop, which is terminated
by return statements in basic blocks which are leaves
Additionally:
- Block linking has been removed as it is (mostly) obsoleted by these
changes. It may later be reactived for call instructions
- The code generator API has been extended to generate the code for the state machine
- The iterations of the state machine are limited in order to avoid
infinite loops that can't be interrupted
These instructions, if included within a compiled JIT block, may alter the
state_flags of a block entry (such as whether flat segmentation is used or not),
which may invalidate the block that is running - this caused bugs in OpenBSD
because of a block like this being compiled:
0xF81F2: 8E DB mov ds, bx
0xF81F4: 8E D3 mov ss, bx
0xF81F6: 66 8B 26 B8 F5 mov esp, dword ptr [0xf5b8] <--
0xF81FB: 66 89 36 B8 F5 mov dword ptr [0xf5b8], esi <--
The memory accesses implicitly use DS. If we include flat-segmenetation as a
flag within state_flags and optimize calls to get_seg based on it, this behavior
would cause issues (and did, in OpenBSD).
By marking these instructions as block boundaries, we remediate that issue.
We generate a version of the push/pop instruction with the stack_size_32 fixed,
since the state tends not to change much. If it does change, state_flags won't
match the output of pack_current_state_flags and the cache entry will therefore
be invalidated.
See:
https://gist.github.com/AmaanC/faff7066d16f1dee4bbbd6b73a72d831
From the geek32[1] sheet, the criteria for nonfaulting instructions used was:
```
groups = ['arith', 'logical', 'conver', 'datamov', 'datamov arith', 'shftrot', 'flgctrl'];
// Excluded because they may trigger faults, so the optimization can't apply to them
excluded_opcodes = [
// May trigger_ud
'0x8C',
// switch_seg may fault
'0x8E',
// mov to/from seg:offset (memory accesses)
'0xA0',
'0xA1',
'0xA2',
'0xA3',
// Unimplemented in v86
'0xD50A',
'0x0F38F0',
'0x0F38F1',
];
// Keywords that indicate a group/instruction which may fault
excluded_row_words = ['fpu', 'simd', 'mmx', 'sse', 'vmx', 'XLAT', 'DIV', 'AMX', 'AAM', 'CLI', 'STI', 'CMPXCHG8B'];
```
[1] http://ref.x86asm.net/geek32.html#x0F90
2e469796 Minor
fab422ef Improve generation of 0f instructions
08ad7fe9 Improved if-else generation
3f81014d Minor: Align test output
4a3a84ef Generate modrm tests
61aa1875 Simplify
a6e47954 Generate decoding of immediate operands
435b2c10 Fix warnings
e4933042 Add missing immediate operand
3f3810c7 Generate immediate operands for instructions with modrm byte
a0aa7b1f Make memory layout in nasm tests clearer
6b8ef212 Remove 'g' property from instruction table (implied by 'e')
bf15c58c Remove unused declarations
1e543035 Remove useless `| 0` and `>>> 0` javascriptisms
1ccc5d53 Fix headers
8b40c532 Update qemu tests with changes from qemu
ec9b0fb5 Port xchg instructions to C
c73613e7 Port virt_boundary_* to C
d61d1241 Add headers
fd19f22c Make written value in write8 and write16 int32_t
497dcaec Generate read_imm for instructions with a modrm byte
8b7003d6 Generate read_imm8s
0cc75498 Remove read_op
9d716086 Trigger unimplemented_sse for partial sse instructions with prefix
8d5edd03 Remove unimplmented sse c-to-js hack
585d3565 Remove | 0
308124b2 Use int32_t as return value
f193f8e1 Use JS version of cvttsd2si for now
12747b97 Generate trigger_ud for missing modrm branches
770f674e Split 0f00 and 0f01 into multiple instructions depending on modrm bits
1cb372a3 Generate decoder for some 0f-prefix instructions
cec7bc63 Disable unused parameter warnings in instruction functions
807665b1 Generate read_imm for 0f/jmpcc
cdf6eccc Generate modrm decoding for shld
04528429 Create temporary files in /tmp/, not cwd
d8f3fbd8 Generate modrm/imm decoding for shld
00ef0942 Generate modrm decoding for bts
f531984b Generate modrm decoding for shrd and imul
07569c53 Generate modrm decoding cmpxchg
535ff190 Generate modrm decoding for lfs/lgs/lss
2f8ced8d Generate modrm decoding for btr and btc
95de6c66 Generate modrm decoding for movzx
c4d07e7e Generate modrm decoding for bsf and bsr
f0985d26 Generate modrm decoding for movsx
4b30937a Generate modrm decoding for xadd
a422eb27 Generate modrm decoding for movnti
e5501d3c Generate modrm decoding for mov to/from control/debug registers
bce11ec5 Generate modrm decoding for lar/lsl
5729a23c Fix access to DR4 and DR5 when cr4.DE is clear
44269a81 Specify immediate size explicitly instead of inferring it
82b2867a Fix STR instruction
98a9cc89 Log failing assertion
6d2f9964 Fix rdtsc
00260694 Log GP exceptions
7916883d Port trigger_ud and trigger_nm to C
36fedae9 Remove unused code
e08fabd0 Generate modrm decoding for 0f00 and 0f01
8ae8174d Generate modrm decoding for 0fae and 0fc7 (fxsave, cmpxchg8, etc.)
26168164 Generate modrm+immediate decoding for 0fba (bit test with immediate operand)
6adf7fa7 Simplify create_tests.js (unused prefix call)
c77cbdd8 Add comments about the implementation of pop [addr]
4640b4fe Simplify prefix call
a81a5497 Don't use var
3ca5d13d Separate call name and arguments in code generator
3191a543 Simplify other prefix call (8D/lea)
5185080e Update generated code (stylistic changes and #ud generation)
93b51d41 Remove unused wasm externals
e4af0a7f Avoid hardcoding special cases in code generator (lea, pop r/m)
654a2b49 Avoid hardcoding special cases in code generator (enter/jumpf/callf)
fd1a1e86 Commit generated code (only stylistic changes)
7310fd1a Simplify code generator by merging code for with and without 0f prefix
e7eae4af Simplify code generator by merging code for immediate operands
00fafd8a Improve assertions
db084e49 Simplify code generator (modrm if-else)
0a0e4c9e Improve code generation of switch-case
ce292795 Clarify some comments
37cf33fa Generate code in if/else blocks
cbcc33fc Document naming scheme
e30b97eb Generate modrm decoding for 0f12 (sse) instruction
24b72c2f movlpd cannot be used for register-to-register moves
72d72995 Generate modrm decoding for 0f13 (sse) instruction and disable register-to-register moving
75d76fbb Generate modrm decoding for 0f14 (sse) instruction
ac8965a7 Generate modrm decoding for 0f28-0f2b (movap, movntp)
e919d33e Generate modrm decoding for cvttsd2si
5f2ca2b4 Generate modrm decoding for andp and xorp
c8d1c6de Generate modrm decoding for 0f60-0f70 (sse instructions)
ae4ed46d Add multi-byte nop and prefetch to nasm test, generate modrm decoding
718a1acf Print qemu test error message more useful
d1ecc37e Generate modrm decoding for 0f70-0f80 (sse instructions)
6a7219a5 Generate modrm decoding for popcnt
25278217 Generate modrm decoding for 0f71-0f73 (sse shift with immediate byte)
ed1ec81b Generate modrm decoding for the remaining sse instructions (0fc0-0fff)
42bc5a6f Use 64-bit multiplication for native code
dda3fb39 Remove old modrm-decoding functions
717975ef Move register access functions to cpu.c
aee8138f Remove read_op, read_sib, read_op0F, read_disp
f31317f2 Rename xmm/mmx register access functions
a525e70b Remove 32-bit access to reg_xmm and reg_mmx
c803eabc Rename s8/s16/s32 to i8/i16/i32
9fbd2ddf Don't use uninitialised structs
942eb4f7 Use 64-bit load for mmx registers and assert reg64 and reg128 size
f94ec612 Use 64-bit writes for write_xmm64
08022de9 Use more efficient method for some 128-bit stores
9d5b084c Make timestamp counter unsigned
2ef388b3 Pass 64-bit value to safe_write64
4cb2b1be Optimise safe_write64 and safe_write128
b0ab09fb Implement psllq (660ff3)
9935e5d4 Optimise safe_read64s and safe_read128s
af9ea1cc Log cl in cpuid only if relevant
be5fe23e Add multi-op cache (disabled by default through ENABLE_JIT macro) and JIT paging test (similar to QEMU test).
aa2f286e Don't initialise group_dirtiness with 1 as it increases the binary size significantly
b8e14ed9 Remove unused reg_xmm32s
bc726e03 Implement dbg_log for native code with format characters 'd' and 'x'
454039d6 Fix store task register
63a4b349 Remove unnecessary parens and clean up some log statements
4cc96814 Add logop and dbg_trace imports
7940655d Only inhibit interrupts if the interrupt flag was 0 in STI
876c68a7 Split create_tests into create_tests and generate_interpreter
aa82499f Move detection of string instructions to x86_table
f3840ec2 Move C ast to separate file
90400703 Skip tests for lfence/mfence/sfence, clarify their encoding
4a9d8204 elf: Hide log messages when log level is zero
a601c526 Allow setting log level via settings
8a624453 Add cpu_exception_hook to debug builds
f9e335bf Nasm: Test exceptions
599ad088 logop: Format instruction pointer as unsigned
f95cf22b Don't skip zero dividing tests
2a655a0e Remove get_seg_prefix_ds from read_moffs (preparation for calling read_moffs from the code generator)
bc580b71 Remove obsolete comment
e556cee0 Fix nasmtest dependencies in makefile and clean
dcb1e72b Use all cores on travis
86efa737 Replace all instances of u32 & 0xFFFF with the respective u16 accesses
98b9f439 Use u8 instead of bit-shifts and masks from u32
b43f6569 Replace all instances of u32 >> 16 with the respective u16 accesses
9bfa72c7 Remove unnecessary parens
9cf93734 Clean up remaining instance of u32 with a mask instead of u16
22d4117f Correct order of writes in virt_boundary_write32
6734c7c1 Fix keyboard on ios, fixes#105
858a4506 Add missing file, c_ast.js
1d62e39e Move instruction naming scheme into function
f4816852 Reorder some code
69d49788 Minor improvements
0493e05f Add util.js
af9000c1 Improve full test
e5feba31 Add missing export
c7c42065 Replace prefix_call with custom_resolve_modrm
3186e6ad Add support for "%%" format string to dbg_log_wasm for printf import
efe54fad Add barebones instrumentation profiler (disabled by default).
c9f0d462 Implement movlps m64, xmm and enable its test
42869a12 Add tests for cross-page reads/writes confirmed with byte reads/writes
d68976ea Mask word values in port byte reads
9758d51e Add PS2_LOG_VERBOSE
5f52f037 Update NASM Makefile to include all dependencies to prevent unnecessary recompilation
2c71f927 Have NASM test generator use a seedable PRNG to allow for faster incremental tests
e4aa45bb Add chunk{16,32}_rw paging tests; instructions that read and write to memory
bdf538a2 add codegen to cpu constructor
aa76ce8e add resolve_modrm16
14d7ecf1 refactor codegen
b710319f [rebased] Merge branch codegen
0565ea42 minor refactoring
071dff3f temporary fix for automatic cast warnings
57c504f2 fix modrm16 issue
c2db5d9e jit modrm32
85c04245 reinstate modrm_fn0 and modrm_fn1
be65dafd add ip and previous ip manipulating functions
ae00ef89 update codegen js interface
530a74fa squashed commit for refactor
2c692199 add codegen-test to build
c15afe68 prefix gen to codegen api
c9611533 codegen tests fixes
