summaryrefslogtreecommitdiff
path: root/yjit_iface.c
Commit message (Collapse)AuthorAgeFilesLines
* Change darray size to size_t and add functions that use GC mallocPeter Zhu2022-02-161-2/+2
| | | | | | | Changes size and capacity of darray to size_t to support more elements. Adds functions to darray that use GC allocation functions.
* Parenthesize a macro expressionNobuyoshi Nakada2022-02-161-1/+1
| | | | The modulo in `rb_yjit_code_page_alloc` seems interpreted wrongly.
* YJIT: Avoid pointer size assumption with intptr_tAlan Wu2021-12-301-2/+1
| | | | Cast to `void *` first to use the definition of `intptr_t`.
* YJIT: Use proper size prefix and conversion where IL32LLP64Nobuyoshi Nakada2021-12-291-1/+2
|
* YJIT: Fix warning in iface.cJohn Hawthorn2021-12-291-0/+1
| | | | | | | Fixes warning: <internal:yjit>:16: warning: undefining the allocator of T_DATA class RubyVM::YJIT::Block
* Rename --jit to --mjit (#5248)Takashi Kokubun2021-12-131-1/+1
| | | | | | | | | | | | | | | * Rename --jit to --mjit [Feature #18349] * Fix a few more --jit references * Fix MJIT Actions * More s/jit/mjit/ and re-introduce --disable-jit * Update NEWS.md * Fix test_bug_reporter_add
* YJIT: Fix incomplete invalidation from opt_setinlinecacheAlan Wu2021-12-061-10/+23
| | | | | | | | | | | | | | | | As part of YJIT's strategy for promoting Ruby constant expressions into constants in the output native code, the interpreter calls rb_yjit_constant_ic_update() from opt_setinlinecache. The block invalidation loop indirectly calls rb_darray_remove_unordered(), which does a shuffle remove. Because of this, looping with an incrementing counter like done previously can miss some elements in the array. Repeatedly invalidate the first element instead. The bug this commit resolves does not seem to cause crashes or divergent behaviors. Co-authored-by: Jemma Issroff <jemmaissroff@gmail.com>
* YJIT: Bounds check every byte in the assemblerAlan Wu2021-12-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
* Check that cb / ocb exist before marking executableAaron Patterson2021-12-011-2/+9
| | | | | | | | | If YJIT isn't enabled, or hasn't finished booting, cb / ocb could be null. This commit just checks to make sure they're available before marking as executable Co-Authored-By: Maxime Chevalier-Boisvert <maxime.chevalierboisvert@shopify.com> Co-Authored-By: Kevin Newton <kddnewton@gmail.com>
* Mark JIT code as writeable / executable depending on the situationAaron Patterson2021-12-011-1/+10
| | | | | | | | | | | | | | | | Some platforms don't want memory to be marked as writeable and executable at the same time. When we write to the code block, we calculate the OS page that the buffer position maps to. Then we call `mprotect` to allow writes on that particular page. As an optimization, we cache the "last written" aligned page which allows us to amortize the cost of the `mprotect` call. In other words, sequential writes to the same page will only call `mprotect` on the page once. When we're done writing, we call `mprotect` on the entire JIT buffer. This means we don't need to keep track of which pages were marked as writeable, we let the OS take care of that. Co-authored-by: John Hawthorn <john@hawthorn.email>
* YJIT: Fail gracefully while OOM for new entry pointsAlan Wu2021-12-011-2/+1
| | | | | | | | | | | | | | | | | | | | Previously, YJIT crashes with rb_bug() when asked to compile new methods while out of executable memory. To handle this situation gracefully, this change keeps track of all the blocks compiled each invocation in case YJIT runs out of memory in the middle of a compliation sequence. The list is used to free all blocks in case compilation fails. yjit_gen_block() is renamed to gen_single_block() to make it distinct from gen_block_version(). Call to limit_block_version() and block_t allocation is moved into the function to help tidy error checking in the outer loop. limit_block_version() now returns by value. I feel that an out parameter with conditional mutation is unnecessarily hard to read in code that does not need to go for last drop performance. There is a good chance that the optimizer is able to output identical code anyways.
* YJIT: Add ability to exit to interpreter from stubsAlan Wu2021-11-261-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, YJIT assumed that it's always possible to generate a new basic block when servicing a stub in branch_stub_hit(). When YJIT is out of executable memory, for example, this assumption doesn't hold up. Add handling to branch_stub_hit() for servicing stubs without consuming more executable memory by adding a code path that exits to the interpreter at the location the branch stub represents. The new code path reconstructs interpreter state in branch_stub_hit() and then exits with a new snippet called `code_for_exit_from_stub` that returns `Qundef` from the YJIT native stack frame. As this change adds another place where we regenerate code from `branch_t`, extract the logic for it into a new function and call it regenerate_branch(). While we are at it, make the branch shrinking code path in branch_stub_hit() more explicit. This new functionality is hard to test without full support for out of memory conditions. To verify this change, I ran `RUBY_YJIT_ENABLE=1 make check -j12` with the following patch to stress test the new code path: ```diff diff --git a/yjit_core.c b/yjit_core.c index 4ab63d9806..5788b8c5ed 100644 --- a/yjit_core.c +++ b/yjit_core.c @@ -878,8 +878,12 @@ branch_stub_hit(branch_t *branch, const uint32_t target_idx, rb_execution_contex cb_set_write_ptr(cb, branch->end_addr); } +if (rand() < RAND_MAX/2) { // Compile the new block version p_block = gen_block_version(target, target_ctx, ec); +}else{ + p_block = NULL; +} if (!p_block && branch_modified) { // We couldn't generate a new block for the branch, but we modified the branch. ``` We can enable the new test along with other OOM tests once full support lands. Other small changes: * yjit_utils.c (print_str): Update to work with new native frame shape. Follow up for 8fa0ee4d404. * yjit_iface.c (rb_yjit_init): Run yjit_init_core() after yjit_init_codegen() so `cb` and `ocb` are available.
* YJIT: Make block invalidation more robustAlan Wu2021-11-221-9/+30
| | | | | | | | | | | | | | | | | | | | | This commit adds an entry_exit field to block_t for use in invalidate_block_version(). By patching the start of the block while invalidating it, invalidate_block_version() can function correctly while there is no executable memory left for new branch stubs. This change additionally fixes correctness for situations where we cannot patch incoming jumps to the invalidated block. In situations such as Shopify/yjit#226, the address to the start of the block is saved and used later, possibly after the block is invalidated. The assume_* family of function now generate block->entry_exit before remembering blocks for invalidation. RubyVM::YJIT.simulate_oom! is introduced for testing out of memory conditions. The test for it is disabled for now because OOM triggers other failure conditions not addressed by this commit. Fixes Shopify/yjit#226
* Add --yjit-no-type-prop so we can test YJIT without type propagation (#5135)Maxime Chevalier-Boisvert2021-11-181-0/+6
| | | | | | | * Add --yjit-no-type-prop so we can test YJIT without type propagation * Fix typo in command line option * Leave just two test workflows enable for YJIT
* Suppress unused-function warningsNobuyoshi Nakada2021-11-051-0/+4
|
* YJIT code pages refactoring for code GC (#5073)Maxime Chevalier-Boisvert2021-11-041-21/+120
| | | | | | | | | | | | | | | * New code page allocation logic * Fix leaked globals * Fix leaked symbols, yjit asm tests * Make COUNTED_EXIT take a jit argument, so we can eliminate global ocb * Remove extra whitespace * Change block start_pos/end_pos to be pointers instead of uint32_t * Change branch end_pos and start_pos to end_addr, start_addr
* remove the repeat 'the'1809092021-11-021-1/+1
|
* Rename ::YJIT to RubyVM::YJITAlan Wu2021-10-281-1/+1
| | | | | Since the YJIT Ruby module is CRuby specific and not meant for general use, it should live under RubyVM instead of at top level.
* YJIT: move --yjit-stats at_exit call into RubyAlan Wu2021-10-271-13/+6
| | | | | | This change fixes `-v --yjit-stats`. Previously in this situation, YJIT._print_stats wasn't defined as yjit.rb is not evaluated when there is only "-v" and no Ruby code to run.
* YJIT: Don't take VM lock on constant IC fill when disabledAlan Wu2021-10-221-0/+2
| | | | | While theoretically it's fine to take the lock and then immediately release it, we don't need to do it when YJIT is off.
* Don't enable YJIT by default. More tests on both Ubuntu and MacOS.Noah Gibbs2021-10-201-1/+1
| | | | | Add RUBY_YJIT_ENABLE env var and YJIT_FORCE_ENABLE compile-time constant. Rename YJIT_STATS to RUBY_YJIT_STATS.
* Put YJIT into a single compilation unitAlan Wu2021-10-201-24/+19
| | | | | | | | | | | | | | | | | | | | | For upstreaming, we want functions we export either prefixed with "rb_" or made static. Historically we haven't been following this rule, so we were "leaking" a lot of symbols as `make leak-globals` would tell us. This change unifies everything YJIT into a single compilation unit, yjit.o, and makes everything unprefixed static to pass `make leak-globals`. This manual "unified build" setup is similar to that of vm.o. Having everything in one compilation unit allows static functions to be visible across YJIT files and removes the need for declarations in headers in some cases. Unnecessary declarations were removed. Other changes of note: - switched to MJIT_SYMBOL_EXPORT_BEGIN which indicates stuff as being off limits for native extensions - the first include of each YJIT file is change to be "internal.h" - undefined MAP_STACK before explicitly redefining it since it collide's with a definition in system headers. Consider renaming?
* Remove unused functionAlan Wu2021-10-201-16/+0
|
* Fix changes from rebaseNoah Gibbs2021-10-201-3/+3
|
* style: align pointer "*" to the rightAlan Wu2021-10-201-13/+13
|
* Add counters for version invalidation reasonsAlan Wu2021-10-201-10/+27
| | | | | I noticed that there were two st_table iterators that do exactly the same thing so I merged them into one.
* Try to break the code page refactoring into smaller stepsMaxime Chevalier-Boisvert2021-10-201-0/+42
|
* Fix excessive invalidation for opt_getinlinecacheAlan Wu2021-10-201-1/+9
| | | | | | | | | | | | | | | | | | | YJIT expects the VM to invalidate opt_getinlinecache when updating the constant cache, and the invalidation used to happen even when YJIT can't use the cached value. Once the first invalidation happens, the block for opt_getinlinecache becomes a stub. When the stub is hit, YJIT fails to compile the instruction as the cache is not usable. The stub becomes a block that exits for opt_getinlinecache which can be invalidated again. Some workloads that bust the interpreter's constant cache can create an invalidation loop with this behavior. Check if the cache is usable become doing invalidation to fix this problem. In the test harness, evaluate the test script in a lambda instead of a proc so `return` doesn't return out of the harness.
* Add counters for tracking invalidationsAlan Wu2021-10-201-0/+3
|
* Fix warnings about redefining YJIT_STATSAlan Wu2021-10-201-6/+1
| | | | | | Follow up for ecb5b383a0c17550b9b27663005049ddac871edb. Now that YJIT_STATS is defined in yjit.h, it shoudl be the only place that defines it.
* Prevent stats being enabled late at run-timeMaxime Chevalier-Boisvert2021-10-201-7/+0
|
* Implement invokesuper using cfp->ep[ME] checkJohn Hawthorn2021-10-201-1/+1
| | | | | This fixes and re-enables invokesuper, replacing the existing guards with a guard on the method entry for the EP.
* Store block callee_cme in darrayJohn Hawthorn2021-10-201-18/+28
| | | | This allows a block version to have dependencies on multiple CMEs.
* Allow to toggle YJIT stats collection from runtimeJean Boussier2021-10-201-0/+7
| | | | | | | | For use cases where you want to collect the metrics for a specific piece of code (typically a web request) you can have the stats turned off by default and then turn them on at runtime before executing the code you care about.
* TracePoint supportAlan Wu2021-10-201-0/+11
| | | | | | | | | | | | | | | | | | | | | | This change fixes some cases where YJIT fails to fire tracing events. Most of the situations YJIT did not handle correctly involves enabling tracing while running inside generated code. A new operation to invalidate all generated code is added, which uses patching to make generated code exit at the next VM instruction boundary. A new routine called `jit_prepare_routine_call()` is introduced to facilitate this and should be used when generating code that could allocate, or could otherwise use `RB_VM_LOCK_ENTER()`. The `c_return` event is fired in the middle of an instruction as opposed to at an instruction boundary, so it requires special handling. C method call return points are patched to go to a fucntion which does everything the interpreter does, including firing the `c_return` event. The generated code for C method calls normally does not fire the event. Invalided code should not change after patching so the exits are not clobbered. A new variable is introduced to track the region of code that should not change.
* Allow to compile with --yjit-stats support but not the full RUBY_DEBUGJean Boussier2021-10-201-9/+16
| | | | | | | | | | | RUBY_DEBUG have a very significant performance overhead. Enough that YJIT with RUBY_DEBUG is noticeably slower than the interpreter without RUBY_DEBUG. This makes it hard to collect yjit-stats in production environments. By allowing to collect JIT statistics without the RUBy_DEBUG overhead, I hope to make such use cases smoother.
* If codeblock is NULL because YJIT is disabled, YJIT.runtime_stats should ↵Noah Gibbs2021-10-201-0/+5
| | | | return Qnil
* Add flag so we can easily tell if all stats avail. Comment out broken test.Maxime Chevalier-Boisvert2021-10-201-1/+3
|
* Make sure we can still compile with the JIT disabledAaron Patterson2021-10-201-3/+5
| | | | | | | If `--disable-jit-support` is passed to configure, then `jit_func` is removed from the iseq body and we can't compile YJIT. This commit detects when the JIT function pointer is gone and disables YJIT in that case.
* Remove the scraperAaron Patterson2021-10-201-18/+1
| | | | | Now that we're using the jit function entry point, we don't need the scraper. Thank you for your service, scraper. ❤️
* make compiler happyAaron Patterson2021-10-201-2/+3
|
* Always use `ret` to return to the interpreterAaron Patterson2021-10-201-5/+8
| | | | | | | | | | | | | | | | | | | | | Always using `ret` to return to the interpreter means that we never have to check the VM_FRAME_FLAG_FINISH flag. In the case that we return `Qundef`, the interpreter will execute the cfp. We can take advantage of this by setting the PC to the instruction we can't handle, and let the interpreter pick up the ball from there. If we return a value other than Qundef, the interpreter will take that value as the "return value" from the JIT and push that to the SP of the caller The leave instruction puts the return value on the top of the calling frame's stack. YJIT does the same thing for leave instructions. However, when we're returning back to the interpreter, the leave instruction _should not_ put the return value on the top of the stack, but put it in RAX and use RET. This commit pops the last value from the stack pointer and puts it in RAX so that the interpreter is happy with SP.
* YJIT stats should always include the inlined and outlined sizes, regardless ↵Noah Gibbs2021-10-201-12/+12
| | | | of RUBY_DEBUG and --yjit-stats/YJIT_STATS settings
* Add (void) for no arg functionsMaxime Chevalier-Boisvert2021-10-201-1/+1
|
* First pass at code page GC object.Maxime Chevalier-Boisvert2021-10-201-0/+35
|
* rb_struct_define_under needs a trailing NULLAaron Patterson2021-10-201-1/+1
| | | | | The last parameter to rb_struct_define_under needs to be NULL otherwise we can get a SEGV.
* Use snprintf rather than double strncpy.Noah Gibbs2021-10-201-2/+1
|
* Change strcpy of a static string to strncpyNoah Gibbs2021-10-201-1/+1
|
* Better comments where we add exits-by-opcode to the stats hash, plus a ↵Noah Gibbs2021-10-201-4/+5
| | | | presumably-unneeded strncpy just to be sure.
* Add back ifdefs for RUBY_DEBUG, accidentally removedNoah Gibbs2021-10-201-0/+2
|
View Lorry Controller Status