summaryrefslogtreecommitdiff
path: root/mjit.c
Commit message (Collapse)AuthorAgeFilesLines
* mjit_mark_cc_entries: should consider VM_CALLCACHE_UNMARKABLE卜部昌平2020-06-091-1/+1
| | | | | Now that vm_empty_cc is VM_CALLCACHE_UNMARKABLE, it has to be properly ruled out from being GCed.
* Eliminate a call instruction on JIT cancel pathTakashi Kokubun2020-05-261-2/+34
| | | | | | | by calling combined functions specialized for each cancel type. I'm hoping to improve locality of hot code, but this patch's impact should be insignificant.
* sed -i 's|ruby/impl|ruby/internal|'卜部昌平2020-05-111-1/+1
| | | | To fix build failures.
* sed -i s|ruby/3|ruby/impl|g卜部昌平2020-05-111-1/+1
| | | | This shall fix compile errors.
* Deduplicate functions in compacted JIT codeTakashi Kokubun2020-05-011-2/+2
| | | | | | | | | | | | | | to improve code locality. Using benchmark-driver/sinatra with 100 methods JIT-ed, [Before] 12149.97 rps 1.3M /tmp/_ruby_mjit_p31171u145.so [After] 12818.83 rps 260K /tmp/_ruby_mjit_p32155u145.so (VM is 13714.89 rps)
* Do not stop the world during JIT compactionTakashi Kokubun2020-04-301-7/+11
| | | | | | | | | | Running C compiler for JIT compaction inside a critical section may lock main thread for a long time when it triggers GC. As I'm planning to increase this duration a bit, I'd like to make sure this doesn't stop the world. For now, I chose to give up unloading units when it's during JIT compaction, assuming other calls may unload them later.
* Add MJIT_COUNTER macro to dump total_callsTakashi Kokubun2020-04-121-0/+20
|
* Merge pull request #2991 from shyouhei/ruby.h卜部昌平2020-04-081-1/+1
| | | Split ruby.h
* Add debug counter for unload_unitsTakashi Kokubun2020-03-151-0/+2
| | | | changing add_iseq_to_process's debug counter name as well for comparison
* Mark all cc_entries associated to compiled_iseqTakashi Kokubun2020-03-121-1/+2
|
* Move code to mark jit_unit's cc_entries to mjit.cTakashi Kokubun2020-03-121-3/+15
|
* Capture inlined iseq's cc entries in root iseq'sTakashi Kokubun2020-03-101-16/+0
| | | | | | | | jit_unit to avoid marking wrong cc entries when inlined iseq is compiled multiple times, resolving the TODO added by daf7c48d88. This obviates pseudo jit_unit in inlined iseq introduced by 7ec2359374 and fixes memory leak of the adhoc unit.
* Prevent unloading methods used in root_fiber while calling another Fiber (#2939)Takashi Kokubun2020-02-281-0/+4
| | | | | | Fixing SEGVs like: http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744905 http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2744420 http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2741400
* Avoid infinite loop on --jit-waitTakashi Kokubun2020-02-281-1/+8
|
* Internalize rb_mjit_unit definition againTakashi Kokubun2020-02-261-1/+7
| | | | Fixed a TODO in b9007b6c548f91e88fd3f2ffa23de740431fa969
* Remove an unused declarationTakashi Kokubun2020-02-251-2/+0
| | | | It was unnecessary in b9007b6c548f91e88fd3f2ffa23de740431fa969
* should be initialize jit_unit->cc_entries.Koichi Sasada2020-02-251-1/+1
| | | | | GC can invoke just after allocation of jit_unit->cc_entries so it should be zero-cleared.
* prevent GC from mjit worker.Koichi Sasada2020-02-251-6/+9
| | | | | | | ALLOC_N() can causes GC. Sometimes `mjit_copy_job_handler()` can be called by mjit_worker thread which is not a Ruby thread, so we need to prevent GC in this function. This patch has some issues, but I introduce it to pass the tests.
* Introduce disposable call-cache.Koichi Sasada2020-02-221-6/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | This patch contains several ideas: (1) Disposable inline method cache (IMC) for race-free inline method cache * Making call-cache (CC) as a RVALUE (GC target object) and allocate new CC on cache miss. * This technique allows race-free access from parallel processing elements like RCU. (2) Introduce per-Class method cache (pCMC) * Instead of fixed-size global method cache (GMC), pCMC allows flexible cache size. * Caching CCs reduces CC allocation and allow sharing CC's fast-path between same call-info (CI) call-sites. (3) Invalidate an inline method cache by invalidating corresponding method entries (MEs) * Instead of using class serials, we set "invalidated" flag for method entry itself to represent cache invalidation. * Compare with using class serials, the impact of method modification (add/overwrite/delete) is small. * Updating class serials invalidate all method caches of the class and sub-classes. * Proposed approach only invalidate the method cache of only one ME. See [Feature #16614] for more details.
* VALUE size packed callinfo (ci).Koichi Sasada2020-02-221-4/+0
| | | | | | | | | | | | | | | | | | | | Now, rb_call_info contains how to call the method with tuple of (mid, orig_argc, flags, kwarg). Most of cases, kwarg == NULL and mid+argc+flags only requires 64bits. So this patch packed rb_call_info to VALUE (1 word) on such cases. If we can not represent it in VALUE, then use imemo_callinfo which contains conventional callinfo (rb_callinfo, renamed from rb_call_info). iseq->body->ci_kw_size is removed because all of callinfo is VALUE size (packed ci or a pointer to imemo_callinfo). To access ci information, we need to use these functions: vm_ci_mid(ci), _flag(ci), _argc(ci), _kwarg(ci). struct rb_call_info_kw_arg is renamed to rb_callinfo_kwarg. rb_funcallv_with_cc() and rb_method_basic_definition_p_with_cc() is temporary removed because cd->ci should be marked.
* function pointers are not void*卜部昌平2020-02-061-1/+1
| | | | The same as 8427fca49bd85205f5a8766292dd893f003c0e48.
* decouple internal.h headers卜部昌平2019-12-261-3/+9
| | | | | | | | | | | | | | | | | | Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies).
* mjit.c: fix a mismatch of malloc'ed typeYusuke Endoh2019-12-031-2/+2
| | | | Coverity Scan found this issue.
* Simplify variable declaration by C99Takashi Kokubun2019-12-011-8/+6
|
* Constified mjit_initNobuyoshi Nakada2019-12-011-1/+1
|
* ConstifiedNobuyoshi Nakada2019-12-011-2/+2
|
* Fix a mistake excluding NULL in the endTakashi Kokubun2019-12-011-2/+2
|
* Workaround missing strndup on WindowsTakashi Kokubun2019-12-011-1/+11
| | | | https://ci.appveyor.com/project/ruby/ruby/builds/29230976/job/c910t37313edb97k
* Allow specifying arbitrary MJIT flags by --jit-debugTakashi Kokubun2019-12-011-0/+32
| | | | | | | | This is a secret feature for me. It's only for testing and any behavior with this flag override is unsupported. I needed this because I sometimes want to add debug options but do not want to disable optimizations, for using Linux perf.
* Combine call info and cache to speed up method invocationAlan Wu2019-10-241-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | To perform a regular method call, the VM needs two structs, `rb_call_info` and `rb_call_cache`. At the moment, we allocate these two structures in separate buffers. In the worst case, the CPU needs to read 4 cache lines to complete a method call. Putting the two structures together reduces the maximum number of cache line reads to 2. Combining the structures also saves 8 bytes per call site as the current layout uses separate two pointers for the call info and the call cache. This saves about 2 MiB on Discourse. This change improves the Optcarrot benchmark at least 3%. For more details, see attached bugs.ruby-lang.org ticket. Complications: - A new instruction attribute `comptime_sp_inc` is introduced to calculate SP increase at compile time without using call caches. At compile time, a `TS_CALLDATA` operand points to a call info struct, but at runtime, the same operand points to a call data struct. Instruction that explicitly define `sp_inc` also need to define `comptime_sp_inc`. - MJIT code for copying call cache becomes slightly more complicated. - This changes the bytecode format, which might break existing tools. [Misc #16258]
* Remove the quick stop path after convert_unit_to_funcTakashi Kokubun2019-10-131-2/+0
| | | | | | | | | | | | | | Now I'm not exactly sure why I needed to check `stop_worker_p` after `mjit_copy_cache_from_main_thread` of `convert_unit_to_func` in 4161674b2fbea6bdd01783ac5d3b39d88db22972. If it's for avoiding deadlock under `in_gc` condition, we should keep it. However, if it's not the case and it's just for retrying accidental compilation failure or just to avoid `MJIT_ATOMIC_SET` and `compact_all_jit_code`, I think this quick stop path is not mandatory. Because this path is somewhat problematic in my upcoming fix in mjit_worker, let me try to remove this first and see how CI goes.
* RubyVM::MJIT.pause(wait: true) should waitTakashi Kokubun2019-09-261-0/+2
| | | | | | | | | | | for all compilations and compaction. Prior to this commit, the last-compiled code has not been used because MJIT worker is stopped before setting the code, and compaction has also been skipped. But it was not intentional and `wait: true` pause should wait until those two things by its feature.
* Fix wrong allocation failure handlingTakashi Kokubun2019-09-201-2/+2
| | | | | `iseq->body->jit_unit->compile_info` should not be referenced before the null check of `iseq->body->jit_unit`.
* * expand tabs. [ci skip]git2019-08-221-1/+1
| | | | | Tabs are expanded because previously the file did not have any tab indentation. Please update your editor config, and use misc/expand_tabs.rb in the pre-commit hook.
* Improve the description of MJIT cancel-all [ci skip]Takashi Kokubun2019-08-221-0/+1
|
* Rename rb_gc_new_location to rb_gc_locationAaron Patterson2019-05-181-2/+2
| | | | | The function will return new or existing locations depending on whether or not the object actually moved, so give it a more appropriate name.
* Rename mjit_gc_finish_hook to mjit_gc_exit_hookTakashi Kokubun2019-05-151-3/+3
| | | | | because @ko1 said "gc_finish" is confusing like a finish of entire GC process
* glibc says memcpy cannot take NULLUrabe, Shyouhei2019-04-291-1/+1
| | | | | | | | At least since 2004, glibc's <string.h> annotates memcpy as __attribute__((__nonnull__)). This basedir is passed to it. When LOAD_RELATIVE is not defined and MJIT_SEARCH_BUILD_DIR is not set, this variable is never updated. Should initialize with meaningful default value.
* Fix wrong critical section labelk0kubun2019-04-201-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Prefix rb_ to MJIT wait call since it's exportedk0kubun2019-04-201-5/+5
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Revert "Skip ISeq reference for stale_units for debugging""k0kubun2019-04-201-1/+15
| | | | | | | | This reverts commit b79899b56a9ebadf5c493c4df2419ebf63934381. I wanted to test r67638 first. Now let me try this as well. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Invalidate JIT-ed code if ISeq is moved by GC.compactk0kubun2019-04-201-0/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Skip ISeq reference for stale_units for debugging"k0kubun2019-04-201-15/+1
| | | | | | | | This reverts commit 4bd4d4e4cfd164f767e88f844cf10b06faee14b9. Sorry, let me test another one first git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67637 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Skip ISeq reference for stale_units for debuggingk0kubun2019-04-201-1/+15
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67636 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Check ISeq references in stale_units took0kubun2019-04-201-0/+10
| | | | | | This is a possible bug from recent "JIT recompile" introduction. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67631 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Update ISeq references in stale_units as wellk0kubun2019-04-201-0/+11
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* fix strict compile errorstenderlove2019-04-201-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67625 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Update MJIT referencestenderlove2019-04-201-0/+13
| | | | | | ISeq can move, so we need to tell MJIT where the new location is. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Do not execute MJIT copy job when ISeq is GC-edk0kubun2019-04-141-1/+9
| | | | | | | | | I assumed that ISeq is never GC-ed by `in_jit` + `mjit_mark` on copy job ISeq, but unfortunately I found SEGV on `mjit_copy_job_handler` in which iseq->body was somehow Qnil. And it seems to be fixed by disabling the job when `mjit_free_iseq` is called for the ISeq. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67551 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Prefer uintptr_t over ptrdiff_t for i686k0kubun2019-04-141-1/+1
| | | | | | | | as debugged in PR [close https://github.com/ruby/ruby/pull/2130] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67549 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
View Lorry Controller Status