| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
This reverts commit b7dc04e51823f9fe8b5355c30a304ecdb11fe5ed.
This should be fine, rather necessary, too.
|
|
|
|
|
|
| |
This reverts commit 73b07c437e24711c23dd2dd01d3ffc5f1012e046.
This was, of course, innocent.
|
|
|
|
| |
list_for_each seems to cause all the SEGVs we've seen.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Revert "Lock the entire active_units loop"
This reverts commit 5c2ff88be2e515613dfe54823e8429656f688e9f.
Revert "Lock active_units references on compaction"
This reverts commit 556a7285080c1344c75bb93a333c9bfc5d631c61.
Revert "Wait for GC before unload_units"
This reverts commit a8f16df615daa55901bb351efe038e86b61fbb92.
Well, the previous revert actually didn't fix it, but this series of
reverts seems to rollback the situation a little.
|
|
|
|
|
|
|
| |
This reverts commit 3319ce37651aa7e50c31b5fba14871938318b37a.
I still haven't figured out why, but this seems to have increased the
failure rate.
|
| |
|
|
|
|
| |
The previous fix seems not working. Let me test if this works.
|
|
|
|
| |
to convert list_node to rb_mjit_unit easily in gdb.
|
|
|
|
| |
This might race with mjit_recompile.
|
|
|
|
|
| |
--jit-wait CI can be stuck when the workaround is throttled
http://ci.rvm.jp/results/trunk-mjit-wait@phosphorus-docker/3274091
|
|
|
|
|
|
|
| |
The compilation for JIT compaction is very heavy. Triggering a second
compaction to include one more new method is probably not worth it. So
this triggers JIT compaction for ten more new methods after each
compaction.
|
|
|
|
|
|
|
|
|
|
| |
Because d80226e7bd often reduces the number of unloaded units, it
increases the number of unload_units calls, which are heavy.
To mitigate that, this throttles unload_units per `max_cache_size / 10`.
Also hoping to fix
https://ci.appveyor.com/project/ruby/ruby/builds/36552382/job/kjmjgw9cjyf2ksd7
|
|
|
|
| |
instead of just unloading worst 10% methods.
|
| |
|
|
|
|
|
|
|
|
| |
to avoid "Too many JIT code, but skipped unloading units for JIT compaction".
Now we can forget the `in_compact` locking.
Moving some functions from mjit.c to mjit_worker.c because mjit_worker.c
should have functions executed in the JIT worker.
|
| |
|
|
|
|
|
|
|
|
|
| |
* Re-generate C files for JIT compaction every time
* Refactor in_jit return logic
* Just write code in a single file
* Add a TODO comment [ci skip]
|
|
|
|
| |
This was a leftover of 27d5af59a359909e0d434459c30cfc0940f60a5b.
|
|
|
|
|
|
|
| |
convert_unit_to_func's c_func / so_func construction is unnecessarily
complicated while it's not really safer than what compact_all_jit_code
does. So I changed convert_unit_to_func to be consistent with
compact_all_jit_code.
|
|
|
|
|
|
|
|
|
|
|
| |
This has been a TODO since 79df14c04b. While adcf0316d1 covered the
root_fiber of the initial thread, it didn't cover root_fibers of other
threads. Now it's hooked properly in rb_threadptr_root_fiber_setup.
With regards to "XXX: Is this mjit_cont `mjit_cont_free`d?", when
rb_threadptr_root_fiber_release is called, although I'm not sure when
th->root_fiber is truthy, fiber_free seems to call cont_free and
mjit_cont_free. So mjit_conts of root_fibers seem to be freed properly.
|
|
|
|
| |
It's calculated inside compile_c_to_so again.
|
|
|
|
| |
Apparently #ifdef is always true
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
_MSC_VER used to be the macro to switch JIT compaction. However, since
d4381d2ceb, the correct macro to switch it was changed from _MSC_VER
to _WIN32. As I didn't properly replace all relevant _MSC_VER usages
to _WIN32, these macros have been used inconsistently.
nobu replaced _WIN32 with USE_HEADER_TRANSFORMATION in 5eb446d12f3.
Therefore we had USE_HEADER_TRANSFORMATION and _MSC_VER. This commit
makes sure such inconsistent _MSC_VER usages will be unified to the new
header, also renaming it to USE_JIT_COMPACTION to be more precise about
the requirements. The header transformation itself is not quite relevant
to places changed in this commit.
|
|
|
|
|
|
|
|
| |
Isn't setting `in_compact = true` enough to avoid a race condition
between JIT compaction and unload_units? Now I think it is.
This change will make it easier to spend more time on compile_compact_jit_code.
For now it seems to take only 0.0723ms though.
|
|
|
|
| |
Thanks to Ractor (https://github.com/ruby/ruby/pull/2888 and https://github.com/ruby/ruby/pull/3662),
inline caches support parallel access now.
|
|
|
|
|
|
|
|
| |
This reverts commit 6cb6d5abc36ede9d5158c2cd90734134838e6bfb.
This reverts commit 1484b786aee8d411a9e2278ac6d6e44aedbf6662.
I think we don't need these assertions anymore. I believe the problem
is solved by abf678a4397c6c00a1bb686043e377d372e695a4
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We are seeing an error where code that is generated with MJIT contains
references to objects that have been moved. I believe this is due to a
race condition in the compaction function.
`gc_compact` has two steps:
1. Run a full GC to pin objects
2. Compact / update references
Step one is executed with `garbage_collect`. `garbage_collect` calls
`gc_enter` / `gc_exit`, these functions acquire a JIT lock and release a
JIT lock. So a lock is held for the duration of step 1.
Step two is executed by `gc_compact_after_gc`. It also holds a JIT
lock.
I believe the problem is that the JIT is free to execute between step 1
and step 2. It copies call cache values, but doesn't pin them when it
copies them. So the compactor thinks it's OK to move the call cache
even though it is not safe.
We need to hold a lock for the duration of `garbage_collect` *and*
`gc_compact_after_gc`. This patch introduces a lock level which
increments and decrements. The compaction function can increment and
decrement the lock level and prevent MJIT from executing during both
steps.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
```
compiling ../mjit.c
In file included from ../mjit.c:28:
../mjit_worker.c:1270:33: warning: incompatible pointer to integer conversion passing 'const struct rb_callcache *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(BUILTIN_TYPE(cc) != T_MOVED);
^~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../include/ruby/internal/value_type.h:153:23: note: passing argument to parameter 'obj' here
RB_BUILTIN_TYPE(VALUE obj)
^
In file included from ../mjit.c:28:
../mjit_worker.c:1271:33: warning: incompatible pointer to integer conversion passing 'const struct rb_callable_method_entry_struct *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(BUILTIN_TYPE(vm_cc_cme(cc)) != T_MOVED);
^~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../include/ruby/internal/value_type.h:153:23: note: passing argument to parameter 'obj' here
RB_BUILTIN_TYPE(VALUE obj)
^
In file included from ../mjit.c:28:
../mjit_worker.c:1272:50: warning: incompatible pointer to integer conversion passing 'const struct rb_callcache *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(!rb_objspace_garbage_object_p(cc));
^~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../gc.h:128:40: note: passing argument to parameter 'obj' here
int rb_objspace_garbage_object_p(VALUE obj);
^
In file included from ../mjit.c:28:
../mjit_worker.c:1273:50: warning: incompatible pointer to integer conversion passing 'const struct rb_callable_method_entry_struct *' to parameter of type 'VALUE' (aka 'unsigned long') [-Wint-conversion]
assert(!rb_objspace_garbage_object_p(vm_cc_cme(cc)));
^~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(__builtin_expect(!(e), 0) ? __assert_rtn(__func__, __FILE__, __LINE__, #e) : (void)0)
^
../gc.h:128:40: note: passing argument to parameter 'obj' here
int rb_objspace_garbage_object_p(VALUE obj);
^
4 warnings generated.
```
|
|
|
|
|
|
| |
This is a temporary commit to try to find a GC issue. It seems like
mjit is pointing at a moved address in the call cache. I want to assert
that they aren't TMOVED or garbage objects at the time they get copied
|
|
|
|
|
| |
Devs do not love "3". The only exception is RUBY3_KEYWORDS in parse.y,
which seems unrelated to our interests.
|
| |
|
|
|
|
|
|
|
|
| |
For some reason 0776198486 didn't fail
https://rubyci.org/logs/rubyci.s3.amazonaws.com/solaris10-gcc/ruby-master/log/20200503T230004Z.log.html.gz
even while it was before 9aa5fe1bf8.
Anyway, there's no need to keep the change anymore.
|
|
|
|
| |
I think 9aa5fe1bf89db8cd215b24d8ddfb668714681b83 helps this issue too.
|
|
|
|
|
|
|
|
| |
MinGW test_jit fails with no error message. Perhaps linker flags should
not be passed when compilation is happening.
Anyway splitting these stages doesn't matter for performance. So let me
just split it to fix the issue. Probably this helps Solaris's issue too.
|
|
|
|
|
|
| |
using -Winvalid-pch
https://rubyci.org/logs/rubyci.s3.amazonaws.com/solaris10-gcc/ruby-master/log/20200501T170004Z.fail.html.gz
|
|
|
|
| |
This support was accidentally removed in 818d6d33368a396d9cd3d1a34a84015a9e76c5c8.
|
|
|
|
|
|
|
| |
It's to avoid memory leak for actual usage (because they don't get
unloaded properly), but also for fixing CI timed out due to JIT
compaction taking too long time on --jit-wait (which runs every time)
http://ci.rvm.jp/results/trunk-mjit-wait@silicon-docker/2911601
|
|
|
|
|
|
|
| |
to fix
https://ci.appveyor.com/project/ruby/ruby/builds/32577700/job/yh61rom35wt2uv39
It was for JIT compaction, and we don't support it on mswin.
|
|
|
|
|
|
| |
as well. And also unit->c_file doesn't exist in mswin.
https://github.com/ruby/ruby/runs/635915704
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
to improve code locality.
Using benchmark-driver/sinatra with 100 methods JIT-ed,
[Before] 12149.97 rps
1.3M /tmp/_ruby_mjit_p31171u145.so
[After] 12818.83 rps
260K /tmp/_ruby_mjit_p32155u145.so
(VM is 13714.89 rps)
|
|
|
|
|
| |
I'm trying to make it possible to include all JIT-ed code in a single C
file. This is needed to guarantee uniqueness of all function names
|
|
|
|
|
|
|
|
|
|
| |
Running C compiler for JIT compaction inside a critical section may lock
main thread for a long time when it triggers GC. As I'm planning to
increase this duration a bit, I'd like to make sure this doesn't stop
the world.
For now, I chose to give up unloading units when it's during JIT
compaction, assuming other calls may unload them later.
|
|
|
|
| |
Namely icc defines __GNUC__, but doesn't have -Wdeprecated-declarations
|
|
|
|
|
|
|
|
|
| |
```
c:\projects\ruby\mjit_worker.c(1219) : warning C4090: 'function' : different 'const' qualifiers
```
It seems confused by passing "pointer to pointer to const object",
not "pointer to const object".
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
for perf output like:
Samples: 100K of event 'cycles:ppp', Event count (approx.): 1007750000
Children Self Command Shared Object Symbol
+ 81.58% 1.47% ruby ruby [.] rb_vm_exec
+ 81.06% 7.61% ruby ruby [.] vm_exec_core
+ 80.16% 0.00% ruby ruby [.] vm_sendish (inlined)
+ 75.03% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 74.37% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 73.42% 0.22% ruby _ruby_mjit_p11277u42.so [.] _mjit42_rack_method_override_rb_call
+ 73.25% 0.10% ruby _ruby_mjit_p11277u41.so [.] _mjit41_sinatra_show_exceptions_rb_call
+ 73.19% 0.22% ruby _ruby_mjit_p11277u44.so [.] _mjit44_rack_head_rb_call
+ 73.03% 0.15% ruby _ruby_mjit_p11277u45.so [.] _mjit45_sinatra_base_rb_call
+ 72.87% 0.26% ruby _ruby_mjit_p11277u49.so [.] _mjit49_rack_logger_rb_call
+ 70.56% 0.11% ruby _ruby_mjit_p11277u40.so [.] _mjit40_sinatra_base_rb_call
+ 68.70% 0.11% ruby _ruby_mjit_p11277u39.so [.] _mjit39_sinatra_base_rb_call
+ 68.39% 0.29% ruby _ruby_mjit_p11277u56.so [.] _mjit56_rack_protection_frame_options_rb_call
+ 67.89% 0.18% ruby _ruby_mjit_p11277u37.so [.] _mjit37_sinatra_base_rb_block_in_call
+ 67.04% 0.16% ruby _ruby_mjit_p11277u34.so [.] _mjit34_sinatra_base_rb_synchronize
Reverting deb1c7b97d, fixing `sprint_funcname`'s argument in `compact_all_jit_code`.
Also updating common.mk.
|
|
|
|
|
|
| |
This reverts commit cecebf55c476ae936f3e880477dfb62149143c46.
debugging test failure...
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
for perf output like:
Samples: 100K of event 'cycles:ppp', Event count (approx.): 1007750000
Children Self Command Shared Object Symbol
+ 81.58% 1.47% ruby ruby [.] rb_vm_exec
+ 81.06% 7.61% ruby ruby [.] vm_exec_core
+ 80.16% 0.00% ruby ruby [.] vm_sendish (inlined)
+ 75.03% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 74.37% 0.00% ruby ruby [.] mjit_exec (inlined)
+ 73.42% 0.22% ruby _ruby_mjit_p11277u42.so [.] _mjit42_rack_method_override_rb_call
+ 73.25% 0.10% ruby _ruby_mjit_p11277u41.so [.] _mjit41_sinatra_show_exceptions_rb_call
+ 73.19% 0.22% ruby _ruby_mjit_p11277u44.so [.] _mjit44_rack_head_rb_call
+ 73.03% 0.15% ruby _ruby_mjit_p11277u45.so [.] _mjit45_sinatra_base_rb_call
+ 72.87% 0.26% ruby _ruby_mjit_p11277u49.so [.] _mjit49_rack_logger_rb_call
+ 70.56% 0.11% ruby _ruby_mjit_p11277u40.so [.] _mjit40_sinatra_base_rb_call
+ 68.70% 0.11% ruby _ruby_mjit_p11277u39.so [.] _mjit39_sinatra_base_rb_call
+ 68.39% 0.29% ruby _ruby_mjit_p11277u56.so [.] _mjit56_rack_protection_frame_options_rb_call
+ 67.89% 0.18% ruby _ruby_mjit_p11277u37.so [.] _mjit37_sinatra_base_rb_block_in_call
+ 67.04% 0.16% ruby _ruby_mjit_p11277u34.so [.] _mjit34_sinatra_base_rb_synchronize
|
|
|
|
|
|
|
|
| |
OpenBSD RubyCI has failed with SEGV since 4bcd5981e80d3e1852c8723741a0069779464128.
https://rubyci.org/logs/rubyci.s3.amazonaws.com/openbsd-current/ruby-master/log/20200312T223005Z.fail.html.gz
This was because `status->cc_entries` could be stale after `realloc` call
for inlined iseqs.
|
|
|
|
|
|
|
|
| |
jit_unit to avoid marking wrong cc entries when inlined iseq is compiled
multiple times, resolving the TODO added by daf7c48d88.
This obviates pseudo jit_unit in inlined iseq introduced by 7ec2359374
and fixes memory leak of the adhoc unit.
|
| |
|