summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* * 2021-12-05 [ci skip]git2021-12-051-1/+1
|
* YJIT: Enable out of memory testsAlan Wu2021-12-041-3/+3
| | | | | | | | | | | | As of [1] and [2], YJIT has enough support for out of memory conditions to pass these two basic tests. OOM code paths are prone to bugs since they are rarely exercised in common workloads. We might want to add CI runs that stress test these code paths. Maybe outside of GitHub Actions for capacity reasons. [1]: f41b4d44f95978dfa97af04af00055dc3fbf7978 [2]: b5b6ab4194f16e96ee5004288cc469ac1bca41a3
* Remove unversioned phony target for pkgconfig file [Bug #18374]Nobuyoshi Nakada2021-12-042-1/+2
| | | | | It results in a circular dependency when `--with-ruby-pc=ruby.pc` is given. [ci skip]
* Fix circular dependencies specific to in-place build [Bug #18374]Nobuyoshi Nakada2021-12-041-4/+6
| | | | | | | | | | * Move the rubyspec running recipe after the rule for rubyspec C-API extension library, so that separate dummy recipe is not needed. * Add a dummy recipe for rubyspec.h before the rubyspec running recipe, so that the dependency of extensions do not fire the latter.
* Turn nd_type_p into an inline functionNobuyoshi Nakada2021-12-041-1/+5
|
* YJIT: Bounds check every byte in the assemblerAlan Wu2021-12-036-62/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, YJIT assumed that basic blocks never consume more than 1 KiB of memory. This assumption does not hold for long Ruby methods such as the one in the following: ```ruby eval(<<RUBY) def set_local_a_lot #{'_=0;'*0x40000} end RUBY set_local_a_lot ``` For low `--yjit-exec-mem-size` values, one basic block could exhaust the entire buffer. Introduce a new field `codeblock_t::dropped_bytes` that the assembler sets whenever it runs out of space. Check this field in gen_single_block() to respond to out of memory situations and other error conditions. This design avoids making the control flow graph of existing code generation functions more complex. Use POSIX shell in misc/test_yjit_asm.sh since bash is expanding `0%/*/*` differently. Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
* NEWS for [GH-#5146] [ci skip] (#5210)John Hawthorn2021-12-031-0/+4
|
* [rubygems/rubygems] Let original EACCES error be raisedDavid Rodríguez2021-12-042-15/+0
| | | | | | | | | | | | | | This block of code already wraps file operations with `SharedHelpers.filesystem_access`, which rescues and re-raises more friendly errors. Also, I'm not fully sure creating a temporary directory can end up raising an `Errno::EACCES` error from reading `tmpdir` sources. Finally, this rescue block apparently leads to some false positives when firewall is blocking the ruby executable on Windows, or at least that's what we've got reported. In any case, I think it's best to let the original error be raised. https://github.com/rubygems/rubygems/commit/f7dbe54404
* YJIT: Add ivar counter exitseileencodes2021-12-032-2/+3
| | | | | | | | | | | | On Rails we're seeing a lot of exits for ivars in the Active Record tests. In trying to track them down it was hard to find what code is exiting. This change adds a counted exit for when an object is "megamorphic". In these cases there are too many specializations in the Ruby code so YJIT exits. Co-authored-by: Aaron Patterson tenderlove@ruby-lang.org
* Fix link (#5208)Burdette Lamar2021-12-031-1/+1
|
* Refactor GC functions to have consistent namingPeter Zhu2021-12-031-18/+18
| | | | | Refactor function names for consistency. Function with name xyz_page should have a corresponding function named xyz_plane.
* * 2021-12-04 [ci skip]git2021-12-041-1/+1
|
* Add `nd_type_p` macroS.H2021-12-045-101/+102
|
* Adding links to literals and Kernel (#5192)Burdette Lamar2021-12-0310-58/+102
| | | | * Adding links to literals and Kernel
* TestClass#test_subclass_gc reduce the number of iteration by 10xJean Boussier2021-12-031-2/+2
| | | | | The test was taking 10 seconds on my machine and did timeout on CI once.
* [rubygems/rubygems] Don't unnecessarily loop twice through dependenciesDavid Rodríguez2021-12-031-4/+2
| | | | https://github.com/rubygems/rubygems/commit/06b4a7994d
* [rubygems/rubygems] Improve source gemfile/lockfile equivalence checksDavid Rodríguez2021-12-032-25/+15
| | | | | | | | | | Since we no longer have multiple global sources, each top level dependency is always pinned to a single source, so it makes little sense to talk about adding or removing a source. Instead, source changes always mean to change the source one or more dependencies are pinned to. This logic can now be much simpler. https://github.com/rubygems/rubygems/commit/f1d33fa0df
* [rubygems/rubygems] Improve sources representationDavid Rodríguez2021-12-039-21/+26
| | | | | | | | | | | | | We have two representations of a source. Once used for sorting, which should not depend on the source's state, but solely on its static information, like remotes. Another one used for error and informational messages, which should properly inform about the exact state of the source when the message is printed. This commit makes the latter be the default implementation of `to_s`, so that error and informational messages are more accurate by default. https://github.com/rubygems/rubygems/commit/b5f2b88957
* [rubygems/rubygems] Don't overwrite locked dependency sources too earlyDavid Rodríguez2021-12-032-2/+2
| | | | | | Otherwise we hide some useful message about dependency source changes. https://github.com/rubygems/rubygems/commit/c926673c5b
* [rubygems/rubygems] Fix incorrect order in changed sources messageDavid Rodríguez2021-12-032-2/+2
| | | | https://github.com/rubygems/rubygems/commit/6f1b5f68de
* [rubygems/rubygems] Remove unnecessary codeDavid Rodríguez2021-12-031-6/+0
| | | | | | | | Somehow this is trying to relax frozen mode constraints for path sources. It doesn't make sense to me and it's not covered by any spec so I'm killing it. https://github.com/rubygems/rubygems/commit/17c978e161
* [rubygems/rubygems] Reuse `locked_dependencies` helperDavid Rodríguez2021-12-031-1/+1
| | | | | | It makes the code more consistent with the above line. https://github.com/rubygems/rubygems/commit/f28d05a548
* [rubygems/rubygems] Reformat for consistency with the above lineDavid Rodríguez2021-12-031-3/+1
| | | | https://github.com/rubygems/rubygems/commit/11193be3f1
* [rubygems/rubygems] Remove unnecessary nil checksDavid Rodríguez2021-12-031-2/+2
| | | | https://github.com/rubygems/rubygems/commit/d047b8935d
* Add tests of string argument to Time.atNobuyoshi Nakada2021-12-031-0/+2
|
* Update bundled_gems at 2021-12-03git2021-12-031-1/+1
|
* [DOC] Fix Time.new description [ci skip]Nobuyoshi Nakada2021-12-031-1/+1
|
* Lazily create singletons on instance_{exec,eval} (#5146)John Hawthorn2021-12-0210-78/+197
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Lazily create singletons on instance_{exec,eval} Previously when instance_exec or instance_eval was called on an object, that object would be given a singleton class so that method definitions inside the block would be added to the object rather than its class. This commit aims to improve performance by delaying the creation of the singleton class unless/until one is needed for method definition. Most of the time instance_eval is used without any method definition. This was implemented by adding a flag to the cref indicating that it represents a singleton of the object rather than a class itself. In this case CREF_CLASS returns the object's existing class, but in cases that we are defining a method (either via definemethod or VM_SPECIAL_OBJECT_CBASE which is used for undef and alias). This also happens to fix what I believe is a bug. Previously instance_eval behaved differently with regards to constant access for true/false/nil than for all other objects. I don't think this was intentional. String::Foo = "foo" "".instance_eval("Foo") # => "foo" Integer::Foo = "foo" 123.instance_eval("Foo") # => "foo" TrueClass::Foo = "foo" true.instance_eval("Foo") # NameError: uninitialized constant Foo This also slightly changes the error message when trying to define a method through instance_eval on an object which can't have a singleton class. Before: $ ruby -e '123.instance_eval { def foo; end }' -e:1:in `block in <main>': no class/module to add method (TypeError) After: $ ./ruby -e '123.instance_eval { def foo; end }' -e:1:in `block in <main>': can't define singleton (TypeError) IMO this error is a small improvement on the original and better matches the (both old and new) message when definging a method using `def self.` $ ruby -e '123.instance_eval{ def self.foo; end }' -e:1:in `block in <main>': can't define singleton (TypeError) Co-authored-by: Matthew Draper <matthew@trebex.net> * Remove "under" argument from yield_under * Move CREF_SINGLETON_SET into vm_cref_new * Simplify vm_get_const_base * Fix leaf VM_SPECIAL_OBJECT_CONST_BASE Co-authored-by: Matthew Draper <matthew@trebex.net>
* [rubygems/rubygems] Move required_ruby_version gemspec attribute to ↵Josef Šimánek2021-12-031-5/+9
| | | | | | recommended section. https://github.com/rubygems/rubygems/commit/de6552ac30
* [Bug #18382] Fix crash in compaction for ObjectSpace.trace_object_allocationsPeter Zhu2021-12-021-1/+2
| | | | | ObjectSpace.trace_object_allocations can crash when auto-compaction is enabled.
* YJIT: Fix side-exit typo in comments [ci skip]Adam Hess2021-12-021-12/+12
|
* [ruby/irb] Examine indentation of in keyword when trying to type includeKaíque Kandy Koga2021-12-032-1/+38
| | | | | | | | Use in_keyword_case_scope? Return fast https://github.com/ruby/irb/commit/8acc7f8dc7
* [ruby/irb] Add Alt+d keycode when convert-meta isn't usedima1zumi2021-12-031-0/+1
| | | | | | | I pushed reline#389 for when convert-meta is not turned on in .inputrc. Alt+D in irb also needs to be set to the keycode for not using convert-meta. https://github.com/ruby/irb/commit/328eddf851
* [ruby/irb] Fix bug infinite loop when pasting multilines fo code in Ruby 2.6manga_osyo2021-12-031-1/+1
| | | | | | | | Fix bug infinite loop when pasting multilines fo code in Ruby 2.6. This is not reproduced in Ruby 2.7. Changes added in https://github.com/ruby/irb/pull/242/files#diff-612b926e42ed78aed1a889ac1944f7d22229b3a489cc08f837a7f75eca3d3399R155 are also reflected in Ruby 2.6. https://github.com/ruby/irb/commit/0a77f75bf0
* [ruby/reline] No need to check `x > 0`ima1zumi2021-12-031-2/+2
| | | | | | Checked in the previous line. https://github.com/ruby/reline/commit/bf774c0f2c
* * 2021-12-03 [ci skip]git2021-12-031-1/+1
|
* Don't clear the constant cache when finishing compactionMatt Valentine-House2021-12-021-1/+0
| | | | References are being updated correctly, so this is no longer necessary
* [ruby/reline] fix `TerminfoError` -> `Reline::Terminfo::TerminfoError`ima1zumi2021-12-021-3/+3
| | | | https://github.com/ruby/reline/commit/c07b0ace6a
* [ruby/reline] Rescue if tigetstr(capname) cannot be obtainedima1zumi2021-12-021-16/+40
| | | | | | | | | fix https://github.com/ruby/reline/issues/384 If `$TERM` is `vt102`, there are no `kend`, `khome`, `civis`, or `cnorm` in capabilities. `TerminfoError` is raised in `Reline::Terminfo.tigetstr(capname)`, so it is rescued if it does not exist. https://github.com/ruby/reline/commit/c9f5112702
* Cache downloaded filesNobuyoshi Nakada2021-12-0210-0/+42
| | | | | | | | | Cache downloaded external libraries/gems, which are expected not changed so frequently. Also sometimes downloading from zlib returns the current time as the date header in unexpected format, and checksums mismatch at that time.
* update-bundled_gems.rb: Fix typo and version comparisonNobuyoshi Nakada2021-12-021-1/+1
|
* Define Ripper::Lexer::Elem#to_sNobuyoshi Nakada2021-12-021-0/+2
| | | | | | Alias `#inspect` as `#to_s` also in the new `Ripper::Lexer::Elem` class, so that `puts Ripper::Lexer.new(code).scan` shows the attributes.
* [rubygems/rubygems] This warrants a commentDavid Rodríguez2021-12-021-0/+14
| | | | https://github.com/rubygems/rubygems/commit/8fa29e5e55
* [rubygems/rubygems] Fix gemspec source unlocking also for prereleases like ↵David Rodríguez2021-12-022-1/+34
| | | | | | | | | 0.0.0.SNAPSHOT The default prerelease requirement in rubygems doesn't actually match things like "0.0.0.SNAPSHOT". https://github.com/rubygems/rubygems/commit/711498b342
* [rubygems/rubygems] Fix materialization of locked 0 prereleasesDavid Rodríguez2021-12-022-1/+36
| | | | | | | | | | Since the default requirement in rubygems is ">= 0", it was failing to match 0 prereleases. Changing the default globally to be ">= 0.a" instead is a major refactoring that's quite tricky to make backwards compatible, so I'm special casing this where needed for now to fix the regression. https://github.com/rubygems/rubygems/commit/68fe37937c
* [MSWin] Stop passing non-numeric string to `exit` commandNobuyoshi Nakada2021-12-021-2/+1
|
* Deprecate `Lexer::Elem#[]` and `Lexer::State#[]`schneems2021-12-021-0/+31
| | | | | | | Discussed in https://github.com/ruby/ruby/pull/5093#issuecomment-964426481. > it would be enough to mimic only [] for almost all cases This adds back the `Lexer::Elem#[]` and `Lexer::State#[]` and adds deprecation warnings for them.
* Compatibility with IRBschneems2021-12-023-67/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of accessing the struct as an array, access it via methods. There are other places inside of this file already using this API (for example https://github.com/ruby/ruby/blob/e0a5c3d2b71dfad038d7562fdd33f02ffd79232d/lib/irb/ruby-lex.rb#L829-L830). This commit moves all struct array-ish calls to use their method calls instead. It is also ~1.23 faster accessing values via a method instead of as an array according to this microbenchmark: ```ruby Elem = Struct.new(:pos, :event, :tok, :state, :message) do def initialize(pos, event, tok, state, message = nil) super(pos, event, tok, State.new(state), message) end # ... def to_a a = super a.pop unless a.empty? a end end class ElemClass attr_accessor :pos, :event, :tok, :state, :message def initialize(pos, event, tok, state, message = nil) @pos = pos @event = event @tok = tok @state = State.new(state) @message = message end def to_a if @message [@pos, @event, @tok, @state, @message] else [@pos, @event, @tok, @state] end end end # stub state class creation for now class State; def initialize(val); end; end ``` ```ruby Benchmark.ips do |x| x.report("struct") { struct[1] } x.report("class ") { from_class.event } x.compare! end; nil ``` ``` Warming up -------------------------------------- struct 1.624M i/100ms class 1.958M i/100ms Calculating ------------------------------------- struct 17.139M (± 2.6%) i/s - 86.077M in 5.025801s class 21.104M (± 3.4%) i/s - 105.709M in 5.015193s Comparison: class : 21103826.3 i/s struct: 17139201.5 i/s - 1.23x (± 0.00) slower ```
* Only iterate Lexer heredoc arraysschneems2021-12-021-10/+12
| | | | The last element in the `@buf` may be either an array or an `Elem`. In the case it is an `Elem` we iterate over every element, when we do not need to. This check guards that case by ensuring that we only iterate over an array of elements.
* ~1.10x faster Change Ripper.lex structs to classesschneems2021-12-021-8/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## Concept I am proposing we replace the Struct implementation of data structures inside of ripper with real classes. This will improve performance and the implementation is not meaningfully more complicated. ## Example Struct versus class comparison: ```ruby Elem = Struct.new(:pos, :event, :tok, :state, :message) do def initialize(pos, event, tok, state, message = nil) super(pos, event, tok, State.new(state), message) end # ... def to_a a = super a.pop unless a.empty? a end end class ElemClass attr_accessor :pos, :event, :tok, :state, :message def initialize(pos, event, tok, state, message = nil) @pos = pos @event = event @tok = tok @state = State.new(state) @message = message end def to_a if @message [@pos, @event, @tok, @state, @message] else [@pos, @event, @tok, @state] end end end # stub state class creation for now class State; def initialize(val); end; end ``` ## MicroBenchmark creation ```ruby require 'benchmark/ips' require 'ripper' pos = [1, 2] event = :on_nl tok = "\n".freeze state = Ripper::EXPR_BEG Benchmark.ips do |x| x.report("struct") { Elem.new(pos, event, tok, state) } x.report("class ") { ElemClass.new(pos, event, tok, state) } x.compare! end; nil ``` Gives ~1.2x faster creation: ``` Warming up -------------------------------------- struct 263.983k i/100ms class 303.367k i/100ms Calculating ------------------------------------- struct 2.638M (± 5.9%) i/s - 13.199M in 5.023460s class 3.171M (± 4.6%) i/s - 16.078M in 5.082369s Comparison: class : 3170690.2 i/s struct: 2638493.5 i/s - 1.20x (± 0.00) slower ``` ## MicroBenchmark `to_a` (Called by Ripper.lex for every element) ```ruby require 'benchmark/ips' require 'ripper' pos = [1, 2] event = :on_nl tok = "\n".freeze state = Ripper::EXPR_BEG struct = Elem.new(pos, event, tok, state) from_class = ElemClass.new(pos, event, tok, state) Benchmark.ips do |x| x.report("struct") { struct.to_a } x.report("class ") { from_class.to_a } x.compare! end; nil ``` Gives 1.46x faster `to_a`: ``` Warming up -------------------------------------- struct 612.094k i/100ms class 893.233k i/100ms Calculating ------------------------------------- struct 6.121M (± 5.4%) i/s - 30.605M in 5.015851s class 8.931M (± 7.9%) i/s - 44.662M in 5.039733s Comparison: class : 8930619.0 i/s struct: 6121358.9 i/s - 1.46x (± 0.00) slower ``` ## MicroBenchmark data access ```ruby require 'benchmark/ips' require 'ripper' pos = [1, 2] event = :on_nl tok = "\n".freeze state = Ripper::EXPR_BEG struct = Elem.new(pos, event, tok, state) from_class = ElemClass.new(pos, event, tok, state) Benchmark.ips do |x| x.report("struct") { struct.pos[1] } x.report("class ") { from_class.pos[1] } x.compare! end; nil ``` Gives ~1.17x faster data access: ``` Warming up -------------------------------------- struct 1.694M i/100ms class 1.868M i/100ms Calculating ------------------------------------- struct 16.149M (± 6.8%) i/s - 81.318M in 5.060633s class 18.886M (± 2.9%) i/s - 95.262M in 5.048359s Comparison: class : 18885669.6 i/s struct: 16149255.8 i/s - 1.17x (± 0.00) slower ``` ## Full benchmark integration of this inside of Ripper.lex Inside of this repo with this commit ``` $ cd ext/ripper $ make $ cat test.rb file = File.join(__dir__, "../../array.rb") source = File.read(file) bench = Benchmark.measure do 10_000.times.each do Ripper.lex(source) end end puts bench ``` Then execute with and without this change 50 times: ``` rm new.txt rm old.txt for i in {0..50} do `ruby -Ilib -rripper -rbenchmark ./test.rb >> new.txt` `ruby -rripper -rbenchmark ./test.rb >> old.txt` done ``` I used derailed benchmarks internals to compare the results: ``` dir = Pathname(".") branch_info = {} branch_info["old"] = { desc: "Struct lex", time: Time.now, file: dir.join("old.txt"), name: "old" } branch_info["new"] = { desc: "Class lex", time: Time.now, file: dir.join("new.txt"), name: "new" } stats = DerailedBenchmarks::StatsFromDir.new(branch_info) stats.call.banner ``` Which gave us: ``` ❤️ ❤️ ❤️ (Statistically Significant) ❤️ ❤️ ❤️ [new] (3.3139 seconds) "Class lex" ref: "new" FASTER 🚀🚀🚀 by: 1.1046x [older/newer] 9.4700% [(older - newer) / older * 100] [old] (3.6606 seconds) "Struct lex" ref: "old" Iterations per sample: Samples: 51 Test type: Kolmogorov Smirnov Confidence level: 99.0 % Is significant? (max > critical): true D critical: 0.30049534876137013 D max: 0.9607843137254902 Histograms (time ranges are in seconds): [new] description: [old] description: "Class lex" "Struct lex" ┌ ┐ ┌ ┐ [3.0, 3.3) ┤▇ 1 [3.0, 3.3) ┤ 0 [3.3, 3.6) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 47 [3.3, 3.6) ┤ 0 [3.5, 3.8) ┤▇▇ 2 [3.5, 3.8) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 46 [3.8, 4.1) ┤▇ 1 [3.8, 4.1) ┤▇▇▇ 4 [4.0, 4.3) ┤ 0 [4.0, 4.3) ┤ 0 [4.3, 4.6) ┤ 0 [4.3, 4.6) ┤▇ 1 └ ┘ └ ┘ # of runs in range # of runs in range ``` To sum this up, the "new" version of this code (using real classes instead of structs) is 10% faster across 50 runs with a statistical significance confidence level of 99%. Histograms are for visual checksum.
View Lorry Controller Status