Issue #19465 has been reported by normalperson (Eric Wong).
----------------------------------------
Feature #19465: [PATCH] reuse open(2) from rb_file_load_ok on POSIX-like system
https://bugs.ruby-lang.org/issues/19465
* Author: normalperson (Eric Wong)
* Status: Open
* Priority: Normal
----------------------------------------
```
When loading Ruby source files, we can save the result of
successful opens as open(2)/openat(2) are a fairly expensive
syscalls. This also avoids a time-of-check-to-time-of-use
(TOCTTOU) problem.
This reduces open(2) syscalls during `require'; but should be
most apparent when users have a small $LOAD_PATH. Users with
large $LOAD_PATH will benefit less since there'll be more
open(2) failures due to ENOENT.
With `strace -c -e openat ruby -e exit' under Linux, this
results in a ~14% reduction of openat(2) syscalls
(glibc uses openat(2) to implement open(2)).
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
0.00 0.000000 0 296 110 openat
0.00 0.000000 0 254 110 openat
Additionally, the introduction of `struct ruby_file_load_state'
may make future optimizations more apparent.
This change cannot benefit binary (.so) loading since the
dlopen(3) API requires a filename and I'm not aware of an
alternative that takes a pre-existing FD. In typical
situations, Ruby source files outnumber the mount of .so
files.
I've only tested this lightly on small apps since I don't have
large codebases to test on. However, I think organizing various
on-stack variables into `struct ruby_file_load_state' can be
beneficial if we end up using io-uring on Linux.
```
---Files--------------------------------
0001-reuse-open-2-from-rb_file_load_ok-on-POSIX-like-syst.patch (15.2 KB)
--
https://bugs.ruby-lang.org/
Issue #16694 has been updated by k0kubun (Takashi Kokubun).
Sorry for not fixing this for a while, but I replaced the implementation in [Feature #19420] and it should indeed remove this problem. Closing.
----------------------------------------
Bug #16694: JIT vs hardened GCC with PCH
https://bugs.ruby-lang.org/issues/16694#change-102191
* Author: vo.x (Vit Ondruch)
* Status: Assigned
* Priority: Normal
* Assignee: k0kubun (Takashi Kokubun)
* ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
Preparing Ruby package for RHEL 8, I observe the following error:
~~~
$ cd /builddir/build/BUILD/ruby-2.6.3/
$ make test-all TESTS=test/ruby/test_rubyvm_mjit.rb
Run options: "--ruby=./miniruby -I./lib -I. -I.ext/common ./tool/runruby.rb --extout=.ext -- --disable-gems" --excludes-dir=./test/excludes --name=!/memory_leak/
# Running tests:
[1/4] TestRubyVMMJIT#test_pause = 0.24 s
1) Failure:
TestRubyVMMJIT#test_pause [/builddir/build/BUILD/ruby-2.6.3/test/ruby/test_rubyvm_mjit.rb:32]:
unexpected stdout:
'''
truefalsefalse```
stderr:
'''
/tmp/_ruby_mjit_p712u0.c:1:37: error: one or more PCH files were found, but they were invalid
#include "/tmp/_ruby_mjit_hp712u0.h"
^
compilation terminated due to -Wfatal-errors.
/tmp/_ruby_mjit_p712u1.c:1:37: error: one or more PCH files were found, but they were invalid
#include "/tmp/_ruby_mjit_hp712u0.h"
^
compilation terminated due to -Wfatal-errors.
/tmp/_ruby_mjit_p712u2.c:1:37: error: one or more PCH files were found, but they were invalid
#include "/tmp/_ruby_mjit_hp712u0.h"
^
compilation terminated due to -Wfatal-errors.
/tmp/_ruby_mjit_p712u3.c:1:37: error: one or more PCH files were found, but they were invalid
#include "/tmp/_ruby_mjit_hp712u0.h"
^
compilation terminated due to -Wfatal-errors.
/tmp/_ruby_mjit_p712u4.c:1:37: error: one or more PCH files were found, but they were invalid
#include "/tmp/_ruby_mjit_hp712u0.h"
^
compilation terminated due to -Wfatal-errors.
Successful MJIT finish
```.
<5> expected but was
<0>.
Finished tests in 0.937667s, 4.2659 tests/s, 24.5290 assertions/s.
4 tests, 23 assertions, 1 failures, 0 errors, 0 skips
ruby -v: ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
make: *** [uncommon.mk:761: yes-test-all] Error 1
~~~
As it turns out, this is because GCC in RHEL is fully hardened. Unfortunately, due to GCC design, when GCC is fully hardened, it cannot properly handle PCH due to memory address relocation. Moreover, PCH are also security risk, so it seems they are going to be disabled entirely on RHEL.
Now I wonder what is the impact on Ruby JIT. I worry that with disabled PCH, the Ruby performance with JIT will be even worser without JIT. May be it is not good idea to use GCC for JIT. What are your thoughts?
The original ticket with all the details is here [1].
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1721553
--
https://bugs.ruby-lang.org/
Issue #18277 has been updated by k0kubun (Takashi Kokubun).
Status changed from Open to Closed
Closing as we replaced the implementation in [Feature #19420] and I believe this doesn't happen anymore.
----------------------------------------
Bug #18277: buffer error (Zlib::BufError) in Zlib::Deflate#deflate when using MJIT
https://bugs.ruby-lang.org/issues/18277#change-102189
* Author: Eregon (Benoit Daloze)
* Status: Closed
* Priority: Normal
* Assignee: k0kubun (Takashi Kokubun)
* ruby -v: ruby 3.1.0dev (2021-10-23T04:38:40Z master cda8bc3657) +JIT [x86_64-linux]
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
To repro:
```
$ git clone https://github.com/Shopify/yjit-bench.git
$ cd yjit-bench
$ ruby --jit -I./harness benchmarks/hexapdf/benchmark.rb
...
Bundle complete! 1 Gemfile dependency, 4 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
/home/eregon/prefix/ruby-master/lib/ruby/gems/3.1.0/gems/hexapdf-0.16.0/lib/hexapdf/filter/flate_decode.rb:94:in `deflate': buffer error (Zlib::BufError)
from /home/eregon/prefix/ruby-master/lib/ruby/gems/3.1.0/gems/hexapdf-0.16.0/lib/hexapdf/filter/flate_decode.rb:94:in `block in encoder'
```
This fails every time with MJIT, and it does not fail without MJIT or with YJIT (on ruby master @ cda8bc3657).
It fails on master MJIT: `ruby 3.1.0dev (2021-10-23T04:38:40Z master cda8bc3657) +JIT [x86_64-linux]`
It works with 3.0.2 MJIT: `ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) +JIT [x86_64-linux]`.
--
https://bugs.ruby-lang.org/
Issue #17995 has been updated by k0kubun (Takashi Kokubun).
Status changed from Open to Closed
Closing this since I replaced the implementation in [Feature #19420]. It doesn't mean that it works well with Ractor, but the current issue with Ractor should be reported separately since it's a completely different symptom.
----------------------------------------
Bug #17995: Slow down when mjit and Ractor are being used at same time
https://bugs.ruby-lang.org/issues/17995#change-102188
* Author: nekoyama32767 (Jinsong Yu)
* Status: Closed
* Priority: Normal
* Assignee: k0kubun (Takashi Kokubun)
* ruby -v: Ruby 3.0.1p64 (2021-04-05 revison 0fb782ee38)[x86-64-linux] ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x64-mingw32]
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
When using --jit and Ractor at same time, benchmark would be slow down with recursion function such like Tarai function an Fibonacci function.
The slow down is confirmed under both gcc(linux) and mingw64(windows MSYS2).
Test environment:
Cpu:Ryzen9 5900HS(a)3.3Ghz 8core/16thread
Memory:32GB@3200Mhz
OS:Ubuntu 21.04/Windows 10
Because of this cpu has 8 physic core,the parallel thread number is set to 8
Slow down is also confirmed with using only ractor (without sequence running).
When runing 'par' before 'seq' and using --mjit,'seq' is more slower than runing 'seq' berfore 'par'
Tarai function
```ruby
def tarai(x, y, z) =
x <= y ? y : tarai(tarai(x-1, y, z),
tarai(y-1, z, x),
tarai(z-1, x, y))
require 'benchmark'
Benchmark.bm do |x|
# sequential version
x.report('seq'){ 8.times{ tarai(14, 7, 0) } }
# parallel version
x.report('par'){
8.times.map do
Ractor.new { tarai(14, 7, 0) }
end.each(&:take)
}
end
```
Fibonacci function
```ruby
def fib(n)
if n==1 then return 1 end
if n==2 then return 1 end
fib(n-1)+fib(n-2)
end
require 'benchmark'
Benchmark.bm do |x|
# sequential version
x.report('fib:seq'){ 8.times{ fib(40) } }
# parallel version
x.report('fib:par '){
8.times.map do
Ractor.new {fib(40) }
end.each(&:take)
}
end
```
--
https://bugs.ruby-lang.org/
Issue #18808 has been updated by k0kubun (Takashi Kokubun).
Status changed from Assigned to Closed
I replaced the build process in [Feature #19420], and this issue seems irrelevant to today's build system. Closing.
----------------------------------------
Bug #18808: Cannot compile ruby 3.1.2 on powerpc64le-linux without disabling the jit features
https://bugs.ruby-lang.org/issues/18808#change-102187
* Author: npn (John Davis)
* Status: Closed
* Priority: Normal
* Assignee: k0kubun (Takashi Kokubun)
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
The system I am trying to compile on is a ppc64el system running RedHat 7 (3.10.0-1160.62.1.el7.ppc64le). I can successfully compile on a x86_64 system running the same version RedHat 7 (3.10.0-1160.62.1.el7.x86_64)
When I configure with the following, make completes fine, but if I remove the `--disable-jit-support` it fails with the following error.
./configure --enable-shared --enable-load-relative --disable-install-doc --prefix=$RUBY_PREFIX --exec-prefix=$RUBY_PREFIX/rh_ppc --disable-jit-support
building rb_mjit_header.h
rb_mjit_header.h updated
building .ext/include/powerpc64le-linux/rb_mjit_min_header-3.1.2.h
error in final header file:
In file included from /tmp/20220525-12786-q2ndz2.c:1:0:
/tmp/20220525-12786-vf3xbh.h:16627:1: error: multiple storage classes in declaration specifiers
__attribute__ ((__visibility__("default"))) extern
^
compilation terminated due to -Wfatal-errors.
make: *** [.ext/include/powerpc64le-linux/rb_mjit_min_header-3.1.2.h] Error 1
---Files--------------------------------
config.log (1.45 MB)
rb_mjit_min_header-3.2.0.h (1000 KB)
rb_mjit_header.h (1010 KB)
rb_mjit_header.h (1010 KB)
config.log (1.37 MB)
--
https://bugs.ruby-lang.org/
Issue #18058 has been updated by k0kubun (Takashi Kokubun).
Status changed from Assigned to Closed
I replaced the entire implementation in [Feature #19420], and I haven't seen this issue with today's implementation.
----------------------------------------
Bug #18058: 3.1.0-dev with MJIT enabled Zlib::BufError during `gem install`
https://bugs.ruby-lang.org/issues/18058#change-102186
* Author: byroot (Jean Boussier)
* Status: Closed
* Priority: Normal
* Assignee: k0kubun (Takashi Kokubun)
* ruby -v: ruby 3.1.0dev (2021-08-01T00:54:36Z master 605421f4eb) +JIT [x86_64-linux]
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN
----------------------------------------
It doesn't always happen (maybe 5-10% of the cases), but since I added MJIT to our CI builds I've been seeing this error a lot.
I was able to reproduce it in `docker run -it rubylang/ruby:master-nightly-bionic` after 15 or so tries:
```
# ruby -v
ruby 3.1.0dev (2021-08-01T00:54:36Z master 605421f4eb) [x86_64-linux]
# gem uninstall bundler -v 2.2.21; env RUBYOPT="--enable-jit --jit-max-cache=10000 --jit-min-calls=10 $RUBYOPT" gem install --no-document --user-install bundler -v 2.2.21
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Exception while verifying /usr/local/lib/ruby/gems/3.1.0/cache/bundler-2.2.21.gem
ERROR: While executing gem ... (Zlib::BufError)
buffer error
```
e.g.
```
$ docker run -it rubylang/ruby:master-nightly-bionic bash -c 'while env RUBYOPT="--enable-jit --jit-max-cache=10000 --jit-min-calls=10 $RUBYOPT" gem install --no-document --user-install bundler -v 2.2.21; do gem uninstall bundler -v 2.2.21; done'
Fetching bundler-2.2.21.gem
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Successfully installed bundler-2.2.21
1 gem installed
Successfully uninstalled bundler-2.2.21
WARNING: You don't have /root/.local/share/gem/ruby/3.1.0/bin in your PATH,
gem executables will not run.
Exception while verifying /usr/local/lib/ruby/gems/3.1.0/cache/bundler-2.2.21.gem
ERROR: While executing gem ... (Zlib::BufError)
buffer error
```
--
https://bugs.ruby-lang.org/
Issue #19481 has been reported by duerst (Martin Dürst).
----------------------------------------
Bug #19481: make: *** No rule to make target 'defs/universal.mk'. Stop.
https://bugs.ruby-lang.org/issues/19481
* Author: duerst (Martin Dürst)
* Status: Open
* Priority: Normal
* ruby -v: ruby 3.1.0dev (2021-06-03T06:59:33Z master 7e14762159) [x86_64-linux]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN
----------------------------------------
When I tried to compile Ruby with `make up; make`, I got the message in the Subject.
The line before said:
`GNUmakefile:30: defs/universal.mk: No such file or directory`
Below is the start of the output. git seems to correctly download new stuff. But apparently something is missing. I hope this problem can be fixed soon.
```
duerst@Kloentalersee:~/15ruby$ make up; make
remote: Enumerating objects: 4839, done.
remote: Counting objects: 100% (4839/4839), done.
remote: Compressing objects: 100% (2467/2467), done.
remote: Total 4839 (delta 3132), reused 2965 (delta 2049), pack-reused 0
Receiving objects: 100% (4839/4839), 3.20 MiB | 2.53 MiB/s, done.
Resolving deltas: 100% (3132/3132), completed with 391 local objects.
From git.ruby-lang.org:ruby
cb40432c74..f68580890f master -> origin/master
a4ab9921a1..0602df301c ruby_3_1 -> origin/ruby_3_1
31819e82c8..f93c7b9f58 ruby_3_2 -> origin/ruby_3_2
8f9b896b51..926b22ef5c refs/notes/commits -> refs/notes/commits
Updating cb40432c74..f68580890f
Fast-forward
```
--
https://bugs.ruby-lang.org/