[ruby-core:113572] [Ruby master Bug#19683] ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby
Issue #19683 has been reported by jeremyevans0 (Jeremy Evans). ---------------------------------------- Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby https://bugs.ruby-lang.org/issues/19683 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0preview1 (2023-05-12 master a1b01e7701) [x86_64-openbsd7.3] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ruby-3.3.0-preview1 does not build with BSD make without `--with-baseruby`. However, it builds fine with GNU make and when using BSD make with `--with-baseruby`. Previous Ruby tarball releases have built fine with BSD make without `--with-baseruby`, so this issue has been introduced recently. When building with BSD make without `--with-baseruby`, building fails after generating RI format with: ``` /bin/sh: false: not found *** Error 127 in . (exts.mk:94 'ruby': @make UPDATE_LIBRARIES=no EXTENCS=dmyenc.o BASERUBY=echo\ executable\ host\ ruby\ is\ required.\ \ ...) *** Error 2 in /home/jeremy/local/ruby-3.3.0-preview1 (Makefile:948 'build-ext': @make -f exts.mk libdir="/usr/local/lib" LIBRUBY_EXTS=./...) ``` -- https://bugs.ruby-lang.org/
Issue #19683 has been updated by jeremyevans0 (Jeremy Evans). This is still a problem in 3.3.0-preview2. I've been using the following bad hack to work around it: ```diff Index: common.mk --- common.mk.orig +++ common.mk @@ -378,10 +378,10 @@ ext/configure-ext.mk: $(PREP) all-incs $(MKFILES) $(RB configure-ext: $(EXTS_MK) build-ext: $(EXTS_MK) - $(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \ + -$(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \ EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \ UPDATE_LIBRARIES=no $(EXTSTATIC) - $(Q)$(MAKE) $(EXTS_NOTE) + -$(Q)$(MAKE) $(EXTS_NOTE) exts-note: $(EXTS_MK) $(Q)$(MAKE) $(EXTS_NOTE) ``` ---------------------------------------- Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby https://bugs.ruby-lang.org/issues/19683#change-104634 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0preview1 (2023-05-12 master a1b01e7701) [x86_64-openbsd7.3] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ruby-3.3.0-preview1 does not build with BSD make without `--with-baseruby`. However, it builds fine with GNU make and when using BSD make with `--with-baseruby`. Previous Ruby tarball releases have built fine with BSD make without `--with-baseruby`, so this issue has been introduced recently. When building with BSD make without `--with-baseruby`, building fails after generating RI format with: ``` /bin/sh: false: not found *** Error 127 in . (exts.mk:94 'ruby': @make UPDATE_LIBRARIES=no EXTENCS=dmyenc.o BASERUBY=echo\ executable\ host\ ruby\ is\ required.\ \ ...) *** Error 2 in /home/jeremy/local/ruby-3.3.0-preview1 (Makefile:948 'build-ext': @make -f exts.mk libdir="/usr/local/lib" LIBRUBY_EXTS=./...) ``` -- https://bugs.ruby-lang.org/
Issue #19683 has been updated by jeremyevans0 (Jeremy Evans). Target version set to 3.3 This was still a problem in 3.3.0-preview3, and is still a problem in 3.3.0-rc1. It should definitely be fixed before the final release of 3.3.0. ---------------------------------------- Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby https://bugs.ruby-lang.org/issues/19683#change-105628 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Target version: 3.3 * ruby -v: ruby 3.3.0preview1 (2023-05-12 master a1b01e7701) [x86_64-openbsd7.3] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ruby-3.3.0-preview1 does not build with BSD make without `--with-baseruby`. However, it builds fine with GNU make and when using BSD make with `--with-baseruby`. Previous Ruby tarball releases have built fine with BSD make without `--with-baseruby`, so this issue has been introduced recently. When building with BSD make without `--with-baseruby`, building fails after generating RI format with: ``` /bin/sh: false: not found *** Error 127 in . (exts.mk:94 'ruby': @make UPDATE_LIBRARIES=no EXTENCS=dmyenc.o BASERUBY=echo\ executable\ host\ ruby\ is\ required.\ \ ...) *** Error 2 in /home/jeremy/local/ruby-3.3.0-preview1 (Makefile:948 'build-ext': @make -f exts.mk libdir="/usr/local/lib" LIBRUBY_EXTS=./...) ``` -- https://bugs.ruby-lang.org/
Issue #19683 has been updated by jeremyevans0 (Jeremy Evans). I traced this issue to commit:a1b01e7701f9fc370f8dff777aad6d39a2c5a3e3 (lrama), because that introduced the use of BASERUBY, which is what currently breaks this: ```diff --- a/common.mk +++ b/common.mk @@ -298,7 +298,8 @@ configure-ext: $(EXTS_MK) build-ext: $(EXTS_MK) $(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \ - EXTENCS="$(ENCOBJS)" MINIRUBY="$(MINIRUBY)" UPDATE_LIBRARIES=no $(EXTSTATIC) + EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \ + UPDATE_LIBRARIES=no $(EXTSTATIC) $(Q)$(MAKE) $(EXTS_NOTE) exts-note: $(EXTS_MK) @@ -1208,8 +1209,8 @@ $(srcdir)/ext/ripper/ripper.c: $(srcdir)/ext/ripper/tools/preproc.rb $(srcdir)/p $(Q) $(CHDIR) $(@D) && \ $(CAT_DEPEND) depend | \ $(exec) $(MAKE) -f - $(mflags) \ - Q=$(Q) ECHO=$(ECHO) RM="$(RM1)" BISON="$(YACC)" top_srcdir=../.. srcdir=. VPATH=../.. \ - RUBY="$(BASERUBY)" PATH_SEPARATOR="$(PATH_SEPARATOR)" LANG=C + Q=$(Q) ECHO=$(ECHO) RM="$(RM1)" top_srcdir=../.. srcdir=. VPATH=../.. \ + RUBY="$(BASERUBY)" BASERUBY="$(BASERUBY)" PATH_SEPARATOR="$(PATH_SEPARATOR)" LANG=C ``` This isn't necessarily specific to BASERUBY, though. The underlying issue is that `exts.mk` (created from `templates/exts.mk.tmpl`) uses `$(MFLAGS)`: ``` ruby: $(Q)$(MAKE) $(MFLAGS) $(SUBMAKEOPTS) $@ ``` This doesn't work as the `;` inside BASERUBY (defined as `echo executable host ruby is required. use --with-baseruby option.; false`) is not quoted. I'm not sure how to get it quoted correctly (neither the `Q` or `QL` modifiers work), I'm checking with the OpenBSD make maintainer. One possible way to workaround the issue would be to ship a `tool/missing-baseruby` file that contained: ``` #!/bin/sh echo "executable host ruby is required. use --with-baseruby option." exit 1 ``` And use that as `BASERUBY`. However, I think the best fix would be to not use `BASERUBY` here. This is after `miniruby` and `ruby` are built, we should be able to use the built ruby instead of `BASERUBY` when building extensions, unless I'm missing something. ---------------------------------------- Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby https://bugs.ruby-lang.org/issues/19683#change-105718 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Target version: 3.3 * ruby -v: ruby 3.3.0preview1 (2023-05-12 master a1b01e7701) [x86_64-openbsd7.3] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ruby-3.3.0-preview1 does not build with BSD make without `--with-baseruby`. However, it builds fine with GNU make and when using BSD make with `--with-baseruby`. Previous Ruby tarball releases have built fine with BSD make without `--with-baseruby`, so this issue has been introduced recently. When building with BSD make without `--with-baseruby`, building fails after generating RI format with: ``` /bin/sh: false: not found *** Error 127 in . (exts.mk:94 'ruby': @make UPDATE_LIBRARIES=no EXTENCS=dmyenc.o BASERUBY=echo\ executable\ host\ ruby\ is\ required.\ \ ...) *** Error 2 in /home/jeremy/local/ruby-3.3.0-preview1 (Makefile:948 'build-ext': @make -f exts.mk libdir="/usr/local/lib" LIBRUBY_EXTS=./...) ``` -- https://bugs.ruby-lang.org/
Issue #19683 has been updated by jeremyevans0 (Jeremy Evans). I tried the following patch and I was able to build master (which requires BASERUBY) and 3.3.0-rc1 (which does not) with it using OpenBSD make: ```diff diff --git a/common.mk b/common.mk index 0125645e37..6c637e0936 100644 --- a/common.mk +++ b/common.mk @@ -395,8 +395,7 @@ configure-ext: $(EXTS_MK) build-ext: $(EXTS_MK) $(Q)$(MAKE) -f $(EXTS_MK) $(mflags) libdir="$(libdir)" LIBRUBY_EXTS=$(LIBRUBY_EXTS) \ - EXTENCS="$(ENCOBJS)" BASERUBY="$(BASERUBY)" MINIRUBY="$(MINIRUBY)" \ - UPDATE_LIBRARIES=no $(EXTSTATIC) + EXTENCS="$(ENCOBJS)" MINIRUBY="$(MINIRUBY)" UPDATE_LIBRARIES=no $(EXTSTATIC) $(Q)$(MAKE) $(EXTS_NOTE) exts-note: $(EXTS_MK) ``` It passed on CI except on Windows when compiling with Visual Studio. It failed on Windows due to `ripper`. `ext/ripper/depend` uses `BASERUBY`: ``` BISON = $(BASERUBY) $(top_srcdir)/tool/lrama/exe/lrama ``` This cannot be easily switched to `RUBY`, because `lrama` requires `strscan`, and `strscan` is currently not built until after `ripper`. Even if you work around that to build `strscan` before `ripper`, `RUBY` here does not allow loading the `strscan` extension library: ``` ../../miniruby -I'../..' -I'../.././lib' -I'../../.ext/x86_64-openbsd7.4' -I'../../.ext/common' ../.././tool/lrama/exe/lrama -t -v -oripper.c - ripper.y < ripper.y /home/jeremy/tmp/ruby/tool/lrama/lib/lrama/lexer.rb:1:in `require': this executable file can't load extension libraries (LoadError) ``` For Ruby 3.3.0, the `tool/missing-baseruby` approach I mentioned earlier appears to work. I submitted a pull request for that approach: https://github.com/ruby/ruby/pull/9299 Longer term alternative possible solutions to avoid need for `BASERUBY` in `build-ext` task for use by `ripper`: * Change `lrama` so it doesn't need `strscan` and can use `RUBY` instead of `BASERUBY` * Create `ripper.y.c` when `parse.y.c` is created, at top level * Maybe `ripper` will be removed or replaced by prism and will not need lrama? ---------------------------------------- Bug #19683: ruby-3.3.0-preview1 does not build with BSD make without --with-baseruby https://bugs.ruby-lang.org/issues/19683#change-105742 * Author: jeremyevans0 (Jeremy Evans) * Status: Open * Priority: Normal * Target version: 3.3 * ruby -v: ruby 3.3.0preview1 (2023-05-12 master a1b01e7701) [x86_64-openbsd7.3] * Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- ruby-3.3.0-preview1 does not build with BSD make without `--with-baseruby`. However, it builds fine with GNU make and when using BSD make with `--with-baseruby`. Previous Ruby tarball releases have built fine with BSD make without `--with-baseruby`, so this issue has been introduced recently. When building with BSD make without `--with-baseruby`, building fails after generating RI format with: ``` /bin/sh: false: not found *** Error 127 in . (exts.mk:94 'ruby': @make UPDATE_LIBRARIES=no EXTENCS=dmyenc.o BASERUBY=echo\ executable\ host\ ruby\ is\ required.\ \ ...) *** Error 2 in /home/jeremy/local/ruby-3.3.0-preview1 (Makefile:948 'build-ext': @make -f exts.mk libdir="/usr/local/lib" LIBRUBY_EXTS=./...) ``` -- https://bugs.ruby-lang.org/
participants (2)
-
jeremyevans0 (Jeremy Evans) -
jeremyevans0 (Jeremy Evans)