[ruby-core:112560] [Ruby master Bug#19462] MJIT not enabled with universal macOS x86_64 + arm64 build

Issue #19462 has been reported by benhamilton (Ben Hamilton). ---------------------------------------- Bug #19462: MJIT not enabled with universal macOS x86_64 + arm64 build https://bugs.ruby-lang.org/issues/19462 * Author: benhamilton (Ben Hamilton) * Status: Open * Priority: Normal * ruby -v: ruby 3.3.0dev (2023-02-21T21:59:54Z master ae9e1aee59) [universal.x86_64-darwin22] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I'm trying to build a universal macOS x86_64 + arm64 ruby from upstream @ ae9e1aee59b0db1e61aa0473556165f9fd719cde on x86_64 macOS Ventura 13.2, Xcode 14.2, Apple clang version 14.0.0 (clang-1400.0.29.202). When I run `autoconf && ./configure --with-arch=x86_64,arm64`, MJIT support is `yes` and YJIT support is `no` (I assume YJIT is disabled because I'm building for arm64): ``` Configuration summary for ruby version 3.3.0 * Installation prefix: /usr/local * exec prefix: ${prefix} * arch: universal-darwin22 * site arch: ${arch} * RUBY_BASE_NAME: ruby * ruby lib prefix: ${libdir}/${RUBY_BASE_NAME} * site libraries path: ${rubylibprefix}/${sitearch} * vendor path: ${rubylibprefix}/vendor_ruby * target OS: darwin22 * compiler: clang * with thread: pthread * with coroutine: universal * enable shared libs: no * dynamic library ext: bundle * CFLAGS: -fdeclspec ${optflags} ${debugflags} ${warnflags} * LDFLAGS: -L. -fstack-protector-strong * DLDFLAGS: -Wl,-multiply_defined,suppress * optflags: -O3 -fno-fast-math * debugflags: -ggdb3 * warnflags: -Wall -Wextra -Wextra-tokens \ -Wdeprecated-declarations -Wdivision-by-zero \ -Wdiv-by-zero -Wimplicit-function-declaration \ -Wimplicit-int -Wmisleading-indentation \ -Wpointer-arith -Wshorten-64-to-32 \ -Wwrite-strings -Wold-style-definition \ -Wmissing-noreturn -Wno-cast-function-type \ -Wno-constant-logical-operand -Wno-long-long \ -Wno-missing-field-initializers \ -Wno-overlength-strings -Wno-parentheses-equality \ -Wno-self-assign -Wno-tautological-compare \ -Wno-unused-parameter -Wno-unused-value \ -Wunused-variable -Wundef * strip command: strip -A -n * install doc: rdoc * MJIT support: yes * YJIT support: no * man page type: doc * BASERUBY -v: ruby 2.6.10p210 (2022-04-12 revision 67958) \ [universal.x86_64-darwin22] ``` However, when I compile with `make -j2`, I see that it's disabled: ``` % make -j2 BASERUBY = /usr/bin/ruby --disable=gems CC = clang LD = ld LDSHARED = clang -dynamiclib CFLAGS = -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wundef -pipe -arch x86_64 -arch arm64 XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -fPIE -I. -I.ext/include/universal-darwin22 -I./include -I. -I./enc/unicode/15.0.0 CPPFLAGS = -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT DLDFLAGS = -Wl,-multiply_defined,suppress -fstack-protector-strong -Wl,-pie -framework CoreFoundation -arch x86_64 -arch arm64 SOLIBS = -ldl -lobjc -lpthread LANG = en_US.UTF-8 LC_ALL = LC_CTYPE = MFLAGS = - --jobserver-fds=3,4 -j RUSTC = rustc YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/Users/benhamilton/Developer/OpenSource/ruby/yjit/target/release/' ./yjit/src/lib.rs Apple clang version 14.0.0 (clang-1400.0.29.202) Target: x86_64-apple-darwin22.3.0 Thread model: posix InstalledDir: /Applications/Xcode_14.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin [snip] building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-x86_64.h building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-arm64.h This compiler (clang -fdeclspec -g -pipe -arch arm64 -w) looks not supported for MJIT. Giving up to generate MJIT header. This compiler (clang -fdeclspec -g -pipe -arch x86_64 -w) looks not supported for MJIT. Giving up to generate MJIT header. ``` The problem is here: * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/c... * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/t... * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/t... This logic runs `vm.c` through the C preprocessor and looks through the output for the `__clang__` preprocessor macro. However, `clang`'s C preprocessor only prints macro values with the `-dM` flag. We can add this flag, but it won't work with the MSVC compiler, so we need to only add it for gcc/clang. -- https://bugs.ruby-lang.org/

Issue #19462 has been updated by k0kubun (Takashi Kokubun). Status changed from Open to Assigned Assignee set to k0kubun (Takashi Kokubun) ---------------------------------------- Bug #19462: MJIT not enabled with universal macOS x86_64 + arm64 build https://bugs.ruby-lang.org/issues/19462#change-102013 * Author: benhamilton (Ben Hamilton) * Status: Assigned * Priority: Normal * Assignee: k0kubun (Takashi Kokubun) * ruby -v: ruby 3.3.0dev (2023-02-21T21:59:54Z master ae9e1aee59) [universal.x86_64-darwin22] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I'm trying to build a universal macOS x86_64 + arm64 ruby from upstream @ ae9e1aee59b0db1e61aa0473556165f9fd719cde on x86_64 macOS Ventura 13.2, Xcode 14.2, Apple clang version 14.0.0 (clang-1400.0.29.202). When I run `autoconf && ./configure --with-arch=x86_64,arm64`, MJIT support is `yes` and YJIT support is `no` (I assume YJIT is disabled because I'm building for arm64): ``` Configuration summary for ruby version 3.3.0 * Installation prefix: /usr/local * exec prefix: ${prefix} * arch: universal-darwin22 * site arch: ${arch} * RUBY_BASE_NAME: ruby * ruby lib prefix: ${libdir}/${RUBY_BASE_NAME} * site libraries path: ${rubylibprefix}/${sitearch} * vendor path: ${rubylibprefix}/vendor_ruby * target OS: darwin22 * compiler: clang * with thread: pthread * with coroutine: universal * enable shared libs: no * dynamic library ext: bundle * CFLAGS: -fdeclspec ${optflags} ${debugflags} ${warnflags} * LDFLAGS: -L. -fstack-protector-strong * DLDFLAGS: -Wl,-multiply_defined,suppress * optflags: -O3 -fno-fast-math * debugflags: -ggdb3 * warnflags: -Wall -Wextra -Wextra-tokens \ -Wdeprecated-declarations -Wdivision-by-zero \ -Wdiv-by-zero -Wimplicit-function-declaration \ -Wimplicit-int -Wmisleading-indentation \ -Wpointer-arith -Wshorten-64-to-32 \ -Wwrite-strings -Wold-style-definition \ -Wmissing-noreturn -Wno-cast-function-type \ -Wno-constant-logical-operand -Wno-long-long \ -Wno-missing-field-initializers \ -Wno-overlength-strings -Wno-parentheses-equality \ -Wno-self-assign -Wno-tautological-compare \ -Wno-unused-parameter -Wno-unused-value \ -Wunused-variable -Wundef * strip command: strip -A -n * install doc: rdoc * MJIT support: yes * YJIT support: no * man page type: doc * BASERUBY -v: ruby 2.6.10p210 (2022-04-12 revision 67958) \ [universal.x86_64-darwin22] ``` However, when I compile with `make -j2`, I see that it's disabled: ``` % make -j2 BASERUBY = /usr/bin/ruby --disable=gems CC = clang LD = ld LDSHARED = clang -dynamiclib CFLAGS = -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wundef -pipe -arch x86_64 -arch arm64 XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -fPIE -I. -I.ext/include/universal-darwin22 -I./include -I. -I./enc/unicode/15.0.0 CPPFLAGS = -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT DLDFLAGS = -Wl,-multiply_defined,suppress -fstack-protector-strong -Wl,-pie -framework CoreFoundation -arch x86_64 -arch arm64 SOLIBS = -ldl -lobjc -lpthread LANG = en_US.UTF-8 LC_ALL = LC_CTYPE = MFLAGS = - --jobserver-fds=3,4 -j RUSTC = rustc YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/Users/benhamilton/Developer/OpenSource/ruby/yjit/target/release/' ./yjit/src/lib.rs Apple clang version 14.0.0 (clang-1400.0.29.202) Target: x86_64-apple-darwin22.3.0 Thread model: posix InstalledDir: /Applications/Xcode_14.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin [snip] building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-x86_64.h building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-arm64.h This compiler (clang -fdeclspec -g -pipe -arch arm64 -w) looks not supported for MJIT. Giving up to generate MJIT header. This compiler (clang -fdeclspec -g -pipe -arch x86_64 -w) looks not supported for MJIT. Giving up to generate MJIT header. ``` The problem is here: * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/c... * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/t... * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/t... This logic runs `vm.c` through the C preprocessor and looks through the output for the `__clang__` preprocessor macro. However, `clang`'s C preprocessor only prints macro values with the `-dM` flag. We can add this flag, but it won't work with the MSVC compiler, so we need to only add it for gcc/clang. -- https://bugs.ruby-lang.org/

Issue #19462 has been updated by k0kubun (Takashi Kokubun). Status changed from Assigned to Closed The build process is replaced in [Feature #19420], so this issue should be irrelevant today. ---------------------------------------- Bug #19462: MJIT not enabled with universal macOS x86_64 + arm64 build https://bugs.ruby-lang.org/issues/19462#change-102185 * Author: benhamilton (Ben Hamilton) * Status: Closed * Priority: Normal * Assignee: k0kubun (Takashi Kokubun) * ruby -v: ruby 3.3.0dev (2023-02-21T21:59:54Z master ae9e1aee59) [universal.x86_64-darwin22] * Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN ---------------------------------------- I'm trying to build a universal macOS x86_64 + arm64 ruby from upstream @ ae9e1aee59b0db1e61aa0473556165f9fd719cde on x86_64 macOS Ventura 13.2, Xcode 14.2, Apple clang version 14.0.0 (clang-1400.0.29.202). When I run `autoconf && ./configure --with-arch=x86_64,arm64`, MJIT support is `yes` and YJIT support is `no` (I assume YJIT is disabled because I'm building for arm64): ``` Configuration summary for ruby version 3.3.0 * Installation prefix: /usr/local * exec prefix: ${prefix} * arch: universal-darwin22 * site arch: ${arch} * RUBY_BASE_NAME: ruby * ruby lib prefix: ${libdir}/${RUBY_BASE_NAME} * site libraries path: ${rubylibprefix}/${sitearch} * vendor path: ${rubylibprefix}/vendor_ruby * target OS: darwin22 * compiler: clang * with thread: pthread * with coroutine: universal * enable shared libs: no * dynamic library ext: bundle * CFLAGS: -fdeclspec ${optflags} ${debugflags} ${warnflags} * LDFLAGS: -L. -fstack-protector-strong * DLDFLAGS: -Wl,-multiply_defined,suppress * optflags: -O3 -fno-fast-math * debugflags: -ggdb3 * warnflags: -Wall -Wextra -Wextra-tokens \ -Wdeprecated-declarations -Wdivision-by-zero \ -Wdiv-by-zero -Wimplicit-function-declaration \ -Wimplicit-int -Wmisleading-indentation \ -Wpointer-arith -Wshorten-64-to-32 \ -Wwrite-strings -Wold-style-definition \ -Wmissing-noreturn -Wno-cast-function-type \ -Wno-constant-logical-operand -Wno-long-long \ -Wno-missing-field-initializers \ -Wno-overlength-strings -Wno-parentheses-equality \ -Wno-self-assign -Wno-tautological-compare \ -Wno-unused-parameter -Wno-unused-value \ -Wunused-variable -Wundef * strip command: strip -A -n * install doc: rdoc * MJIT support: yes * YJIT support: no * man page type: doc * BASERUBY -v: ruby 2.6.10p210 (2022-04-12 revision 67958) \ [universal.x86_64-darwin22] ``` However, when I compile with `make -j2`, I see that it's disabled: ``` % make -j2 BASERUBY = /usr/bin/ruby --disable=gems CC = clang LD = ld LDSHARED = clang -dynamiclib CFLAGS = -fdeclspec -O3 -fno-fast-math -ggdb3 -Wall -Wextra -Wextra-tokens -Wdeprecated-declarations -Wdivision-by-zero -Wdiv-by-zero -Wimplicit-function-declaration -Wimplicit-int -Wmisleading-indentation -Wpointer-arith -Wshorten-64-to-32 -Wwrite-strings -Wold-style-definition -Wmissing-noreturn -Wno-cast-function-type -Wno-constant-logical-operand -Wno-long-long -Wno-missing-field-initializers -Wno-overlength-strings -Wno-parentheses-equality -Wno-self-assign -Wno-tautological-compare -Wno-unused-parameter -Wno-unused-value -Wunused-variable -Wundef -pipe -arch x86_64 -arch arm64 XCFLAGS = -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fno-strict-overflow -fvisibility=hidden -DRUBY_EXPORT -fPIE -I. -I.ext/include/universal-darwin22 -I./include -I. -I./enc/unicode/15.0.0 CPPFLAGS = -D_XOPEN_SOURCE -D_DARWIN_C_SOURCE -D_DARWIN_UNLIMITED_SELECT -D_REENTRANT DLDFLAGS = -Wl,-multiply_defined,suppress -fstack-protector-strong -Wl,-pie -framework CoreFoundation -arch x86_64 -arch arm64 SOLIBS = -ldl -lobjc -lpthread LANG = en_US.UTF-8 LC_ALL = LC_CTYPE = MFLAGS = - --jobserver-fds=3,4 -j RUSTC = rustc YJIT_RUSTC_ARGS = --crate-name=yjit --crate-type=staticlib --edition=2021 -g -C lto=thin -C opt-level=3 -C overflow-checks=on '--out-dir=/Users/benhamilton/Developer/OpenSource/ruby/yjit/target/release/' ./yjit/src/lib.rs Apple clang version 14.0.0 (clang-1400.0.29.202) Target: x86_64-apple-darwin22.3.0 Thread model: posix InstalledDir: /Applications/Xcode_14.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin [snip] building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-x86_64.h building .ext/include/universal-darwin22/rb_mjit_min_header-3.3.0-arm64.h This compiler (clang -fdeclspec -g -pipe -arch arm64 -w) looks not supported for MJIT. Giving up to generate MJIT header. This compiler (clang -fdeclspec -g -pipe -arch x86_64 -w) looks not supported for MJIT. Giving up to generate MJIT header. ``` The problem is here: * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/c... * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/t... * https://github.com/ruby/ruby/blob/1fdaa0666086529b3aae2d509a2e71c4247c3a12/t... This logic runs `vm.c` through the C preprocessor and looks through the output for the `__clang__` preprocessor macro. However, `clang`'s C preprocessor only prints macro values with the `-dM` flag. We can add this flag, but it won't work with the MSVC compiler, so we need to only add it for gcc/clang. -- https://bugs.ruby-lang.org/
participants (3)
-
benhamilton (Ben Hamilton)
-
k0kubun (Takashi Kokubun)
-
k0kubun (Takashi Kokubun)