[ruby-core:106834] [Ruby master Bug#18439] Support YJIT for VC++

[ruby-core:106834] [Ruby master Bug#18439] Support YJIT for VC++

[usa (Usaku NAKAMURA)]

Issue #18439 has been reported by usa (Usaku NAKAMURA).

----------------------------------------
Bug #18439: Support YJIT for VC++
https://bugs.ruby-lang.org/issues/18439

* Author: usa (Usaku NAKAMURA)
* Status: Open
* Priority: Normal
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/

[ruby-core:106860] [Ruby master Bug#18439] Support YJIT for VC++

[k0kubun (Takashi Kokubun)]

Issue #18439 has been updated by k0kubun (Takashi Kokubun).

Assignee set to maximecb (Maxime Chevalier-Boisvert)

----------------------------------------
Bug #18439: Support YJIT for VC++
https://bugs.ruby-lang.org/issues/18439#change-95671

* Author: usa (Usaku NAKAMURA)
* Status: Open
* Priority: Normal
* Assignee: maximecb (Maxime Chevalier-Boisvert)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/

[ruby-core:106862] [Ruby master Bug#18439] Support YJIT for VC++

[jhawthorn (John Hawthorn)]

Issue #18439 has been updated by jhawthorn (John Hawthorn).


> YJIT requires OPT_DIRECT_THREADED_CODE or OPT_CALL_THREADED_CODE in rb_yjit_compile_iseq().

What option do we use under windows?

> Maybe ABI differs between VC++ and YJIT's expectation.

Yes. We would need to switch to the Windows calling convention, which is different enough to be a non-trivial change. It only passes 4 arguments via registers vs 6 in the System V ABI (and also uses different registers) as well as requiring 32 bytes of reserved stack space from the callee ("shadow space").

I think we could likely do this with minimal (but non-trivial) changes by:
* Reserving the 32 bytes of stack "shadow space" in the YJIT entrypoint
* Reserving another 16 bytes on the stack in the YJIT entrypoint for the 5th and 6th argument
  * This could be an opportunity to do the same and support more arguments on SysV as well
* Conditionally under Windows defining `C_ARG_REGS` as `RCX, RDX, R8, R9, [RSP+32], [RSP+40]` 
* Rewriting the two instructions (`gen_toregexp`/`gen_newhash`) whose codegen currently modifies the machine stack (we could use a relative stack location instead of push/pop, a callee-saved register, or introduce runtime helper methods)

However I'd like to hear the opinions of YJIT folks when they're back from holidays next week on whether this is the best way to approach this.

----------------------------------------
Bug #18439: Support YJIT for VC++
https://bugs.ruby-lang.org/issues/18439#change-95673

* Author: usa (Usaku NAKAMURA)
* Status: Open
* Priority: Normal
* Assignee: maximecb (Maxime Chevalier-Boisvert)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/

[ruby-core:106935] [Ruby master Bug#18439] Support YJIT for VC++

[alanwu (Alan Wu)]

Issue #18439 has been updated by alanwu (Alan Wu).


Yes, supporting Window's x64 calling convention is non-trivial.
In addition to what John already mentioned we also need to uphold [unwindability](https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention?view=msvc-170#unwindability)
constraints so among other things, `longjmp` can work. I think `longjmp()` is used for Ruby exceptions [on MSVC](https://github.com/ruby/ruby/blob/11b8aaa26a22bb67b144484af6890844771b5f46/win32/Makefile.sub#L864) like on POSIX.
This means YJIT can't generate `PUSH` and `POP` outside of prolog and epilog anymore as they puts the stack pointer temporarily out of
alignment. YJIT will also need to supply unwinding info through [`RtlInstallFunctionTableCallback()`](https://docs.microsoft.com/en-us/windows/win32/api/winnt/nf-winnt-rtlinstallfunctiontablecallback).


----------------------------------------
Bug #18439: Support YJIT for VC++
https://bugs.ruby-lang.org/issues/18439#change-95756

* Author: usa (Usaku NAKAMURA)
* Status: Open
* Priority: Normal
* Assignee: maximecb (Maxime Chevalier-Boisvert)
* Backport: 2.6: UNKNOWN, 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/

[ruby-core:107031] [Ruby master Feature#18439] Support YJIT for VC++

[maximecb (Maxime Chevalier-Boisvert)]

Issue #18439 has been updated by maximecb (Maxime Chevalier-Boisvert).


Supporting Windows is in the plans, but as my colleagues have said it's fairly tricky as it could add a fair bit of complexity in several places. We're hoping to potentially migrate the YJIT codebase to Rust, which would give us more tools to manage the complexity, and could facilitate a project like this.

----------------------------------------
Feature #18439: Support YJIT for VC++
https://bugs.ruby-lang.org/issues/18439#change-95862

* Author: usa (Usaku NAKAMURA)
* Status: Open
* Priority: Normal
* Assignee: maximecb (Maxime Chevalier-Boisvert)
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/

[ruby-core:111763] [Ruby master Feature#18439] Support YJIT for VC++

[hsbt (Hiroshi SHIBATA) via ruby-core]

Issue #18439 has been updated by hsbt (Hiroshi SHIBATA).

Status changed from Open to Assigned
Assignee changed from maximecb (Maxime Chevalier-Boisvert) to yjit

YJIT has been rewrote with rust in Ruby 3.2. Should we close this?

----------------------------------------
Feature #18439: Support YJIT for VC++
https://bugs.ruby-lang.org/issues/18439#change-101167

* Author: usa (Usaku NAKAMURA)
* Status: Assigned
* Priority: Normal
* Assignee: yjit
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:111765] [Ruby master Feature#18439] YJIT: Support Microsoft x86 calling convention

[k0kubun (Takashi Kokubun) via ruby-core]

Issue #18439 has been updated by k0kubun (Takashi Kokubun).

Subject changed from Support YJIT for VC++ to YJIT: Support Microsoft x86 calling convention

What's used for compiling YJIT itself, which was changed from a C compiler to `rustc`, wasn't a huge concern for supporting Windows. Given that @usa has an idea for the `mmap` part, the main challenge for supporting Windows would be ABI support in JITed code. I changed the ticket description to clarify what's to be done.

----------------------------------------
Feature #18439: YJIT: Support Microsoft x86 calling convention
https://bugs.ruby-lang.org/issues/18439#change-101170

* Author: usa (Usaku NAKAMURA)
* Status: Assigned
* Priority: Normal
* Assignee: yjit
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:120333] [Ruby master Feature#18439] YJIT: Support Microsoft x86 calling convention

[dsisnero (Dominic Sisneros) via ruby-core]

Issue #18439 has been updated by dsisnero (Dominic Sisneros).


Any updates, thoughts.  As a forced windows user for work - would like some of the speed improvements that yjit brings.  Thanks

----------------------------------------
Feature #18439: YJIT: Support Microsoft x86 calling convention
https://bugs.ruby-lang.org/issues/18439#change-111108

* Author: usa (Usaku NAKAMURA)
* Status: Assigned
* Assignee: yjit
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:120334] [Ruby master Feature#18439] YJIT: Support Microsoft x86 calling convention

[maximecb (Maxime Chevalier-Boisvert) via ruby-core]

Issue #18439 has been updated by maximecb (Maxime Chevalier-Boisvert).


> Any updates, thoughts. As a forced windows user for work - would like some of the speed improvements that yjit brings. Thanks

Same as before. We would need to invest time into this, but it's not really a priority for us.

The other problem is that it's not easy for us to test things since we don't have windows machines. That would make development a pain.

----------------------------------------
Feature #18439: YJIT: Support Microsoft x86 calling convention
https://bugs.ruby-lang.org/issues/18439#change-111109

* Author: usa (Usaku NAKAMURA)
* Status: Assigned
* Assignee: yjit
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:120346] [Ruby master Feature#18439] YJIT: Support Microsoft x86 calling convention

[YO4 (Yoshinao Muramatsu) via ruby-core]

Issue #18439 has been updated by YO4 (Yoshinao Muramatsu).


The Windows version of ruby limits Integer to 30 bits even in x64 environments.
I am afraid that to support this in yjit would add complexity, which would harm the evolution of yjit.
Would 30bit Integer support actually have a negative impact?

Nevertheless, regardless of yjit support, I would like to see the Windows version of Integer's limitation removed.

----------------------------------------
Feature #18439: YJIT: Support Microsoft x86 calling convention
https://bugs.ruby-lang.org/issues/18439#change-111120

* Author: usa (Usaku NAKAMURA)
* Status: Assigned
* Assignee: yjit
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:120347] [Ruby master Feature#18439] YJIT: Support Microsoft x86 calling convention

[maximecb (Maxime Chevalier-Boisvert) via ruby-core]

Issue #18439 has been updated by maximecb (Maxime Chevalier-Boisvert).


It would definitely be simpler if integers were handled the same way on windows as on Linux and macOS.

----------------------------------------
Feature #18439: YJIT: Support Microsoft x86 calling convention
https://bugs.ruby-lang.org/issues/18439#change-111124

* Author: usa (Usaku NAKAMURA)
* Status: Assigned
* Assignee: yjit
----------------------------------------
I heard that supporting YJIT for VC++ needs mmap from k0kubun-san, so I implemented tiny mmap emulation on Windows and committed it to master.
And, I found we need more changes to actually enabled YJIT for VC++, at least:

- YJIT requires `OPT_DIRECT_THREADED_CODE` or `OPT_CALL_THREADED_CODE` in `rb_yjit_compile_iseq()`.  Really?
- Maybe ABI deffers between VC++ and YJIT's expectation.

Can I get support to fix above?



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/