Fixes https://github.com/Shopify/ruby/issues/902
This pull request adds code generation for dividing fixnums.
Testing confirms the normal case, flooring, and side-exiting on division by zero.
* All Invariant::SingleRactorMode PatchPoint are replaced by
assume_single_ractor_mode() to fix https://github.com/Shopify/ruby/issues/875
for SingleRactorMode patchpoints.
Storing the tagged block handler in profiles is not GC-safe (nice catch,
Kokubun). Store the untagged block handler instead.
Fix bug in https://github.com/ruby/ruby/pull/15051
I made a special kind of `ProfiledType` that looks at specific objects, not just their classes/shapes (https://github.com/ruby/ruby/pull/15051). Then I profiled some of our benchmarks.
For lobsters:
```
Top-6 invokeblock handler (100.0% of total 1,064,155):
megamorphic: 494,931 (46.5%)
monomorphic_iseq: 337,171 (31.7%)
polymorphic: 113,381 (10.7%)
monomorphic_ifunc: 52,260 ( 4.9%)
monomorphic_other: 38,970 ( 3.7%)
no_profiles: 27,442 ( 2.6%)
```
For railsbench:
```
Top-6 invokeblock handler (100.0% of total 2,529,104):
monomorphic_iseq: 834,452 (33.0%)
megamorphic: 818,347 (32.4%)
polymorphic: 632,273 (25.0%)
monomorphic_ifunc: 224,243 ( 8.9%)
monomorphic_other: 19,595 ( 0.8%)
no_profiles: 194 ( 0.0%)
```
For shipit:
```
Top-6 invokeblock handler (100.0% of total 2,104,148):
megamorphic: 1,269,889 (60.4%)
polymorphic: 411,475 (19.6%)
no_profiles: 173,367 ( 8.2%)
monomorphic_other: 118,619 ( 5.6%)
monomorphic_iseq: 84,891 ( 4.0%)
monomorphic_ifunc: 45,907 ( 2.2%)
```
Seems like a monomorphic case for a specific ISEQ actually isn't a bad way of going about this, at least to start...
Fixes https://github.com/Shopify/ruby/issues/814
This change specializes the case of calling `Array#pop` on a non frozen array with no arguments. `Array#pop` exists in the non-inlined C function list in the ZJIT SFR performance burndown list.
If in the future it is helpful, this patch could be extended to support the case where an argument is provided, but this initial work seeks to elide the ruby frame normally pushed in the case of `Array#pop` without an argument.
```
/src/jit.c:19:5: error: ISO C restricts enumerator values to range of 'int' (4294967295 is too large) [-Werror,-Wpedantic]
19 | RB_INVALID_SHAPE_ID = INVALID_SHAPE_ID,
| ^ ~~~~~~~~~~~~~~~~
```
no_rest() trips an assert inside the GC when we allocate with the GC
disabled this way:
(gc_continue) ../src/gc/default/default.c:2029
(newobj_cache_miss+0x128) [0x105040048] ../src/gc/default/default.c:2370
(rb_gc_impl_new_obj+0x7c) [0x105036374] ../src/gc/default/default.c:2482
(newobj_of) ../src/gc.c:995
(rb_method_entry_alloc+0x40) [0x1051e6c64] ../src/vm_method.c:1102
(rb_method_entry_complement_defined_class) ../src/vm_method.c:1180
(prepare_callable_method_entry+0x14c) [0x1051e87b8] ../src/vm_method.c:1728
(callable_method_entry_or_negative+0x1e8) [0x1051e809c] ../src/vm_method.c:1874
It's tries to continue the GC because it was out of space. Looks like
it's not safe to allocate new objects after using
rb_gc_disable_no_rest(); existing usages use it for malloc calls.
Strictly more info than just the builtin_type from `assert_ne!`.
Old:
assertion `left != right` failed: ZJIT should only see live objects
left: 0
right: 0
New:
ZJIT saw a dead object. T_type=0, out-of-heap:0x0000000110d4bb40
Also, the new `VALUE::obj_info` is more flexible for print debugging than the
dump_info() it replaces. It now allows you to use it as part of a `format!`
string instead of always printing to stderr for you.
* ZJIT: Add Insn:: ArrayArefFixnum to accelerate Array#[]
* ZJIT: Use result from GuardType in ArrayArefFixnum
* ZJIT: Unbox index for aref_fixnum
* ZJIT: Change condition and add ArrayArefFixnum test
* ZJIT: Fix ArrayArefFixnum display for InsnPrinter
* ZJIT: Change insta test
Relying on having the same compiler version and behavior across
platforms is brittle, as Kokubun points out. Instead, name the enum so
we don't have to rely on gensym stability.
Fix https://github.com/Shopify/ruby/issues/787
* ZJIT: Support variadic C calls
This reduces the `dynamic_send_count` in `liquid-render` by ~21%
* ZJIT: Reuse gen_push_frame
* ZJIT: Avoid optimizing variadic C call when tracing is enabled
Specialize monomorphic `GetIvar` into:
* `GuardType(HeapObject)`
* `GuardShape`
* `LoadIvarEmbedded` or `LoadIvarExtended`
This requires profiling self for `getinstancevariable` (it's not on the operand
stack).
This also optimizes `GetIvar`s that happen as a result of inlining
`attr_reader` and `attr_accessor`.
Also move some (newly) shared JIT helpers into jit.c.
`toregexp` is fairly similar to `concatstrings`, so this commit extracts
a helper for pushing and popping operands on the native stack.
There's probably opportunity to move some of this into lir (e.g. Alan
suggested a push_many that could use STP on ARM to push 2 at a time),
but I might save that for another day.
ZJIT uses the interpreter to take type profiles of what objects pass through
the code. It stores a compressed record of the history per opcode for the
opcodes we select.
Before this change, we re-used the HIR Type data-structure, a shallow type
lattice, to store historical type information. This was quick for bringup but
is quite lossy as profiles go: we get one bit per built-in type seen, and if we
see a non-built-in type in addition, we end up with BasicObject. Not very
helpful. Additionally, it does not give us any notion of cardinality: how many
of each type did we see?
This change brings with it a much more interesting slice of type history: a
histogram. A Distribution holds a record of the top-N (where N is fixed at Ruby
compile-time) `(Class, ShapeId)` pairs and their counts. It also holds an
*other* count in case we see more than N pairs.
Using this distribution, we can make more informed decisions about when we
should use type information. We can determine if we are strictly monomorphic,
very nearly monomorphic, or something else. Maybe the call-site is polymorphic,
so we should have a polymorphic inline cache. Exciting stuff.
I also plumb this new distribution into the HIR part of the compilation
pipeline.
Fixes `TestZJIT::test_require_rubygems`. It was crashing locally due to
false collection of a live object. See
<https://alanwu.space/post/write-barrier/>.
Co-authored-by: Max Bernstein <max@bernsteinbear.com>
Co-authored-by: Takashi Kokubun <takashi.kokubun@shopify.com>
Co-authored-by: Stan Lo <stan.lo@shopify.com>
