ruby/benchmark
Jeremy Evans c20e819e8b Fix crash when passing large keyword splat to method accepting keywords and keyword splat
The following code previously caused a crash:

```ruby
h = {}
1000000.times{|i| h[i.to_s.to_sym] = i}
def f(kw: 1, **kws) end
f(**h)
```

Inside a thread or fiber, the size of the keyword splat could be much smaller
and still cause a crash.

I found this issue while optimizing method calling by reducing implicit
allocations.  Given the following code:

```ruby
def f(kw: , **kws) end
kw = {kw: 1}
f(**kw)
```

The `f(**kw)` call previously allocated two hashes callee side instead of a
single hash.  This is because `setup_parameters_complex` would extract the
keywords from the keyword splat hash to the C stack, to attempt to mirror
the case when literal keywords are passed without a keyword splat.  Then,
`make_rest_kw_hash` would build a new hash based on the extracted keywords
that weren't used for literal keywords.

Switch the implementation so that if a keyword splat is passed, literal keywords
are deleted from the keyword splat hash (or a copy of the hash if the hash is
not mutable).

In addition to avoiding the crash, this new approach is much more
efficient in all cases.  With the included benchmark:

```
                                1
            miniruby:   5247879.9 i/s
     miniruby-before:   2474050.2 i/s - 2.12x  slower

                        1_mutable
            miniruby:   1797036.5 i/s
     miniruby-before:   1239543.3 i/s - 1.45x  slower

                               10
            miniruby:   1094750.1 i/s
     miniruby-before:    365529.6 i/s - 2.99x  slower

                       10_mutable
            miniruby:    407781.7 i/s
     miniruby-before:    225364.0 i/s - 1.81x  slower

                              100
            miniruby:    100992.3 i/s
     miniruby-before:     32703.6 i/s - 3.09x  slower

                      100_mutable
            miniruby:     40092.3 i/s
     miniruby-before:     21266.9 i/s - 1.89x  slower

                             1000
            miniruby:     21694.2 i/s
     miniruby-before:      4949.8 i/s - 4.38x  slower

                     1000_mutable
            miniruby:      5819.5 i/s
     miniruby-before:      2995.0 i/s - 1.94x  slower
```
2024-02-11 22:48:38 -08:00
..
2023-03-06 22:36:57 -08:00
2021-02-10 19:42:00 +09:00
2023-02-28 10:05:30 -08:00
2023-11-20 14:33:20 +01:00
2023-04-25 08:06:16 -07:00
2023-04-25 08:06:16 -07:00
2021-06-18 10:02:44 -07:00

ruby/benchmark

This directory has benchmark definitions to be run with benchmark_driver.gem.

Normal usage

Execute gem install benchmark_driver and run a command like:

# Run a benchmark script with the ruby in the $PATH
benchmark-driver benchmark/app_fib.rb

# Run benchmark scripts with multiple Ruby executables or options
benchmark-driver benchmark/*.rb -e /path/to/ruby -e '/path/to/ruby --jit'

# Or compare Ruby versions managed by rbenv
benchmark-driver benchmark/*.rb --rbenv '2.5.1;2.6.0-preview2 --jit'

# You can collect many metrics in many ways
benchmark-driver benchmark/*.rb --runner memory --output markdown

# Some are defined with YAML for complex setup or accurate measurement
benchmark-driver benchmark/*.yml

See also:

Usage: benchmark-driver [options] RUBY|YAML...
    -r, --runner TYPE                Specify runner type: ips, time, memory, once, block (default: ips)
    -o, --output TYPE                Specify output type: compare, simple, markdown, record, all (default: compare)
    -e, --executables EXECS          Ruby executables (e1::path1 arg1; e2::path2 arg2;...)
        --rbenv VERSIONS             Ruby executables in rbenv (x.x.x arg1;y.y.y arg2;...)
        --repeat-count NUM           Try benchmark NUM times and use the fastest result or the worst memory usage
        --repeat-result TYPE         Yield "best", "average" or "worst" result with --repeat-count (default: best)
        --alternate                  Alternate executables instead of running the same executable in a row with --repeat-count
        --bundler                    Install and use gems specified in Gemfile
        --filter REGEXP              Filter out benchmarks with given regexp
        --run-duration SECONDS       Warmup estimates loop_count to run for this duration (default: 3)
        --timeout SECONDS            Timeout ruby command execution with timeout(1)
    -v, --verbose                    Verbose mode. Multiple -v options increase visilibity (max: 2)

make benchmark

Using make benchmark, make update-benchmark-driver automatically downloads the supported version of benchmark_driver, and it runs benchmarks with the downloaded benchmark_driver.

# Run all benchmarks with the ruby in the $PATH and the built ruby
make benchmark

# Or compare with specific ruby binary
make benchmark COMPARE_RUBY="/path/to/ruby --jit"

# Run vm benchmarks
make benchmark ITEM=vm

# Run some limited benchmarks in ITEM-matched files
make benchmark ITEM=vm OPTS=--filter=block

# You can specify the benchmark by an exact filename instead of using the default argument:
# ARGS = $$(find $(srcdir)/benchmark -maxdepth 1 -name '*$(ITEM)*.yml' -o -name '*$(ITEM)*.rb')
make benchmark ARGS=benchmark/erb_render.yml

# You can specify any option via $OPTS
make benchmark OPTS="--help"

# With `make benchmark`, some special runner plugins are available:
#   -r peak, -r size, -r total, -r utime, -r stime, -r cutime, -r cstime
make benchmark ITEM=vm_bigarray OPTS="-r peak"