9218 Commits

Author SHA1 Message Date
yui-knk
cf74ff714a Change return value of gets function to be rb_parser_string_t * instead of VALUE
This change reduces parser's dependency on ruby object.
2024-05-04 11:59:10 +09:00
Nobuyoshi Nakada
7662e6fcd5
win32/registry: Shorten with safe navigation operator 2024-05-03 14:49:29 +09:00
Nobuyoshi Nakada
491c38bfcd
win32/registry: Make frozen_string_literal true 2024-05-03 14:49:29 +09:00
Nobuyoshi Nakada
0b091e6675
win32/registry: Refine pack/unpack
* Use 'J' for HANDLE.
* Use 'Q' for QWORD.
* Define template constants.
* Supply zero bytes in `unpackqw` as well as `unpackdw`.
* Use `String#unpack1`.
2024-05-03 14:49:29 +09:00
Jeremy Evans
55720f372d [ruby/openssl] pkcs7: raise PKCS7Error for PKCS7 without content in PKCS7.read_smime
[pkuzco: expanded the fix for other content types]
[ky: adjusted formatting and the exception type]

https://github.com/ruby/openssl/commit/07eceb7f63

Co-authored-by: pkuzco <b.naamneh@gmail.com>
Co-authored-by: Kazuki Yamaguchi <k@rhe.jp>
2024-05-02 16:26:11 +09:00
Jeremy Evans
c9aa63a9e0 [ruby/openssl] pkcs7: raise ArgumentError for PKCS7 with no content in PKCS7.new
Fixes [Bug #19974]

[pkuzco: expanded the fix for other content types]
[ky: adjusted formatting and the exception type]

https://github.com/ruby/openssl/commit/27e11f2d1d

Co-authored-by: pkuzco <b.naamneh@gmail.com>
Co-authored-by: Kazuki Yamaguchi <k@rhe.jp>
2024-05-02 16:26:11 +09:00
Kazuki Yamaguchi
eb6f0000a4 [ruby/openssl] cipher: fix buffer overflow in Cipher#update
OpenSSL::Cipher#update currently allocates the output buffer with size
(input data length)+(the block size of the cipher). This is insufficient
for the id-aes{128,192,256}-wrap-pad (AES keywrap with padding) ciphers.
They have a block size of 8 bytes, but the output may be up to 15 bytes
larger than the input.

Use (input data length)+EVP_MAX_BLOCK_LENGTH (== 32) as the output
buffer size, instead. OpenSSL doesn't provide a generic way to tell the
maximum required buffer size for ciphers, but this is large enough for
all algorithms implemented in current versions of OpenSSL.

Fixes: https://bugs.ruby-lang.org/issues/20236

https://github.com/ruby/openssl/commit/3035559f54
2024-05-02 16:26:11 +09:00
Jaymz Julian
d39993a4ce [ruby/openssl] Fix performance regression in do_write(s)
This causes significant performance issues when using large (>10meg) writes

Fix by adjusting the buffer write function to clear the buffer once, rather than
piece by piece, avoiding a case where a large write (in our case, around
70mbytes) will consume 100% of CPU. This takes a webrick GET request via SSL
from around 200kbyts/sec and consuming 100% of a core, to line speed on gigabit
ethernet and 6% cpu utlization.

https://github.com/ruby/openssl/commit/d4389b425d
2024-05-02 16:24:36 +09:00
Kazuki Yamaguchi
f5af620c61 [ruby/openssl] asn1: check error return from i2d_ASN1_TYPE()
i2d_ASN1_TYPE() is not expected to fail, but the return value should be
checked.

https://github.com/ruby/openssl/commit/21ed3c310e
2024-05-02 07:17:35 +00:00
Bart de Water
9d75d9f07c [ruby/openssl] Update Cipher#name to match Digest#name explanation
https://github.com/ruby/openssl/commit/79e6dead6e
2024-04-30 15:05:50 +00:00
Bart de Water
cc6657e563 [ruby/openssl] Add OpenSSL::Digest.digests to get a list of available digests
https://github.com/ruby/openssl/commit/08dd3c73b7
2024-04-30 15:05:50 +00:00
Samuel Williams
368ce0758e [ruby/openssl] Further clarification of documentation.
https://github.com/ruby/openssl/commit/0697f2f8b4
2024-04-30 14:51:59 +00:00
Samuel Williams
f1afae2459 [ruby/openssl] More documentation.
https://github.com/ruby/openssl/commit/c8377eaf8d
2024-04-30 14:51:59 +00:00
Samuel Williams
1699772ac4 [ruby/openssl] Introduce basic support for close_read and close_write.
https://github.com/ruby/openssl/commit/c99d24cee9
2024-04-30 14:51:58 +00:00
Nobuyoshi Nakada
a6308ca958 ripper: Move DSL line pattern 2024-04-29 08:38:23 +09:00
ydah
f9cf923af2 Use user defined parameterizing rules 2024-04-29 08:38:23 +09:00
卜部昌平
c844968b72 ruby tool/update-deps --fix 2024-04-27 21:55:28 +09:00
Sam Aaron
67b79d484f
Enable Ruby to run on Windows with frozen string literals 2024-04-26 15:10:42 +00:00
Nobuyoshi Nakada
91ff2fd9b5 Fix dllimport warnings
From Visual C:
```
../../../../../src/ext/-test-/load/resolve_symbol_target/resolve_symbol_target.c(5): warning C4273: 'rst_any_method': inconsistent dll linkage
D:\a\ruby\ruby\src\ext\-test-\load\resolve_symbol_target\resolve_symbol_target.h(4): note: see previous definition of 'rst_any_method'
../../../../../src/ext/-test-/load/stringify_target/stringify_target.c(5): warning C4273: 'stt_any_method': inconsistent dll linkage
D:\a\ruby\ruby\src\ext\-test-\load\stringify_target\stringify_target.h(4): note: see previous definition of 'stt_any_method'
```

From MinGW gcc:
```
../../../../../src/ext/-test-/load/resolve_symbol_target/resolve_symbol_target.c:5:1: warning: 'rst_any_method' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
compiling ../../../../../src/ext/-test-/marshal/compat/usrcompat.c
    5 | rst_any_method(VALUE klass)
      | ^~~~~~~~~~~~~~
../../../../../src/ext/-test-/load/stringify_target/stringify_target.c:5:1: warning: 'stt_any_method' redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
    5 | stt_any_method(VALUE klass)
      | ^~~~~~~~~~~~~~
```
2024-04-24 18:02:09 +09:00
yui-knk
33929ef995 Move encoding object conversion outside of parser
Reduce the parser's dependence on `VALUE` and `rb_enc_from_encoding`.
2024-04-23 13:11:46 +09:00
Nobuyoshi Nakada
afa0d58580
Adjust indent [ci skip] 2024-04-23 09:21:38 +09:00
yui-knk
2992e1074a Refactor parser compile functions
Refactor parser compile functions to reduce the dependence
on ruby functions.
This commit includes these changes

1. Refactor `gets`, `input` and `gets_` of `parser_params`

Parser needs two different data structure to get next line, function (`gets`) and input data (`input`).
However `gets_` is used for both function (`call`) and input data (`ptr`).
`call` is used for managing general callback function when `rb_ruby_parser_compile_generic` is used.
`ptr` is used for managing the current pointer on String when `parser_compile_string` is used.
This commit changes parser to used only `gets` and `input` then removes `gets_`.

2. Move parser_compile functions and `gets` functions from parse.y to ruby_parser.c

This change reduces the dependence on ruby functions from parser.

3. Change ruby_parser and ripper to take care of `VALUE input` GC mark

Move the responsibility of calling `rb_gc_mark` for `VALUE input` from parser to ruby_parser and ripper.
`input` is arbitrary data pointer from the viewpoint of parser.

4. Introduce rb_parser_compile_array function

Caller of `rb_parser_compile_generic` needs to take care about GC because ruby_parser doesn’t know
about the detail of `lex_gets` and `input`.
Introduce `rb_parser_compile_array` to reduce the complexity of ast.c.
2024-04-23 07:20:22 +09:00
Nobuyoshi Nakada
801e4a4feb
Remove UPDATE_LIBRARIES
It has not been used since e48375c112022fa321786ccd95dd4e718efd78a3.
2024-04-19 14:12:42 +09:00
Jean byroot Boussier
75154dec73 [ruby/stringio] strio_read: preserve buffer encoding on partial
reads
(https://github.com/ruby/stringio/pull/95)

[[Bug #20418]](https://bugs.ruby-lang.org/issues/20418)

Ruby IO#read preserves the encoding on partial read, but change it when
reading the whole IO

from commit https://github.com/ruby/ruby/commit/0ca7036682da:
> * io.c (read_all): should associate default external encoding.
> * io.c (io_read): should NOT associate default external encoding.

https://github.com/ruby/stringio/commit/073172da31

Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2024-04-17 01:29:18 +00:00
Peter Zhu
f8f542bd9b [ruby/pathname] Remove check for File.birthtime
File.birthtime has existed since Ruby 2.2, and pathname requires Ruby
>= 2.7.0, so the method will always be there.

https://github.com/ruby/pathname/commit/aca9613bbf
2024-04-16 05:48:57 +00:00
Nobuyoshi Nakada
8d5d6ec6e7
[pty] Fix missing or 2024-04-14 20:35:34 +09:00
Nobuyoshi Nakada
3368913be3 [pty] Fix ptsname_r fallback
If `posix_openpt` is available, also `ptsname` should be available.
2024-04-14 20:27:05 +09:00
yui-knk
38b8bdb8ea Remove undefined function's prototype declaration
89cfc152071 removed the definition of these functions.
2024-04-14 10:51:16 +09:00
Nobuyoshi Nakada
e7f8db9079
[pty] Support ptsname_r of glibc
Although glibc `ptsname_r` man page mentions Tru64 and HP-UX, this
function appears to be declared obsolete on both.
2024-04-11 17:57:52 +09:00
Nobuyoshi Nakada
0bc71828b5 [pty] Split chfunc into functions in steps
- start a new session
- obtain the new controlling terminal
- drop privileges
- finally, `exec`
2024-04-09 22:50:09 +09:00
Nobuyoshi Nakada
c4dadfbd47
Fix a typo, missing P in SETPGRP_VOID 2024-04-07 20:54:00 +09:00
Nobuyoshi Nakada
3ac6a03b2e
Revert "hijack SIGCHLD handler for internal use"
This reverts commit 054a412d540e7ed2de63d68da753f585ea6616c3.
SIGCHLD `waidpid`, `waitpid_lock` and related code, have been removed
at ruby/ruby#7527.
2024-04-04 21:48:14 +09:00
Hiroshi SHIBATA
ab65148c78 [flori/json] Bump up 2.7.2
https://github.com/flori/json/commit/036944acc6
2024-04-04 12:46:02 +09:00
Hiroshi SHIBATA
514b415d90 [flori/json] Warn to install ostruct if json couldn't load it
https://github.com/flori/json/commit/fff285968d
2024-04-04 12:46:02 +09:00
tompng
066bd28cff [flori/json] Autoload GenericObject to avoid require ostruct warning in Ruby 3.4
https://github.com/flori/json/commit/b507f9e404
2024-04-04 12:46:01 +09:00
yui-knk
e816ab0b0c Remove rb_imemo_tmpbuf_t from parser
No parser semantic value types are `VALUE` then no need to
use imemo for managing semantic value stack anymore.
2024-04-02 19:37:27 +09:00
yui-knk
799e854897 [Feature #20331] Simplify parser warnings for hash keys duplication and when clause duplication
This commit simplifies warnings for hash keys duplication and when clause duplication,
based on the discussion of https://bugs.ruby-lang.org/issues/20331.
Warnings are reported only when strings are same to ohters.
2024-04-02 08:26:58 +09:00
Hiroshi SHIBATA
7630a89a4b Use www.rfc-editor.org for RFC text.
We use the following site for that now:

* https://tools.ietf.org/ or http
* https://datatracker.ietf.org or http

Today, IETF said the official site of RFC is www.rfc-editor.org.

FYI: https://authors.ietf.org/en/references-in-rfcxml

I replaced them to www.rfc-editor.org.
2024-03-28 11:44:45 +09:00
Nobuyoshi Nakada
51e6becd39 [ruby/stringio] Extract readonly_string_p
https://github.com/ruby/stringio/commit/0da5b725c8
2024-03-27 17:41:38 +00:00
Nobuyoshi Nakada
06563d78a1 [ruby/stringio] Adjust styles [ci skip]
https://github.com/ruby/stringio/commit/4e8e82fc30
2024-03-28 02:27:20 +09:00
Étienne Barrié
0f5ab4ad52 [ruby/stringio] Eagerly defrost chilled strings
[Feature #20390]

https://github.com/ruby/stringio/commit/17ee957f34

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
2024-03-28 02:27:19 +09:00
Peter Zhu
6e34386794
[flori/json] Fix memory leak when exception is raised during JSON generation
If an exception is raised the FBuffer is leaked.

For example, the following script leaks memory:

    o = Object.new
    def o.to_json(a) = raise

    10.times do
      100_000.times do
        begin
          JSON(o)
        rescue
        end
      end

      puts `ps -o rss= -p #{$$}`
    end

Before:

    31824
    35696
    40240
    44304
    47424
    50944
    54000
    58384
    62416
    65296

After:

    24416
    24640
    24640
    24736
    24736
    24736
    24736
    24736
    24736
    24736

https://github.com/flori/json/commit/44df509dc2
2024-03-27 08:24:28 +09:00
Étienne Barrié
2b08406cd0 Expose rb_str_chilled_p
Some extensions (like stringio) may need to differentiate between
chilled strings and frozen strings.

They can now use rb_str_chilled_p but must check for its presence since
the function will be removed when chilled strings are removed.

[Bug #20389]

[Feature #20205]

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
2024-03-26 12:54:54 +01:00
Nobuyoshi Nakada
8265a7531f
Use dedicated methods to abort
When `RUBY_DEBUG` is set, accessing a class in an invalid object will
cause a breakpoint trap instead of a segfault on some implementations.
2024-03-24 01:40:17 +09:00
Nobuyoshi Nakada
678cb80033
Move -test-/fatal/rb_fatal to -test-/fatal 2024-03-24 01:09:29 +09:00
Nobuyoshi Nakada
43fe89d921 [ruby/win32ole] Refine pathspecs for spec.files
https://github.com/ruby/win32ole/commit/8d443417a9
2024-03-23 13:06:40 +00:00
Nobuyoshi Nakada
14fba5a784 [ruby/win32ole] Exclude unused files from packages
https://github.com/ruby/win32ole/commit/f4aff99dda
2024-03-23 10:34:57 +00:00
Nobuyoshi Nakada
af88ca4c07
Match --with/--without options against gem names
The simple names in `default_exclude_exts` do not match extension
paths under gems.

Extract each gem name from the gemspec file at the top level of each
gem directory.  For example, if `ext` is `syslog-0.1.2/ext/syslog/`,
find out `syslog-0.1.2/syslog.gemspec` and take the base name `syslog`
without the suffix `.gemspec`.
2024-03-22 14:09:12 +09:00
Nobuyoshi Nakada
235aa7ad63
Pass blocks to blocks as block parameters 2024-03-22 14:06:55 +09:00
S-H-GAMELINKS
060a71d4e7 Fix Ripper memory allocation size when enabled Universal Parser
The size of `struct parser_params` is 8 bytes difference in `ripper_s_allocate` and `rb_ruby_parser_allocate` when the universal parser is
enabled.
This causes a situation where `*r->p` is not fully initialized in `ripper_s_allocate` as shown below.

```console
(gdb) p *r->p
$2 = {heap = 0x0, lval = 0x0, yylloc = 0x0, lex = {strterm = 0x0, gets = 0x0, input = 0, string_buffer = {head = 0x0, last = 0x0}, lastlin
e = 0x0,
    nextline = 0x0, pbeg = 0x0, pcur = 0x0, pend = 0x0, ptok = 0x0, gets_ = {ptr = 0, call = 0x0}, state = EXPR_NONE, paren_nest = 0, lpar
_seen = 0,
    debug = 0, has_shebang = 0, token_seen = 0, token_info_enabled = 0, error_p = 0, cr_seen = 0, value = 0, result = 0, parsing_thread = 0, s_value = 0,
    s_lvalue = 0, s_value_stack = 2097}
````

This seems to cause `double free or corruption (!prev)` and SEGV.
So, fixing this by introduce `rb_ripper_parser_params_allocate` and `rb_ruby_parser_config` functions for Ripper, and `struct parser_params` same size is returned.
2024-03-21 18:10:02 +09:00