* Consistent and clear.
* Avoids the confusion that "column number" might be understood
as a column in an editor starting at 1 (they all start at 0).
https://github.com/ruby/prism/commit/91f1c4b9d5
In the C API, we want to use slices instead of locations in the
AST. In this case a "slice" is effectively the same thing as the
location, expect it is represented using a 32-bit offset and a
32-bit length. This will cut down on half of the space of all of
the locations in the AST.
Note that from the Ruby/Java/JavaScript side, this is effectively
an invisible change. This only impacts the C/Rust side.
[Bug #21161]
The `tolower` function provided by the libc is locale dependent
and can behave in ways you wouldn't expect for some value
of `LC_CTYPE`.
https://github.com/ruby/prism/commit/e3488256b4
Co-Authored-By: Nobuyoshi Nakada <nobu@ruby-lang.org>
According to the calloc(3) man page, when nmemb or size is 0, `calloc()` can either return NULL or a unique pointer that can be passed to `free()`.
While gcc and clang typically return a unique pointer, mruby's `mrb_calloc()` returns NULL in this case.
Since `pm_constant_pool_init()` is commonly called with capacity=0 during normal operation of Prism, explicitly handle this case by setting `list->ids` to NULL when capacity is 0.
This approach is portable across different calloc implementations and avoids potential issues with mruby's allocation behavior.
This maintains compatibility with `free()` and `realloc()`, as passing NULL pointers to these functions is explicitly allowed by their specifications.
https://github.com/ruby/prism/commit/1c32252df7
It looks like we can possibly do an out of bounds read if size is equal
to 0. This commit adds a conditional to ensure size is actually greater
than 0 before looking backwards in the buffer
https://github.com/ruby/prism/commit/2031b626e6
> ..., and on other POSIX systems we'll use `read`.
As `pm_string_mapped_init`'s doc comment says, it should fall back to
`read(2)`-based implementation on platforms without memory-mapped files
like WASI, but it didn't. This commit fixes it by calling `pm_string_file_init`
in the fallback case.
Also `defined(_POSIX_MAPPED_FILES)` check for `read(2)`-based path is
unnecessary, and it prevents the fallback from being executed, so this
change removes it.
https://github.com/ruby/prism/commit/b3d9064b71
This eliminates the subnode on RationalNode and replaces it with two
integer fields, which represent the ratio for the rational. It also
reduces those two integers if they both fit into 32 bits.
Importantly, this PR does not implement bignum reduction. That's something
I'd like to consider for the future, but it's simple enough for now to
leave them unreduced, which makes it more useful than it used to be.
https://github.com/ruby/prism/commit/86e06c7068
Fixes https://github.com/ruby/prism/pull/2617.
There was an issue with the lexer as follows.
The following are valid regexp options:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/io").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :EOF]
```
The following are invalid regexp options. Unnecessary the `IDENTIFIER` token is appearing:
```console
$ bundle exec ruby -Ilib -rprism -ve 'p Prism.lex("/x/az").value.map {|token| token[0].type }'
ruby 3.3.0 (2023-12-25 revision https://github.com/ruby/prism/commit/5124f9ac75) [x86_64-darwin22]
[:REGEXP_BEGIN, :STRING_CONTENT, :REGEXP_END, :IDENTIFIER, :EOF]
```
As a behavior of Ruby, when given `A` to `Z` and `a` to `z`, they act as invalid regexp options. e.g.,
```console
$ ruby -e '/regexp/az'
-e:1: unknown regexp options - az
/regexp/az
-e: compile error (SyntaxError)
```
Thus, it should probably not be construed as `IDENTIFIER` token.
Therefore, `pm_byte_table` has been adapted to accept those invalid regexp option values.
Whether it is a valid regexp option or not is checked by `pm_regular_expression_flags_create`.
For invalid regexp options, `PM_ERR_REGEXP_UNKNOWN_OPTIONS` is added to diagnostics.
https://github.com/ruby/prism/commit/d2a6096fcf
