You're supposed to return the first argument.
```rb
# Before
[[:stmts_new], [:rescue_mod, nil, nil], [:stmts_add, nil, nil], [:program, nil]]
# After
[[:stmts_new], [:rescue_mod, "1", "2"], [:stmts_add, nil, "1"], [:program, nil]]
```
The correct result would be:
`[[:rescue_mod, "1", "2"], [:stmts_new], [:stmts_add, nil, "1"], [:program, nil]]`
But the order depends on the prism AST so it seems very difficult to match.
https://github.com/ruby/prism/commit/94e0107729
* Handle line continuations.
* Handle space at the end of file in LexCompat.
https://github.com/ruby/prism/commit/32bd13eb7d
Co-authored-by: Earlopain <14981592+Earlopain@users.noreply.github.com>
This is everything that `irb` uses. It works in their test-suite, but there are 20 failures when using the shim that I haven't looked into at all.
`parse` is not used by `irb`. `scan` is, and it's basically `parse` but also including errors. `irb` doesn't seem to care about the errors, so I didn't implement that.
https://github.com/ruby/prism/commit/2c5826b39f
One per version seems excessive.
Do note that `rubocop-ast` used to require individual parser files. I wouldn't consider that to be part of the API since everything is autoloaded.
From a GitHub code search, I didn't find anyone else doing it like that.
https://github.com/ruby/prism/commit/458f622c34
Ripper exposes Ripper::Lexer:State in its output, which is a bit of a problem. To make this work, I basically copy-pasted the implementation.
I'm unsure if that is acceptable and added a test to make sure that these values never go out of sync.
I don't imagine them changing often, prism maps them 1:1 for its own usage.
This also fixed the shim by accident. `Ripper.lex` went to `Translation::Ripper.lex` when it should have been the original. Removing the need for the original resolves that issue.
https://github.com/ruby/prism/commit/2c0bea076d
This PR updates the fallback version for `Prism::Translation::ParserCurrent` from 3.4 to 4.0.
Currently, the fallback resolves to `Parser34`, as shown below:
```console
$ ruby -v -rprism -rprism/translation/parser_current -e 'p Prism::Translation::ParserCurrent'
ruby 3.0.7p220 (2024-04-23 revision https://github.com/ruby/prism/commit/724a071175) [x86_64-darwin23]
warning: `Prism::Translation::Current` is loading Prism::Translation::Parser34, but you are running 3.0.
Prism::Translation::Parser34
```
Following the comment "Keep this in sync with released Ruby.",
it seems like the right time to set this to Ruby 4.0, which is scheduled for release this week.
https://github.com/ruby/prism/commit/115f0a118c
This is more concise and ruby does a better job performance-wise.
This used to be `bsearch_index` already but 6d8358c083 changed it.
https://github.com/ruby/prism/pull/1733#discussion_r1373702087 said:
> Yeah the edge case was that the value matched an element exactly
But surely there would be a test to show this behaviour?
Gets called as part of pretty-printing nodes.
Further reduces the time for `SnapshotsTest` by ~16% for me.
https://github.com/ruby/prism/commit/f448e2b995
While the latter creates an intermediate array of all method names
including all ancestors, the former just traverse the inheritance
chain and can stop if found once.
https://github.com/ruby/prism/commit/6da384dd0e
In the case of attribute writes, there are use cases where you want
to know the location of the = sign. (Internally we actually need
this for translation to the writequark AST.)
https://github.com/ruby/prism/commit/bfc798a7ec
The docs currently say to use `Prism.parse(foo, version: RUBY_VERSION)` for this.
By specifying "current" instead, we can have prism raise a more specifc error.
Note: Does not use `ruby_version` from `ruby/version.h` because writing a test for that is not really possible.
`RUBY_VERSION` is nicely stubbable for both the c-ext and FFI backend.
https://github.com/ruby/prism/commit/9c5cd205cf
An unspecified uplevel is not the same as an uplevel of 1:
```
$ irb
irb(main):001> warn("foo")
foo
=> nil
irb(main):002> warn("foo", uplevel: 1)
/home/user/.rbenv/versions/2.7.8/lib/ruby/gems/2.7.0/gems/irb-1.14.0/lib/irb/workspace.rb:121: warning: foo
=> nil
```
https://github.com/ruby/prism/commit/dcedd14357
Generally I have been good about safely accessing the tokens but failed
to properly guard against no tokens in places
where it could theoretically happen through invalid syntax.
I added a test case for one occurance, other changes are theoretical only.
https://github.com/ruby/prism/commit/4a3866af19
The purpose of this commit is to fix Bug #21188. We need to detect when
stdin has run in to an EOF case. Unfortunately we can't _call_ the eof
function on IO because it will block.
Here is a short script to demonstrate the issue:
```ruby
x = STDIN.gets
puts x
puts x.eof?
```
If you run the script, then type some characters (but _NOT_ a newline),
then hit Ctrl-D twice, it will print the input string. Unfortunately,
calling `eof?` will try to read from STDIN again causing us to need a
3rd Ctrl-D to exit the program.
Before introducing the EOF callback to Prism, the input loop looked
kind of like this:
```ruby
loop do
str = STDIN.gets
process(str)
if str.nil?
p :DONE
end
end
```
Which required 3 Ctrl-D to exit. If we naively changed it to something
like this:
```ruby
loop do
str = STDIN.gets
process(str)
if STDIN.eof?
p :DONE
end
end
```
It would still require 3 Ctrl-D because `eof?` would block. In this
patch, we're wrapping the IO object, checking the buffer for a newline
and length, and then using that to simulate a non-blocking eof? method.
This commit wraps STDIN and emulates a non-blocking `eof` function.
[Bug #21188]
This makes it hard to do version checks against this value. The current version checks work because there are so few possible values at the moment.
As an example, PR 3337 introduces new syntax for ruby 3.5 and uses `PM_OPTIONS_VERSION_LATEST` as its version guard. Because what is considered the latest changes every year, it must later be changed to `parser->version == parser->version == PM_OPTIONS_VERSION_CRUBY_3_5 || parser->version == PM_OPTIONS_VERSION_LATEST`, with one extra version each year.
With this change, the PR can instead write `parser->version >= PM_OPTIONS_VERSION_CRUBY_3_5` which is self-explanatory
and works for future versions.
https://github.com/ruby/prism/commit/8318a113ca
