Skip to content

Conversation

@Earlopain
Copy link
Collaborator

@Earlopain Earlopain commented Jan 26, 2026

Since on_sp is emitted, it doesn't do a whole lot anymore.

This leaves one incompatibility for code like "x#$%"

Ripper confuses this for bare interpolation with a global, but $% is not a valid global name. Still,
it emits two string tokens in such a case. It doesn't make sense for prism to work around this bug,
so the affected files are added as excludes.

@Earlopain Earlopain force-pushed the lex-ripper-plain branch 7 times, most recently from 4a23307 to 1ba9b44 Compare January 26, 2026 11:43
@Earlopain Earlopain marked this pull request as ready for review January 26, 2026 11:46
Copy link
Collaborator

@kddnewton kddnewton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove this API entirely instead of deprecating, I can't imagine anyone is using it.

I think I'd like us to support the splitting though, unless it's super difficult.

@kddnewton
Copy link
Collaborator

We need splitting for escaped newlines in heredocs anyway.

@Earlopain
Copy link
Collaborator Author

I think we can remove this API entirely instead of deprecating, I can't imagine anyone is using it.

I did a codesearch and didn't find anyone using it. Let's remove 👍

I think I'd like us to support the splitting though

Do you mean for "x#$%"? I created an upstream issue for this at https://bugs.ruby-lang.org/issues/21849. Since it's a single string, I would expect ripper to emit a single token for it like prism does. I don't think it makses sense to replicate the buggy behavior. And since it's only for our tests, excludes seem fine.

If you insist I try to implement it but the value of doing that seems very low to me. Doing it correctly is a bit complex since it depends on where the string appears. Quoted heredocs for example don't allow interpolation so ripper does the correct thing there and that doesn't just apply do dash heredocs.

@kddnewton
Copy link
Collaborator

Nah that's okay I don't insist :)

@kddnewton
Copy link
Collaborator

But let's do remove lex_ripper

Since `on_sp` is emitted, it doesn't do a whole lot anymore.

This leaves one incompatibility for code like `"x#$%"`

Ripper confuses this for bare interpolation with a global, but `$%` is not a valid global name. Still,
it emits two string tokens in such a case. It doesn't make sense for prism to work around this bug,
so the affected files are added as excludes.

Since the only usage of this method makes sense for testing in prism itself,
the method is removed instead of deprecated.
@Earlopain Earlopain changed the title Deprecate Prism.lex_ripper Remove Prism.lex_ripper Jan 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants