ruby-core

Issue #15764 has been updated by duerst (Martin Dst).


There may be a question as to what to do with encodings other than UTF-8. I see three possibilities:

1) Ignore (i.e. leave as is), because who still uses them?

2) Use a rule based on character properties such as letter and digit that are supported in these encodings

3) Convert to Unicode and check there

I would be okay with any one of these.

----------------------------------------
Bug #15764: Whitespace and control characters should not be permitted in tokens
https://bugs.ruby-lang.org/issues/15764#change-77647

* Author: BatmanAoD (Kyle Strand)
* Status: Open
* Priority: Normal
* Assignee: matz (Yukihiro Matsumoto)
* Target version: 
* ruby -v: 
* Backport: 2.4: UNKNOWN, 2.5: UNKNOWN, 2.6: UNKNOWN
----------------------------------------
As of Ruby 2.5.1p57, it appears that all valid Unicode code-points above 128 are permitted in tokens. This includes whitespace and control characters.

This was demonstrated here: https://gist.github.com/qrohlf/7045823

I have attached the raw download from the above gist.

The issue has been discussed on StackOverflow: https://stackoverflow.com/q/34455427/1858225

I would say this is arguably a bug, but I am marking this ticket as a "feature" since the current behavior could be considered by-design.

---Files--------------------------------
helloworld.rb (543 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>