[#88240] [Ruby trunk Feature#14759] [PATCH] set M_ARENA_MAX for glibc malloc — sam.saffron@...
Issue #14759 has been updated by sam.saffron (Sam Saffron).
3 messages
2018/08/01
[#88251] Re: [ruby-alerts:8236] failure alert on trunk@P895 (NG (r64134)) — Eric Wong <normalperson@...>
ko1c-failure@atdot.net wrote:
3 messages
2018/08/01
[#88305] [Ruby trunk Bug#14968] [PATCH] io.c: make all pipes nonblocking by default — normalperson@...
Issue #14968 has been reported by normalperson (Eric Wong).
4 messages
2018/08/06
[#88331] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — samuel@...
Issue #13618 has been updated by ioquatix (Samuel Williams).
3 messages
2018/08/08
[#88342] [Ruby trunk Feature#14955] [PATCH] gc.c: use MADV_FREE to release most of the heap page body — ko1@...
Issue #14955 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/08
[#88433] [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid — ko1@...
Issue #13618 has been updated by ko1 (Koichi Sasada).
4 messages
2018/08/10
[#88467] Re: [Ruby trunk Feature#13618] [PATCH] auto fiber schedule for rb_wait_for_single_fd and rb_waitpid
— Eric Wong <normalperson@...>
2018/08/14
ko1@atdot.net wrote:
[#88475] [Ruby trunk Misc#14937] [PATCH] thread_pthread: lazy-spawn timer-thread only on contention — ko1@...
Issue #14937 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/14
[#88491] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test — Eric Wong <normalperson@...>
k0kubun@ruby-lang.org wrote:
5 messages
2018/08/15
[#88495] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test
— Takashi Kokubun <takashikkbn@...>
2018/08/15
I see. Please remove the test if the test is unnecessary.
[#88496] Re: [ruby-cvs:71466] k0kubun:r64374 (trunk): test_function.rb: skip running test
— Eric Wong <normalperson@...>
2018/08/15
Takashi Kokubun <takashikkbn@gmail.com> wrote:
[#88523] [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed — eregontp@...
Issue #14999 has been updated by Eregon (Benoit Daloze).
4 messages
2018/08/17
[#88524] Re: [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed
— Eric Wong <normalperson@...>
2018/08/18
eregontp@gmail.com wrote:
[#88549] [Ruby trunk Bug#14999] ConditionVariable doesn't reacquire the Mutex if Thread#kill-ed — eregontp@...
Issue #14999 has been updated by Eregon (Benoit Daloze).
3 messages
2018/08/19
[#88676] [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations — ko1@...
Issue #15014 has been updated by ko1 (Koichi Sasada).
5 messages
2018/08/27
[#88677] Re: [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations
— Eric Wong <normalperson@...>
2018/08/27
ko1@atdot.net wrote:
[#88678] Re: [Ruby trunk Misc#15014] thread.c: use rb_hrtime_scalar for high-resolution time operations
— Koichi Sasada <ko1@...>
2018/08/27
On 2018/08/27 16:16, Eric Wong wrote:
[#88716] Re: [ruby-dev:43715] [Ruby 1.9 - Bug #595] Fiber ignores ensure clause — Eric Wong <normalperson@...>
Koichi Sasada wrote:
4 messages
2018/08/29
[#88723] [Ruby trunk Bug#15041] [PATCH] cont.c: set th->root_fiber to current fiber at fork — ko1@...
Issue #15041 has been updated by ko1 (Koichi Sasada).
3 messages
2018/08/29
[#88767] [Ruby trunk Bug#15050] GC after forking with fibers crashes — ko1@...
Issue #15050 has been updated by ko1 (Koichi Sasada).
7 messages
2018/08/31
[#88769] Re: [Ruby trunk Bug#15050] GC after forking with fibers crashes
— Eric Wong <normalperson@...>
2018/08/31
Koichi Sasada <ko1@atdot.net> wrote:
[#88774] Re: [ruby-alerts:8955] failure alert on trunk@P895 (NG (r64594)) — Eric Wong <normalperson@...>
ko1c-failure@atdot.net wrote:
3 messages
2018/08/31
[ruby-core:88728] [Ruby trunk Feature#14975] String#append without changing receiver's encoding
From:
duerst@...
Date:
2018-08-29 10:16:09 UTC
List:
ruby-core #88728
Issue #14975 has been updated by duerst (Martin D端rst).
As mentioned, the general idea of Ruby m17n is that strings that only contain ASCII bytes (8th bits are all 0) are treated as US-ASCII and can be combined with any ASCII-compatible encoding (taking on that encoding as a result).
The problem with this is that the encoding of the first truly non-ASCII string wins, and the second such string (assuming it's in a different encoding) produces an error. I'm not exactly sure this is consistent with the higher-level policy of failing early in case of encoding mixes, but I think it may be difficult to change.
In any way, when adding stuff to a (BINARY) buffer, the right thing conceptually is to change all the pieces to BINARY, not to rely on some of the pieces (be it the first or another) to be BINARY.
The problem with that is that it either changes the appended string's encoding (with `.force_encoding 'BINARY'`) or needs another copy (with `.b`). What I always wished we had is a method that forces the encoding of a string only locally, without leaking.
One way to realize this might be the following (Ruby as pseudocode):
```ruby
class String
alias :old_force_encoding :force_encoding
def force_encoding (encoding)
if block_given?
enc = encoding # remember current encoding
old_force_encoding encoding
yield
old_force_encoding enc # set encoding back
else
old_force_encoding encoding
end
end
end
```
This then would be used e.g. as follows:
```ruby
b = 'a'.force_encoding('BINARY')
u = "\u00ff".force_encoding('UTF-8') # aside: force_encoding here is a no-op,
# because \u automatically produces UTF-8
u.force_encoding('BINARY') do
b << u
end
```
That in my opinion would be the conceptually right way to do things.
What remains is that `<<` for buffers is in many cases not very efficient; it can be way more efficient to collect the `String`s to be appended in an array and then do a join. So we should not only think about this issue for `<<`/`concat`/`append`, but also for more wholesale methods.
----------------------------------------
Feature #14975: String#append without changing receiver's encoding
https://bugs.ruby-lang.org/issues/14975#change-73784
* Author: ioquatix (Samuel Williams)
* Status: Open
* Priority: Normal
* Assignee:
* Target version:
----------------------------------------
I'm not sure where this fits in, but in order to avoid garbage and superfluous function calls, is it possible that `String#<<`, `String#concat` or the (proposed) `String#append` can avoid changing the encoding of the receiver?
Right now it's very tricky to do this in a way that doesn't require extra allocations. Here is what I do:
```ruby
class Buffer < String
BINARY = Encoding::BINARY
def initialize
super
force_encoding(BINARY)
end
def << string
if string.encoding == BINARY
super(string)
else
super(string.b) # Requires extra allocation.
end
return self
end
alias concat <<
end
```
When the receiver is binary, but contains byte sequences, appending UTF_8 can fail:
```
"Foobar".b << "F淡淡bar"
=> "FoobarF淡淡bar"
> "F淡淡bar".b << "F淡淡bar"
Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8
```
So, it's not possible to append data, generally, and then call `force_encoding(Encoding::BINARY)`. One must ensure the string is binary before appending it.
It would be nice if there was a solution which didn't require additional allocations/copies/linear scans for what should basically be a `memcpy`.
See also: https://bugs.ruby-lang.org/issues/14033 and https://bugs.ruby-lang.org/issues/13626#note-3
There are two options to fix this:
1/ Don't change receiver encoding in any case.
2/ Apply 1, but only when receiver is using `Encoding::BINARY`
--
https://bugs.ruby-lang.org/
Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>