[ruby-core:115764] [Ruby master Feature#20069] Buffer class in stdlib

[ruby-core:115764] [Ruby master Feature#20069] Buffer class in stdlib

[pynix (Pynix wang) via ruby-core]

Issue #20069 has been reported by pynix (Pynix wang).

----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069

* Author: pynix (Pynix wang)
* Status: Open
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115765] [Ruby master Feature#20069] Buffer class in stdlib

[nobu (Nobuyoshi Nakada) via ruby-core]

Issue #20069 has been updated by nobu (Nobuyoshi Nakada).


What's the use case?
Does it differ from `IO::Buffer`?

----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069#change-105702

* Author: pynix (Pynix wang)
* Status: Open
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115766] [Ruby master Feature#20069] Buffer class in stdlib

[pynix (Pynix wang) via ruby-core]

Issue #20069 has been updated by pynix (Pynix wang).


main use case is deal binary data，a replacement of String.

eg grpc bytes type, crypto key and more.

maybe not same as io buffer, so Bytes or Binary is good for class name.

----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069#change-105703

* Author: pynix (Pynix wang)
* Status: Open
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115767] [Ruby master Feature#20069] Buffer class in stdlib

[pynix (Pynix wang) via ruby-core]

Issue #20069 has been updated by pynix (Pynix wang).


```
irb(main):048> SecureRandom.bytes(10)
=> "\xB6e\x1C\xF3T\x9C\xA1\xDF\xBD\xEA"
irb(main):049> SecureRandom.bytes(10)
=> "\"\xC4;0\xB3\xA6!\x80jn"
irb(main):050> SecureRandom.bytes(10)
=> "\x9B\x9CP\t~\"\xB9\x8EAn"
irb(main):051> SecureRandom.bytes(10)
=> "\xAA\xDEf\x92\x8E\xEE5]\xD0\xB2"
irb(main):052> SecureRandom.bytes(10)
=> "\xFD\xE9\xF5@n\x1D\x9D\xB4\xB7\x8A"
irb(main):053> SecureRandom.bytes(10)
=> "\xCB\x90\xB0\xCB\xDF\xD2\xED\xAA\a\\"
irb(main):054> SecureRandom.bytes(10)
=> "\x16p\x15\x12\xC0\xD6\x02*D\xDB"
irb(main):055> SecureRandom.bytes(10)
=> "F\x97\xC2d\x84\a\x87\xA3P\b"
irb(main):056> SecureRandom.bytes(10)
=> "\x98\xAB\xA1\x96\x15\x91\x92\xF8e5"
irb(main):057> SecureRandom.bytes(10)
=> "\xE3\xD0\xB5P\x95ys\x0E\xCF'"
irb(main):058> SecureRandom.bytes(10)
=> "\xC5\xFB\x04\x97\xFC\xC0\xF5\xEF{\xA2"


use String as Bytes get a non unified representation, some bytes is translated into string, some not.
```

----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069#change-105704

* Author: pynix (Pynix wang)
* Status: Open
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115769] [Ruby master Feature#20069] Buffer class in stdlib

[duerst via ruby-core]

Issue #20069 has been updated by duerst (Martin Dürst).





pynix (Pynix wang) wrote:

> ruby use `String` to deal with bytes



Ruby uses big classes. That avoids duplicating a lot of functionality in many classes, and also avoids a lot of conversion operations.



> this cause error on irb  "invalid byte sequence in utf-8"



Can you show an example?



pynix (Pynix wang) wrote in #note-3:

> irb(main):058> SecureRandom.bytes(10)

> => "\xC5\xFB\x04\x97\xFC\xC0\xF5\xEF{\xA2"



> use String as Bytes get a non unified representation, some bytes is translated into string, some not.



You can get a uniform representation with `SecureRandom.hex(10)`.





----------------------------------------

Feature #20069: Buffer class in stdlib

https://bugs.ruby-lang.org/issues/20069#change-105706



* Author: pynix (Pynix wang)

* Status: Open

* Priority: Normal

----------------------------------------

ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"



can we get a builtin class like Buffer or Bytes that represent as hex string







-- 

https://bugs.ruby-lang.org/

 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115770] [Ruby master Feature#20069] Buffer class in stdlib

[austin (Austin Ziegler) via ruby-core]

Issue #20069 has been updated by austin (Austin Ziegler).


pynix (Pynix wang) wrote:
> ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"
> 
> can we get a builtin class like Buffer or Bytes that represent as hex string

```ruby
class Bytes < String
  def inspect
    bytes.pack("c*").unpack1("H*")
  end
end

s = Bytes.new(SecureRandom.bytes(10))

```

What might be more interesting than suggesting an unnecessary class, but suggesting a different `#inspect` if the encoding is `ASCII-8BIT` or `BINARY` (because `SecureRandom.bytes(10).encoding # => #<Encoding:ASCII-8BIT>`, which will eventually be called `Encoding:BINARY`).I’m not sure what such inspect should be, because the inspect that I wrote above for `Bytes` is both inefficient and incorrect (because the representation is not what is shown on `#inspect`, which differs from other strings).

Maybe:

```ruby
class Bytes < String
  def inspect
    "#<#{encoding}:#{bytes.pack("c*").unpack1("H*")}>"
  end
end

Bytes.new(SecureRandom.bytes(10))
=> #<ASCII-8BIT:1c2dc1463d30c6ed0b9a>
```

----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069#change-105707

* Author: pynix (Pynix wang)
* Status: Open
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115778] [Ruby master Feature#20069] Buffer class in stdlib

[shan (Shannon Skipper) via ruby-core]

Issue #20069 has been updated by shan (Shannon Skipper).


pynix (Pynix wang) wrote:
> ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

I'm curious, did you actually run into an "invalid byte sequence" error? If so, could you show the code that produced the error?

I know IO::Buffer has already been mentioned, but just wanted to point out it inspects with pretty hex.

``` ruby
>> IO::Buffer.for SecureRandom.random_bytes
=> 
#<IO::Buffer 0x00007f7dfa885998+16 EXTERNAL READONLY SLICE>
#0x00000000  17 bc 59 2d 8b 66 4b 6a 56 96 97 98 5e 07 45 d6 ..Y-.fKjV...^.E.
```

----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069#change-105714

* Author: pynix (Pynix wang)
* Status: Open
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115806] [Ruby master Feature#20069] Buffer class in stdlib

[ioquatix (Samuel Williams) via ruby-core]

Issue #20069 has been updated by ioquatix (Samuel Williams).


> ruby use String to deal with bytes, this cause error on irb "invalid byte sequence in utf-8"

This is desirable behaviour. The String with UTF-8 encoding cannot contain invalid byte sequences. If you want to store binary data, use `Encoding::BINARY` encoding.

> can we get a builtin class like Buffer or Bytes that represent as hex string

A Binary string already presents the inspect output as a hex encoded String.

As others have pointed out, if you want actual memory mapped binary buffers, use `IO::Buffer`.

----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069#change-105755

* Author: pynix (Pynix wang)
* Status: Open
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:115811] [Ruby master Feature#20069] Buffer class in stdlib

[matz (Yukihiro Matsumoto) via ruby-core]

Issue #20069 has been updated by matz (Yukihiro Matsumoto).

Status changed from Open to Closed

Use either string with `BINARY` encoding or `IO::Buffer`. If these two lack what you want, open a new issue please.

Matz.


----------------------------------------
Feature #20069: Buffer class in stdlib
https://bugs.ruby-lang.org/issues/20069#change-105760

* Author: pynix (Pynix wang)
* Status: Closed
* Priority: Normal
----------------------------------------
ruby use `String` to deal with bytes, this cause error on irb  "invalid byte sequence in utf-8"

can we get a builtin class like Buffer or Bytes that represent as hex string



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/