Re: [Caml-list] Immutable strings

[Alain Frisch <alain@frisch.fr>]

On 07/04/2014 09:18 PM, Gerd Stolpmann wrote:
> http://blog.camlcity.org/blog/bytes1.html

Coming back to motivating example of this post.

Lexing provides:

val from_channel : in_channel -> lexbuf
val from_string : string -> lexbuf
val from_function : (bytes -> int -> int) -> lexbuf

In particular, from_function expects you to write to a buffer, so it's 
pretty clear that its callback must accept a "bytes", not a "string". 
There is no place for a (string -> int -> int) -> lexbuf function.

Concerning from_string: this function copies the string to an internal 
buffer.  This is purely implemented on the OCaml side without any unsafe 
features.  We could avoid this copy because we know that the generated 
lexers won't actually modify the buffer in that case, but it would be 
very difficult to do this without using an unsafe feature, even if we 
had some sort of generalization of bytes and string.  We would instead 
need a completely different implementation (which would not use 
"stringable" to make the "source" (string or "stream") explicit in the 
lexbuf datastructure.

We could also provide an extra from_bytes function, but it can currently 
be implemented by composing Bytes.to_string and Lexing.from_string.  Are 
you concerned only by the performance overhead of this approach (two 
copies)?  If so, the same argument would apply to the current 
implementation of from_string, and we would need to switch to a 
different approach, for which it's not clear that "stringable" would be 
a big help (see above).  Before doing anything like that, it would be 
interesting to evaluate the exact overhead.  It could very well be 
negligible/acceptable for most cases compared to the cost of actual lexing.

-- Alain