[Caml-list] Buffer.add error

[Caml-list] Buffer.add error

[Mikkel Christiansen]

Hi

I'm having  trouble with the buffer library in ocaml (version 3.06 - 
debian). Program
was compiled to native using ocamlopt.

When generating a rather large string I get the following error:

    Fatal error: exception Failure("Buffer.add: cannot grow buffer")

Is there any way to avoid this (except writing directly to a file)?

Thanks

    -Mikkel





-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

[Caml-list] Buffer.add error

[Basile STARYNKEVITCH]

>>>>> "Mikkel" == Mikkel Christiansen <mixxel@cs.auc.dk> writes:

    Mikkel> Hi I'm having trouble with the buffer library in ocaml
    Mikkel> (version 3.06 - debian). Program was compiled to native
    Mikkel> using ocamlopt.

    Mikkel> When generating a rather large string I get the following
    Mikkel> error:

    Mikkel>     Fatal error: exception Failure("Buffer.add: cannot
    Mikkel> grow buffer")

    Mikkel> Is there any way to avoid this (except writing directly to
    Mikkel> a file)?


I suggest looking into the sources. In ocaml/stdlib/buffer.ml you can
read (near line 43)

    if b.position + more <= Sys.max_string_length
    then new_len := Sys.max_string_length
    else failwith "Buffer.add: cannot grow buffer"

So basically, you are hitting the limits of maximal string size (which
is 2^24 = 16777216 on 32 bits plateform).

Hence you cannot avoid it, except by switching to a 64 bits plateform.

You might change your algorithms - perhaps by keeping in memory a
vector of buffers or lines or paragraphs (whatever these are in your
application).




-- 

Basile STARYNKEVITCH         http://starynkevitch.net/Basile/ 
email: basile<at>starynkevitch<dot>net 
alias: basile<at>tunes<dot>org 
8, rue de la Faïencerie, 92340 Bourg La Reine, France
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Buffer.add error

[Eric C. Cooper]

On Wed, Jan 08, 2003 at 11:08:45AM +0100, Basile STARYNKEVITCH wrote:
> >>>>> "Mikkel" == Mikkel Christiansen <mixxel@cs.auc.dk> writes:
>     Mikkel> When generating a rather large string I get the following
>     Mikkel> error:
>     Mikkel>     Fatal error: exception Failure("Buffer.add: cannot
>     Mikkel> grow buffer")
> ... 
> So basically, you are hitting the limits of maximal string size (which
> is 2^24 = 16777216 on 32 bits plateform).
> 
> Hence you cannot avoid it, except by switching to a 64 bits plateform.

Another possibility is to write a Buffer implementation on top of
Bigarrays.  That way your application wouldn't have to change much.

-- 
Eric C. Cooper          e c c @ c m u . e d u
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

Re: [Caml-list] Buffer.add error

[Xavier Leroy]

> > So basically, you are hitting the limits of maximal string size (which
> > is 2^24 = 16777216 on 32 bits plateform).
> > 
> > Hence you cannot avoid it, except by switching to a 64 bits plateform.
> 
> Another possibility is to write a Buffer implementation on top of
> Bigarrays.  That way your application wouldn't have to change much.

But you'd still hit the limit on string length when extracting the
contents of the buffer with Buffer.contents...

- Xavier Leroy
-------------------
To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr
Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners