HISTSIZE

HISTSIZE

[Hrvoje Niksic]

I have noticed that when I set HISTSIZE (the maximum size of the
history list) in zsh to something huge, zsh takes huge amounts of
memory.

When I converted from bash, I have easily ported all the init files,
and HISTSIZE caused terrible problems (especially since I didn't know
HISTSIZE was causing them). The reason I set HISTSIZE to such a big
value is just to keep it from bothering me -- it is just a *maximum*
value I am changing. In bash it worked exactly that way (no additional
memory expense). But it seems that the organization of history storage
requires resizehistevents() to reallocate the list to HISTSIZE history
entries.

I wonder how difficult would it be to patch hist.c in such a way that
the actual size of the history list changes dynamically, HISTSIZE
being only an upper limit to that change.

-- 
hniksic@srce.hr              |  Student of electrical engineering
hniksic@fly.cc.fer.hr        |  University of Zagreb, Croatia
------------------------------------------------------------------
"Learning XEmacs is a lifelong activity. I've been using Emacs for over
a decade now, and I'm still discovering new features." -- from XEmacs FAQ

Re: HISTSIZE

[Zoltan Hidvegi]

> I have noticed that when I set HISTSIZE (the maximum size of the
> history list) in zsh to something huge, zsh takes huge amounts of
> memory.
> 
> When I converted from bash, I have easily ported all the init files,
> and HISTSIZE caused terrible problems (especially since I didn't know
> HISTSIZE was causing them). The reason I set HISTSIZE to such a big
> value is just to keep it from bothering me -- it is just a *maximum*
> value I am changing. In bash it worked exactly that way (no additional
> memory expense). But it seems that the organization of history storage
> requires resizehistevents() to reallocate the list to HISTSIZE history
> entries.
> 
> I wonder how difficult would it be to patch hist.c in such a way that
> the actual size of the history list changes dynamically, HISTSIZE
> being only an upper limit to that change.

Zsh uses a HISTSIZE long array to store history entries.  gethistent(X) is
defined as (histentarr+((X)%histentct)) (with proper bounds checking at
apropriate places).  This is usually the most compact representation.  A
linked list can be used but then the pointers would require extra memory.
histentarr is almost full in most cases since it is filled from $HISTFILE
upon invocation.  Most people do like to limit the number of stored history
events since loading and saving big histories takes a long time.  Of course
SAVEHIST can be smaller than HISTSIZE but most people do not issue more
than 1000 commands from one shell so HISTSIZE = SAVEHIST + 1000 should be
enough.

Zoltan

Re: HISTSIZE

[Hrvoje Niksic]

Zoltan Hidvegi (hzoli@cs.elte.hu) wrote:
> > I wonder how difficult would it be to patch hist.c in such a way that
> > the actual size of the history list changes dynamically, HISTSIZE
> > being only an upper limit to that change.
> 
> Zsh uses a HISTSIZE long array to store history entries.  gethistent(X) is
> defined as (histentarr+((X)%histentct)) (with proper bounds checking at
> apropriate places).  This is usually the most compact representation.  A
> linked list can be used but then the pointers would require extra memory.
> histentarr is almost full in most cases since it is filled from $HISTFILE
> upon invocation.  Most people do like to limit the number of stored history
> events since loading and saving big histories takes a long time.  Of course
> SAVEHIST can be smaller than HISTSIZE but most people do not issue more
> than 1000 commands from one shell so HISTSIZE = SAVEHIST + 1000 should be
> enough.

Of course one must be able to limit the number of history items. But
if you want to unlimit it (which I like to do) by setting it to
100,000 or to a 1,000,000, is where the problems arise.

As far as I have seen, bash also represents the history list as an
array, but it starts off with a default size and realloc's it
dynamically each 50 or so commands. This way HISTSIZE is only a limit,
not a memory consumer.

So: gethistent(X) would still be equally defined, but the array would
change its size dynamically. Is this doable?

-- 
hniksic@srce.hr              |  Student of electrical engineering
hniksic@fly.cc.fer.hr        |  University of Zagreb, Croatia
------------------------------------------------------------------
I'm a Lisp variable -- bind me!