zsh-workers
 help / color / mirror / code / Atom feed
* Patch for curses module
@ 2015-09-11 17:35 Sebastian Gniazdowski
  2015-09-12 14:44 ` Sebastian Gniazdowski
  0 siblings, 1 reply; 13+ messages in thread
From: Sebastian Gniazdowski @ 2015-09-11 17:35 UTC (permalink / raw)
  To: zsh-workers


[-- Attachment #1.1: Type: text/plain, Size: 1549 bytes --]

Hello,
while working on Zsh Navigation Tools I encountered random changing,
blinking of colors (in n-options). This not always reproduced, but luckily
on one FreeBSD box it did always reproduce.

After going into the code I established that following code in curses.c:

    if (zc_color_phase==1 ||
        !(cpn = (Colorpairnode) gethashnode(zcurses_colorpairs,
colorpair))) {

should be:

    if (zc_color_phase==1 ||
        !(cpn = (Colorpairnode) gethashnode2(zcurses_colorpairs,
colorpair))) {

The author defined the hash table with gethashnode2() from the beginning:

            zcurses_colorpairs->getnode     = gethashnode2;
            zcurses_colorpairs->getnode2    = gethashnode2;

However there he called the non-2 version.

Result was as follows:
- only first initialized color pair was found by the non-2 gethashnode()
function, in my case white/black
- each call to set a color pair (zcurses attr) other than the white/black
caused a miss in hash table, and creation of new entry
- I easily obtained number of color pairs like 300 when actually using only
three colors (color pairs)

It seems that gethashnode() can return first hash table entry, but not
others, when called on a hash table like the author's.

I wonder what can be done to be able to use colors without memory leaks or
at least without occasional blinking on zsh <= 5.1. It's not good that so
many zsh's are affected by the bug, making colors in zcurses problematic in
near future if user doesn't obtain a very fresh version.

Best regards,
Sebastian Gniazdowski

[-- Attachment #1.2: Type: text/html, Size: 1897 bytes --]

[-- Attachment #2: curses.patch --]
[-- Type: application/octet-stream, Size: 456 bytes --]

diff --git a/Src/Modules/curses.c b/Src/Modules/curses.c
index 62dbd55..0054aef 100644
--- a/Src/Modules/curses.c
+++ b/Src/Modules/curses.c
@@ -339,7 +339,7 @@ zcurses_colorget(const char *nam, char *colorpair)
 	return NULL;
 
     if (zc_color_phase==1 ||
-	!(cpn = (Colorpairnode) gethashnode(zcurses_colorpairs, colorpair))) {
+	!(cpn = (Colorpairnode) gethashnode2(zcurses_colorpairs, colorpair))) {
 	zc_color_phase = 2;
 	cp = ztrdup(colorpair);
 

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-11 17:35 Patch for curses module Sebastian Gniazdowski
@ 2015-09-12 14:44 ` Sebastian Gniazdowski
  2015-09-12 14:57   ` Mikael Magnusson
  2015-09-12 16:08   ` Bart Schaefer
  0 siblings, 2 replies; 13+ messages in thread
From: Sebastian Gniazdowski @ 2015-09-12 14:44 UTC (permalink / raw)
  To: zsh-workers

[-- Attachment #1: Type: text/plain, Size: 3356 bytes --]

Hello,
I was looking for a way to be able to use colors on zsh < 5.1.1. Did
some debugging:

n-preview:zcurses:92: Will ask for colorpair `white/black' [max
cp:32767][colors-1:255]
n-preview:zcurses:92: -- Will create colorpair `white/black'
n-preview:---:92: cp:default/default, pairnum:0, hashval:1
n-preview:zcurses:92: Found zcurses colors f[7], b[0]
n-preview:zcurses:92: Creating colorpair `white/black' num[1], CPnum[256]
n-preview:zcurses:92: Returning colorpair `white/black' num[1] CPnum[256]
n-preview:zcurses:93: Will ask for colorpair `white/black' [max
cp:32767][colors-1:255]
n-preview:==:93: nam:white/black, hashval:5
n-preview:==:93: nam:white/black, hashval:5 DISABLED return
n-preview:zcurses:93: -- Will create colorpair `white/black'
n-preview:---:93: cp:default/default, pairnum:0, hashval:1
n-preview:---:93: cp:white/black, pairnum:1, hashval:5
n-preview:zcurses:93: Found zcurses colors f[7], b[0]
n-preview:zcurses:93: Creating colorpair `white/black' num[2], CPnum[512]
n-preview:zcurses:93: Returning colorpair `white/black' num[2] CPnum[512]
n-preview:zcurses:94: Will ask for colorpair `white/black' [max
cp:32767][colors-1:255]
n-preview:==:94: nam:white/black, hashval:5
n-preview:==:94: nam:white/black, hashval:5 DISABLED return
n-preview:zcurses:94: -- Will create colorpair `white/black'
n-preview:---:94: cp:default/default, pairnum:0, hashval:1
n-preview:---:94: cp:white/black, pairnum:2, hashval:5
n-preview:zcurses:94: Found zcurses colors f[7], b[0]
n-preview:zcurses:94: Creating colorpair `white/black' num[3], CPnum[768]
n-preview:zcurses:94: Returning colorpair `white/black' num[3] CPnum[768]


As it can be seen, hash misses are caused by the DISABLED flag ( "if
(hp->flags & DISABLED) {" in function gethashnode). I attach two files
with debug prints that produced the output. --- lines are hash table
dumps, == are gethashnode() interiors. Can someone explain why
sometimes a node has the DISABLED flag? It's possible that e.g.
initializing two dummy color pairs after the ones that will be
actually used will guarantee them to work. Here red/black fixed the
white/black, but knowing more about the like random changes of
DISABLED flag is needed.

_nlist_print_with_ansi:zcurses:59: Will ask for colorpair `red/black'
[max cp:32767][colors-1:255]
_nlist_print_with_ansi:==:59: nam:red/black, hashval:7
_nlist_print_with_ansi:zcurses:59: -- Will create colorpair `red/black'
_nlist_print_with_ansi:---:59: cp:default/default, pairnum:0, hashval:1
_nlist_print_with_ansi:---:59: cp:white/black, pairnum:4, hashval:5
_nlist_print_with_ansi:zcurses:59: Found zcurses colors f[1], b[0]
_nlist_print_with_ansi:zcurses:59: Creating colorpair `red/black'
num[5], CPnum[1280]
_nlist_print_with_ansi:zcurses:59: Returning colorpair `red/black'
num[5] CPnum[1280]
_nlist_print_with_ansi:zcurses:65: Will ask for colorpair
`white/black' [max cp:32767][colors-1:255]
_nlist_print_with_ansi:==:65: nam:white/black, hashval:5
_nlist_print_with_ansi:==:65: nam:white/black, hashval:5 RETURN (<-- FIXED)
_nlist_print_with_ansi:zcurses:65: Returning colorpair `white/black'
num[4] CPnum[1024]

It's possible that the code still has bugs about the flag.

I clarified that the blinking is caused by creating more than 256
color pairs (it's a long hostory about color handling in curses).

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: curses.c --]
[-- Type: text/x-csrc, Size: 39995 bytes --]

/*
 * curses.c - curses windowing module for zsh
 *
 * This file is part of zsh, the Z shell.
 *
 * Copyright (c) 2007  Clint Adams
 * All rights reserved.
 *
 * Permission is hereby granted, without written agreement and without
 * license or royalty fees, to use, copy, modify, and distribute this
 * software and to distribute modified versions of this software for any
 * purpose, provided that the above copyright notice and the following
 * two paragraphs appear in all copies of this software.
 *
 * In no event shall Clint Adams or the Zsh Development Group be liable
 * to any party for direct, indirect, special, incidental, or consequential
 * damages arising out of the use of this software and its documentation,
 * even if Clint Adams and the Zsh Development Group have been advised of
 * the possibility of such damage.
 *
 * Clint Adams and the Zsh Development Group specifically disclaim any
 * warranties, including, but not limited to, the implied warranties of
 * merchantability and fitness for a particular purpose.  The software
 * provided hereunder is on an "as is" basis, and Clint Adams and the
 * Zsh Development Group have no obligation to provide maintenance,
 * support, updates, enhancements, or modifications.
 *
 */

#define ZSH_CURSES_SOURCE 1

#include "curses.mdh"
#include "curses.pro"

#ifndef MULTIBYTE_SUPPORT
# undef HAVE_GETCCHAR
# undef HAVE_SETCCHAR
# undef HAVE_WADDWSTR
# undef HAVE_WGET_WCH
# undef HAVE_WIN_WCH
# undef HAVE_NCURSESW_NCURSES_H
#endif

#ifdef ZSH_HAVE_CURSES_H
# include "../zshcurses.h"
#endif

#ifdef HAVE_SETCCHAR
# include <wchar.h>
#endif

#include <stdio.h>

void printhashtabinfo(HashTable ht);

enum zc_win_flags {
    /* Window is permanent (probably "stdscr") */
    ZCWF_PERMANENT = 0x0001,
    /* Scrolling enabled */
    ZCWF_SCROLL = 0x0002
};

typedef struct zc_win *ZCWin;

struct zc_win {
    WINDOW *win;
    char *name;
    int flags;
    LinkList children;
    ZCWin parent;
};

struct zcurses_namenumberpair {
    char *name;
    int number;
};

struct colorpairnode {
    struct hashnode node;
    short colorpair;
};
typedef struct colorpairnode *Colorpairnode;

typedef int (*zccmd_t)(const char *nam, char **args);
struct zcurses_subcommand {
    const char *name;
    zccmd_t cmd;
    int minargs;
    int maxargs;
};

static struct ttyinfo saved_tty_state;
static struct ttyinfo curses_tty_state;
static LinkList zcurses_windows;
static HashTable zcurses_colorpairs = NULL;
#ifdef NCURSES_MOUSE_VERSION
/*
 * The following is in principle a general set of flags, but
 * is currently only needed for mouse status.
 */
static int zcurses_flags;
#endif

#define ZCURSES_EINVALID 1
#define ZCURSES_EDEFINED 2
#define ZCURSES_EUNDEFINED 3

#define ZCURSES_UNUSED 1
#define ZCURSES_USED 2

#define ZCURSES_ATTRON 1
#define ZCURSES_ATTROFF 2

static int zc_errno, zc_color_phase=0;
static short next_cp=0;

enum {
    ZCF_MOUSE_ACTIVE,
    ZCF_MOUSE_MASK_CHANGED
};

static const struct zcurses_namenumberpair zcurses_attributes[] = {
    {"blink", A_BLINK},
    {"bold", A_BOLD},
    {"dim", A_DIM},
    {"reverse", A_REVERSE},
    {"standout", A_STANDOUT},
    {"underline", A_UNDERLINE},
    {NULL, 0}
};

static const struct zcurses_namenumberpair zcurses_colors[] = {
    {"black", COLOR_BLACK},
    {"red", COLOR_RED},
    {"green", COLOR_GREEN},
    {"yellow", COLOR_YELLOW},
    {"blue", COLOR_BLUE},
    {"magenta", COLOR_MAGENTA},
    {"cyan", COLOR_CYAN},
    {"white", COLOR_WHITE},
#ifdef HAVE_USE_DEFAULT_COLORS
    {"default", -1},
#endif
    {NULL, 0}
};

#ifdef NCURSES_MOUSE_VERSION
enum zcurses_mouse_event_types {
    ZCME_PRESSED,
    ZCME_RELEASED,
    ZCME_CLICKED,
    ZCME_DOUBLE_CLICKED,
    ZCME_TRIPLE_CLICKED
};

static const struct zcurses_namenumberpair zcurses_mouse_event_list[] = {
    {"PRESSED", ZCME_PRESSED},
    {"RELEASED", ZCME_RELEASED},
    {"CLICKED", ZCME_CLICKED},
    {"DOUBLE_CLICKED", ZCME_DOUBLE_CLICKED},
    {"TRIPLE_CLICKED", ZCME_TRIPLE_CLICKED},
    {NULL, 0}
};

struct zcurses_mouse_event {
    int button;
    int what;
    mmask_t event;
};

static const struct zcurses_mouse_event zcurses_mouse_map[] = {
    { 1, ZCME_PRESSED, BUTTON1_PRESSED },
    { 1, ZCME_RELEASED, BUTTON1_RELEASED },
    { 1, ZCME_CLICKED, BUTTON1_CLICKED },
    { 1, ZCME_DOUBLE_CLICKED, BUTTON1_DOUBLE_CLICKED },
    { 1, ZCME_TRIPLE_CLICKED, BUTTON1_TRIPLE_CLICKED },

    { 2, ZCME_PRESSED, BUTTON2_PRESSED },
    { 2, ZCME_RELEASED, BUTTON2_RELEASED },
    { 2, ZCME_CLICKED, BUTTON2_CLICKED },
    { 2, ZCME_DOUBLE_CLICKED, BUTTON2_DOUBLE_CLICKED },
    { 2, ZCME_TRIPLE_CLICKED, BUTTON2_TRIPLE_CLICKED },

    { 3, ZCME_PRESSED, BUTTON3_PRESSED },
    { 3, ZCME_RELEASED, BUTTON3_RELEASED },
    { 3, ZCME_CLICKED, BUTTON3_CLICKED },
    { 3, ZCME_DOUBLE_CLICKED, BUTTON3_DOUBLE_CLICKED },
    { 3, ZCME_TRIPLE_CLICKED, BUTTON3_TRIPLE_CLICKED },

    { 4, ZCME_PRESSED, BUTTON4_PRESSED },
    { 4, ZCME_RELEASED, BUTTON4_RELEASED },
    { 4, ZCME_CLICKED, BUTTON4_CLICKED },
    { 4, ZCME_DOUBLE_CLICKED, BUTTON4_DOUBLE_CLICKED },
    { 4, ZCME_TRIPLE_CLICKED, BUTTON4_TRIPLE_CLICKED },

#ifdef BUTTON5_PRESSED
    /* Not defined if only 32 bits available */
    { 5, ZCME_PRESSED, BUTTON5_PRESSED },
    { 5, ZCME_RELEASED, BUTTON5_RELEASED },
    { 5, ZCME_CLICKED, BUTTON5_CLICKED },
    { 5, ZCME_DOUBLE_CLICKED, BUTTON5_DOUBLE_CLICKED },
    { 5, ZCME_TRIPLE_CLICKED, BUTTON5_TRIPLE_CLICKED },
#endif
    { 0, 0, 0 }
};

mmask_t zcurses_mouse_mask = ALL_MOUSE_EVENTS;

#endif

/* Autogenerated keypad string/number mapping*/
#include "curses_keys.h"

static char **
zcurses_pairs_to_array(const struct zcurses_namenumberpair *nnps)
{
    char **arr, **arrptr;
    int count;
    const struct zcurses_namenumberpair *nnptr;

    for (nnptr = nnps; nnptr->name; nnptr++)
	;
    count = nnptr - nnps;

    arrptr = arr = (char **)zhalloc((count+1) * sizeof(char *));

    for (nnptr = nnps; nnptr->name; nnptr++)
	*arrptr++ = dupstring(nnptr->name);
    *arrptr = NULL;

    return arr;
}

static const char *
zcurses_strerror(int err)
{
    static const char *errs[] = {
	"unknown error",
	"window name invalid",
	"window already defined",
	"window undefined",
	NULL };

    return errs[(err < 1 || err > 3) ? 0 : err];
}

static LinkNode
zcurses_getwindowbyname(const char *name)
{
    LinkNode node;
    ZCWin w;

    for (node = firstnode(zcurses_windows); node; incnode(node))
	if (w = (ZCWin)getdata(node), !strcmp(w->name, name))
	    return node;

    return NULL;
}

static LinkNode
zcurses_validate_window(char *win, int criteria)
{
    LinkNode target;

    if (win==NULL || strlen(win) < 1) {
	zc_errno = ZCURSES_EINVALID;
	return NULL;
    }

    target = zcurses_getwindowbyname(win);

    if (target && (criteria & ZCURSES_UNUSED)) {
	zc_errno = ZCURSES_EDEFINED;
	return NULL;
    }

    if (!target && (criteria & ZCURSES_USED)) {
	zc_errno = ZCURSES_EUNDEFINED;
	return NULL;
    }

    zc_errno = 0;
    return target;
}

static int
zcurses_free_window(ZCWin w)
{
    if (!(w->flags & ZCWF_PERMANENT) && delwin(w->win)!=OK)
	return 1;

    if (w->name)
	zsfree(w->name);

    if (w->children)
	freelinklist(w->children, (FreeFunc)NULL);

    zfree(w, sizeof(struct zc_win));

    return 0;
}

static struct zcurses_namenumberpair *
zcurses_attrget(WINDOW *w, char *attr)
{
    struct zcurses_namenumberpair *zca;

    if (!attr)
	return NULL;

    for(zca=(struct zcurses_namenumberpair *)zcurses_attributes;zca->name;zca++)
	if (!strcmp(attr, zca->name)) {
	    return zca;
	}

    return NULL;
}

static short
zcurses_color(const char *color)
{
    struct zcurses_namenumberpair *zc;

    for(zc=(struct zcurses_namenumberpair *)zcurses_colors;zc->name;zc++)
	if (!strcmp(color, zc->name)) {
	    return (short)zc->number;
	}

    return (short)-1;
}

static void dump_zcurses_colorpairs();

static Colorpairnode
zcurses_colorget(const char *nam, char *colorpair)
{
    char *bg, *cp;
    short f, b;
    Colorpairnode cpn;

    /* zcurses_colorpairs is only initialised if color is supported */
    if (!zcurses_colorpairs)
	return NULL;

    zwarnnam(nam, "Will ask for colorpair `%s' [max cp:%d][colors-1:%d]", colorpair, COLOR_PAIRS, COLORS-1);

    if (zc_color_phase==1 ||
	!(cpn = (Colorpairnode) gethashnode(zcurses_colorpairs, colorpair))) {

        zwarnnam(nam, "-- Will create colorpair `%s'", colorpair);
        dump_zcurses_colorpairs();

	zc_color_phase = 2;
	cp = ztrdup(colorpair);

	bg = strchr(cp, '/');
	if (bg==NULL) {
	    zsfree(cp);
	    return NULL;
	}

	*bg = '\0';        
	f = zcurses_color(cp);
	b = zcurses_color(bg+1);

	if (f==-1 || b==-1) {
	    if (f == -1)
		zwarnnam(nam, "foreground color `%s' not known", cp);
	    if (b == -1)
		zwarnnam(nam, "background color `%s' not known", bg+1);
	    *bg = '/';
	    zsfree(cp);
	    return NULL;
	} else {
            zwarnnam(nam, "Found zcurses colors f[%d], b[%d]", f, b);
        }
	*bg = '/';

	++next_cp;
	if (next_cp >= COLOR_PAIRS || init_pair(next_cp, f, b) == ERR)  {
	    zsfree(cp);
	    return NULL;
	}

	cpn = (Colorpairnode)zalloc(sizeof(struct colorpairnode));
	
	if (!cpn) {
	    zsfree(cp);
	    return NULL;
	}

	cpn->colorpair = next_cp;

        zwarnnam(nam, "Creating colorpair `%s' num[%d], CPnum[%d]", cp, next_cp, COLOR_PAIR(next_cp));

	addhashnode(zcurses_colorpairs, cp, (void *)cpn);
    }

    zwarnnam(nam, "Returning colorpair `%s' num[%d] CPnum[%d]", colorpair, cpn->colorpair, COLOR_PAIR(cpn->colorpair));

    return cpn;
}

static Colorpairnode cpn_match;

static void
zcurses_colornode(HashNode hn, int cp)
{
    Colorpairnode cpn = (Colorpairnode)hn;
    if (cpn->colorpair == (short)cp)
	cpn_match = cpn;
}

static Colorpairnode
zcurses_colorget_reverse(short cp)
{
    if (!zcurses_colorpairs)
	return NULL;

    cpn_match = NULL;
    scanhashtable(zcurses_colorpairs, 0, 0, 0,
		  zcurses_colornode, cp);
    return cpn_match;
}

static void
dump_node(HashNode hn, int cp)
{
    Colorpairnode cpn = (Colorpairnode)hn;
    int hashval;
    hashval = zcurses_colorpairs->hash(hn->nam) % zcurses_colorpairs->hsize;
    zwarnnam("---", "cp:%s, pairnum:%d, hashval:%d", hn->nam, cpn->colorpair, hashval);
}

static void
dump_zcurses_colorpairs()
{
    if (!zcurses_colorpairs) {
        zwarnnam("---", "Hash table is NULL");
        return;
    }

    cpn_match = NULL;
    scanhashtable(zcurses_colorpairs, 0, 0, 0, dump_node, -1);
}


static void
freecolorpairnode(HashNode hn)
{
    zsfree(hn->nam);
    zfree(hn, sizeof(struct colorpairnode));
}


/*************
 * Subcommands
 *************/

static int
zccmd_init(const char *nam, char **args)
{
    LinkNode stdscr_win = zcurses_getwindowbyname("stdscr");

    if (!stdscr_win) {
	ZCWin w = (ZCWin)zshcalloc(sizeof(struct zc_win));
	if (!w)
	    return 1;

	gettyinfo(&saved_tty_state);
	w->name = ztrdup("stdscr");
	w->win = initscr();
	if (w->win == NULL) {
	    zsfree(w->name);
	    zfree(w, sizeof(struct zc_win));
	    return 1;
	}
	w->flags = ZCWF_PERMANENT;
	zinsertlinknode(zcurses_windows, lastnode(zcurses_windows), (void *)w);
	if (start_color() != ERR) {
	    Colorpairnode cpn;

	    if(!zc_color_phase)
		zc_color_phase = 1;
	    zcurses_colorpairs = newhashtable(8, "zc_colorpairs", NULL);

	    zcurses_colorpairs->hash        = hasher;
	    zcurses_colorpairs->emptytable  = emptyhashtable;
	    zcurses_colorpairs->filltable   = NULL;
	    zcurses_colorpairs->cmpnodes    = strcmp;
	    zcurses_colorpairs->addnode     = addhashnode;
	    zcurses_colorpairs->getnode     = gethashnode2;
	    zcurses_colorpairs->getnode2    = gethashnode2;
	    zcurses_colorpairs->removenode  = removehashnode;
	    zcurses_colorpairs->disablenode = NULL;
	    zcurses_colorpairs->enablenode  = NULL;
	    zcurses_colorpairs->freenode    = freecolorpairnode;
	    zcurses_colorpairs->printnode   = (void*)-1;

#ifdef HAVE_USE_DEFAULT_COLORS
	    use_default_colors();
#endif
	    /* Initialise the default color pair, always 0 */
	    cpn = (Colorpairnode)zalloc(sizeof(struct colorpairnode));
	    if (cpn) {
		cpn->colorpair = 0;
		addhashnode(zcurses_colorpairs,
			    ztrdup("default/default"), (void *)cpn);
	    }
	}
	/*
	 * We use cbreak mode because we don't want line buffering
	 * on input since we'd just need to loop over characters.
	 * We use noecho since the manual says that's the right
	 * thing to do with cbreak.
	 *
	 * Turn these on immediately to catch typeahead.
	 */
	cbreak();
	noecho();
	gettyinfo(&curses_tty_state);
    } else {
	settyinfo(&curses_tty_state);
    }
    return 0;
}


static int
zccmd_addwin(const char *nam, char **args)
{
    int nlines, ncols, begin_y, begin_x;
    ZCWin w;

    if (zcurses_validate_window(args[0], ZCURSES_UNUSED) == NULL &&
	zc_errno) {
	zerrnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0], 0);
	return 1;
    }

    nlines = atoi(args[1]);
    ncols = atoi(args[2]);
    begin_y = atoi(args[3]);
    begin_x = atoi(args[4]);

    w = (ZCWin)zshcalloc(sizeof(struct zc_win));
    if (!w)
	return 1;

    w->name = ztrdup(args[0]);
    if (args[5]) {
	LinkNode node;
	ZCWin worig;

	node = zcurses_validate_window(args[5], ZCURSES_USED);
	if (node == NULL) {
	    zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0],
		     0);
	    zsfree(w->name);
	    zfree(w, sizeof(struct zc_win));
	    return 1;
	}

	worig = (ZCWin)getdata(node);

	w->win = subwin(worig->win, nlines, ncols, begin_y, begin_x);
	if (w->win) {
	    w->parent = worig;
	    if (!worig->children)
		worig->children = znewlinklist();
	    zinsertlinknode(worig->children, lastnode(worig->children),
			    (void *)w);
	}
    } else {
	w->win = newwin(nlines, ncols, begin_y, begin_x);
    }

    if (w->win == NULL) {
	zwarnnam(nam, "failed to create window `%s'", w->name);
	zsfree(w->name);
	zfree(w, sizeof(struct zc_win));
	return 1;
    }

    zinsertlinknode(zcurses_windows, lastnode(zcurses_windows), (void *)w);

    return 0;
}

static int
zccmd_delwin(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    int ret = 0;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    if (w == NULL) {
	zwarnnam(nam, "record for window `%s' is corrupt", args[0]);
	return 1;
    }
    if (w->flags & ZCWF_PERMANENT) {
	zwarnnam(nam, "window `%s' can't be deleted", args[0]);
	return 1;
    }

    if (w->children && firstnode(w->children)) {
	zwarnnam(nam, "window `%s' has subwindows, delete those first",
		 w->name);
	return 1;
    }

    if (delwin(w->win)!=OK) {
	/*
	 * Not sure what to do here, but we are probably stuffed,
	 * so delete the window locally anyway.
	 */
	ret = 1;
    }

    if (w->parent) {
	/* Remove from parent's list of children */
	LinkList wpc = w->parent->children;
	LinkNode pcnode;
	for (pcnode = firstnode(wpc); pcnode; incnode(pcnode)) {
	    ZCWin child = (ZCWin)getdata(pcnode);
	    if (child == w) {
		remnode(wpc, pcnode);
		break;
	    }
	}
	DPUTS(pcnode == NULL, "BUG: child node not found in parent's children");
	/*
	 * We need to touch the parent to get the parent to refresh
	 * properly.
	 */
	touchwin(w->parent->win);
    }
    else
	touchwin(stdscr);

    if (w->name)
	zsfree(w->name);

    zfree((ZCWin)remnode(zcurses_windows, node), sizeof(struct zc_win));

    return ret;
}


static int
zccmd_refresh(const char *nam, char **args)
{
    WINDOW *win;
    int ret = 0;

    if (args[0]) {
	for (; *args; args++) {
	    LinkNode node;
	    ZCWin w;

	    node = zcurses_validate_window(args[0], ZCURSES_USED);
	    if (node == NULL) {
		zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0],
			 0);
		return 1;
	    }

	    w = (ZCWin)getdata(node);

	    if (w->parent) {
		/* This is what the manual says you have to do. */
		touchwin(w->parent->win);
	    }
	    win = w->win;
	    if (wnoutrefresh(win) != OK)
		ret = 1;
	}
	return (doupdate() != OK || ret);
    }
    else
    {
	return (wrefresh(stdscr) != OK) ? 1 : 0;
    }
}


static int
zccmd_move(const char *nam,  char **args)
{
    int y, x;
    LinkNode node;
    ZCWin w;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    y = atoi(args[1]);
    x = atoi(args[2]);

    w = (ZCWin)getdata(node);

    if (wmove(w->win, y, x)!=OK)
	return 1;

    return 0;
}


static int
zccmd_clear(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    if (!args[1]) {
	return werase(w->win) != OK;
    } else if (!strcmp(args[1], "redraw")) {
	return wclear(w->win) != OK;
    } else if (!strcmp(args[1], "eol")) {
	return wclrtoeol(w->win) != OK;
    } else if (!strcmp(args[1], "bot")) {
	return wclrtobot(w->win) != OK;
    } else {
	zwarnnam(nam, "`clear' expects `redraw', `eol' or `bot'");
	return 1;
    }
}


static int
zccmd_char(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
#ifdef HAVE_SETCCHAR
    wchar_t c;
    cchar_t cc;
#endif

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

#ifdef HAVE_SETCCHAR
    if (mbrtowc(&c, args[1], MB_CUR_MAX, NULL) < 1)
	return 1;

    if (setcchar(&cc, &c, A_NORMAL, 0, NULL)==ERR)
	return 1;

    if (wadd_wch(w->win, &cc)!=OK)
	return 1;
#else
    if (waddch(w->win, (chtype)args[1][0])!=OK)
	return 1;
#endif

    return 0;
}


static int
zccmd_string(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;

#ifdef HAVE_WADDWSTR
    int clen;
    wint_t wc;
    wchar_t *wstr, *wptr;
    char *str = args[1];
#endif

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

#ifdef HAVE_WADDWSTR
    mb_charinit();
    wptr = wstr = zhalloc((strlen(str)+1) * sizeof(wchar_t));

    while (*str && (clen = mb_metacharlenconv(str, &wc))) {
	str += clen;
	if (wc == WEOF) /* TODO: replace with space? nicen? */
	    continue;
	*wptr++ = wc;
    }
    *wptr++ = L'\0';
    if (waddwstr(w->win, wstr)!=OK) {
	return 1;
    }
#else
    if (waddstr(w->win, args[1])!=OK)
	return 1;
#endif
    return 0;
}


static int
zccmd_border(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    if (wborder(w->win, 0, 0, 0, 0, 0, 0, 0, 0)!=OK)
	return 1;

    return 0;
}


static int
zccmd_endwin(const char *nam, char **args)
{
    LinkNode stdscr_win = zcurses_getwindowbyname("stdscr");

    if (stdscr_win) {
	endwin();
	/* Restore TTY as it was before zcurses -i */
	settyinfo(&saved_tty_state);
	/*
	 * TODO: should I need the following?  Without it
	 * the screen stays messed up.  Presumably we are
	 * doing stuff with shttyinfo when we shouldn't really be.
	 */
	gettyinfo(&shttyinfo);
    }
    return 0;
}


static int
zccmd_attr(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    char **attrs;
    int ret = 0;

    if (!args[0])
	return 1;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    for(attrs = args+1; *attrs; attrs++) {
	if (strchr(*attrs, '/')) {
	    Colorpairnode cpn;
	    if ((cpn = zcurses_colorget(nam, *attrs)) == NULL ||
		wcolor_set(w->win, cpn->colorpair, NULL) == ERR)
		ret = 1;
	} else {
	    char *ptr;
	    int onoff;
	    struct zcurses_namenumberpair *zca;

	    switch(*attrs[0]) {
	    case '-':
		onoff = ZCURSES_ATTROFF;
		ptr = (*attrs) + 1;
		break;
	    case '+':
		onoff = ZCURSES_ATTRON;
		ptr = (*attrs) + 1;
		break;
	    default:
		onoff = ZCURSES_ATTRON;
		ptr = *attrs;
		break;
	    }
	    if ((zca = zcurses_attrget(w->win, ptr)) == NULL) {
		zwarnnam(nam, "attribute `%s' not known", ptr);
		ret = 1;
	    } else {
		switch(onoff) {
		    case ZCURSES_ATTRON:
			if (wattron(w->win, zca->number) == ERR)
			    ret = 1;
			break;
		    case ZCURSES_ATTROFF:
			if (wattroff(w->win, zca->number) == ERR)
			    ret = 1;
			break;
		}
	    }
	}
    }
    return ret;
}


static int
zccmd_bg(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    char **attrs;
    int ret = 0;
    chtype ch = 0;

    if (!args[0])
	return 1;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    for(attrs = args+1; *attrs; attrs++) {
	if (strchr(*attrs, '/')) {
	    Colorpairnode cpn;
	    if ((cpn = zcurses_colorget(nam, *attrs)) == NULL)
		ret = 1;
	    else if (cpn->colorpair >= 256) {
		/* pretty unlikely, but... */
		zwarnnam(nam, "bg color pair %s has index (%d) too large (max 255)",
			 cpn->node.nam, cpn->colorpair);
		ret = 1;
	    } else {
		ch |= COLOR_PAIR(cpn->colorpair);
	    }
	} else if (**attrs == '@') {
	    ch |= (*attrs)[1] == Meta ? (*attrs)[2] ^ 32 : (*attrs)[1];
	} else {
	    char *ptr;
	    int onoff;
	    struct zcurses_namenumberpair *zca;

	    switch(*attrs[0]) {
	    case '-':
		onoff = ZCURSES_ATTROFF;
		ptr = (*attrs) + 1;
		break;
	    case '+':
		onoff = ZCURSES_ATTRON;
		ptr = (*attrs) + 1;
		break;
	    default:
		onoff = ZCURSES_ATTRON;
		ptr = *attrs;
		break;
	    }
	    if ((zca = zcurses_attrget(w->win, ptr)) == NULL) {
		zwarnnam(nam, "attribute `%s' not known", ptr);
		ret = 1;
	    } else {
		switch(onoff) {
		    case ZCURSES_ATTRON:
			if (wattron(w->win, zca->number) == ERR)
			    ret = 1;
			break;
		    case ZCURSES_ATTROFF:
			if (wattroff(w->win, zca->number) == ERR)
			    ret = 1;
			break;
		}
	    }
	}
    }

    if (ret == 0)
	return wbkgd(w->win, ch) != OK;
    return ret;
}


static int
zccmd_scroll(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    int ret = 0;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    if (!strcmp(args[1], "on")) {
	if (scrollok(w->win, TRUE) == ERR)
	    return 1;
	w->flags |= ZCWF_SCROLL;
    } else if (!strcmp(args[1], "off")) {
	if (scrollok(w->win, FALSE) == ERR)
	    return 1;
	w->flags &= ~ZCWF_SCROLL;
    } else {
	char *endptr;
	zlong sl = zstrtol(args[1], &endptr, 10);
	if (*endptr) {
	    zwarnnam(nam, "scroll requires `on', `off' or integer: %s",
		     args[1]);
	    return 1;
	}
	if (!(w->flags & ZCWF_SCROLL))
	    scrollok(w->win, TRUE);
	if (wscrl(w->win, (int)sl) == ERR)
	    ret = 1;
	if (!(w->flags & ZCWF_SCROLL))
	    scrollok(w->win, FALSE);
    }

    return ret;
}


static int
zccmd_input(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    char *var;
    int keypadnum = -1;
    int nargs = arrlen(args);
#ifdef HAVE_WGET_WCH
    int ret;
    wint_t wi;
    VARARR(char, instr, 2*MB_CUR_MAX+1);
#else
    int ci;
    char instr[3];
#endif

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    if (nargs >= 3) {
	keypad(w->win, TRUE);
    } else {
	keypad(w->win, FALSE);
    }

    if (nargs >= 4) {
#ifdef NCURSES_MOUSE_VERSION
	if (!(zcurses_flags & ZCF_MOUSE_ACTIVE) ||
	    (zcurses_flags & ZCF_MOUSE_MASK_CHANGED)) {
	    if (mousemask(zcurses_mouse_mask, NULL) == (mmask_t)ERR) {
		zwarnnam(nam, "current mouse mode is not supported");
		return 1;
	    }
	    zcurses_flags = (zcurses_flags & ~ZCF_MOUSE_MASK_CHANGED) |
		ZCF_MOUSE_ACTIVE;
	}
#else
	zwarnnam(nam, "mouse events are not supported");
	return 1;
#endif
    }
#ifdef NCURSES_MOUSE_VERSION
    else {
	if (zcurses_flags & ZCF_MOUSE_ACTIVE) {
	    mousemask((mmask_t)0, NULL);
	    zcurses_flags &= ~ZCF_MOUSE_ACTIVE;
	}
    }
#endif

    /*
     * Some documentation for wgetch() says:

       The behavior of getch and friends in the presence of  handled  signals
       is  unspecified  in the SVr4 and XSI Curses documentation.  Under his-
       torical curses implementations, it varied  depending  on  whether  the
       operating system's implementation of handled signal receipt interrupts
       a read(2) call in progress or not, and also (in some  implementations)
       depending  on  whether  an input timeout or non-blocking mode has been
       set.

       Programmers concerned about portability should be prepared for  either
       of  two cases: (a) signal receipt does not interrupt getch; (b) signal
       receipt interrupts getch and causes it to return ERR with errno set to
       EINTR.  Under the ncurses implementation, handled signals never inter-
       rupt getch.

     * The observed behavior, however, is different:  wgetch() consistently
     * returns ERR with EINTR when a signal is handled by the shell "trap"
     * command mechanism.  Further, it consistently returns ERR twice, the
     * second time without even attempting to repeat the interrupted read,
     * which has the side-effect of NOT updating errno.  A third call will
     * then begin reading again.
     *
     * Therefore, to properly implement signal trapping, we must (1) call
     * wgetch() in a loop as long as errno remains EINTR, and (2) clear
     * errno only before beginning the loop, not on every pass.
     *
     * There remains a potential bug here in that, if the caller has set
     * a timeout for the read [see zccmd_timeout()] the countdown is very
     * likely restarted on every call to wgetch(), so an interrupted call
     * might wait much longer than desired.
     */
    errno = 0;

#ifdef HAVE_WGET_WCH
    while ((ret = wget_wch(w->win, &wi)) == ERR) {
	if (errno != EINTR || errflag || retflag || breaks || exit_pending)
	    break;
    }
    switch (ret) {
    case OK:
	ret = wctomb(instr, (wchar_t)wi);
	if (ret == 0) {
	    instr[0] = Meta;
	    instr[1] = '\0' ^ 32;
	    instr[2] = '\0';
	} else {
	    (void)metafy(instr, ret, META_NOALLOC);
	}
	break;

    case KEY_CODE_YES:
	*instr = '\0';
	keypadnum = (int)wi;
	break;

    case ERR:
    default:
	return 1;
    }
#else
    while ((ci = wgetch(w->win)) == ERR) {
	if (errno != EINTR || errflag || retflag || breaks || exit_pending)
	    return 1;
    }
    if (ci >= 256) {
	keypadnum = ci;
	*instr = '\0';
    } else {
	if (imeta(ci)) {
	    instr[0] = Meta;
	    instr[1] = (char)ci ^ 32;
	    instr[2] = '\0';
	} else {
	    instr[0] = (char)ci;
	    instr[1] = '\0';
	}
    }
#endif
    if (args[1])
	var = args[1];
    else
	var = "REPLY";
    if (!setsparam(var, ztrdup(instr)))
	return 1;
    if (nargs >= 3) {
	if (keypadnum > 0) {
#ifdef NCURSES_MOUSE_VERSION
	    if (nargs >= 4 && keypadnum == KEY_MOUSE) {
		MEVENT mevent;
		char digits[DIGBUFSIZE];
		LinkList margs;
		const struct zcurses_mouse_event *zcmmp = zcurses_mouse_map;

		if (!setsparam(args[2], ztrdup("MOUSE")))
		    return 1;
		if (getmouse(&mevent) == ERR) {
		    /*
		     * This may happen if the mouse wasn't in
		     * the window, so set the array to empty
		     * but return success unless the set itself
		     * failed.
		     */
		    return !setaparam(args[3], mkarray(NULL));
		}
		margs = newlinklist();
		sprintf(digits, "%d", (int)mevent.id);
		addlinknode(margs, dupstring(digits));
		sprintf(digits, "%d", mevent.x);
		addlinknode(margs, dupstring(digits));
		sprintf(digits, "%d", mevent.y);
		addlinknode(margs, dupstring(digits));
		sprintf(digits, "%d", mevent.z);
		addlinknode(margs, dupstring(digits));

		/*
		 * We only expect one event, but it doesn't hurt
		 * to keep testing.
		 */
		for (; zcmmp->button; zcmmp++) {
		    if (mevent.bstate & zcmmp->event) {
			const struct zcurses_namenumberpair *zcmelp =
			    zcurses_mouse_event_list;
			for (; zcmelp->name; zcmelp++) {
			    if (zcmelp->number == zcmmp->what) {
				char *evstr = zhalloc(strlen(zcmelp->name)+2);
				sprintf(evstr, "%s%d", zcmelp->name,
					zcmmp->button);
				addlinknode(margs, evstr);

				break;
			    }
			}
		    }
		}
		if (mevent.bstate & BUTTON_SHIFT)
		    addlinknode(margs, "SHIFT");
		if (mevent.bstate & BUTTON_CTRL)
		    addlinknode(margs, "CTRL");
		if (mevent.bstate & BUTTON_ALT)
		    addlinknode(margs, "ALT");
		if (!setaparam(args[3], zlinklist2array(margs)))
		    return 1;
	    } else {
#endif
		const struct zcurses_namenumberpair *nnptr;
		char fbuf[DIGBUFSIZE+1];

		for (nnptr = keypad_names; nnptr->name; nnptr++) {
		    if (keypadnum == nnptr->number) {
			if (!setsparam(args[2], ztrdup(nnptr->name)))
			    return 1;
			return 0;
		    }
		}
		if (keypadnum > KEY_F0) {
		    /* assume it's a function key */
		    sprintf(fbuf, "F%d", keypadnum - KEY_F0);
		} else {
		    /* print raw number */
		    sprintf(fbuf, "%d", keypadnum);
		}
		if (!setsparam(args[2], ztrdup(fbuf)))
		    return 1;
#ifdef NCURSES_MOUSE_VERSION
	    }
#endif
	} else {
	    if (!setsparam(args[2], ztrdup("")))
		return 1;
	}
    }
#ifdef NCURSES_MOUSE_VERSION
    if (keypadnum != KEY_MOUSE && nargs >= 4)
	return !setaparam(args[3], mkarray(NULL));
#endif
    return 0;
}


static int
zccmd_timeout(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    int to;
    char *eptr;

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    to = (int)zstrtol(args[1], &eptr, 10);
    if (*eptr) {
	zwarnnam(nam, "timeout requires an integer: %s", args[1]);
	return 1;
    }

#if defined(__sun__) && defined(__SVR4) && !defined(HAVE_USE_DEFAULT_COLORS)
    /*
     * On Solaris turning a timeout off seems to be problematic.
     * The following fixes it.  We test for Solaris without ncurses
     * (the last test) to be specific; this may turn up in other older
     * versions of curses, but it's difficult to test for.
     */
    if (to < 0) {
	nocbreak();
	cbreak();
    }
#endif
    wtimeout(w->win, to);
    return 0;
}


static int
zccmd_mouse(const char *nam, char **args)
{
#ifdef NCURSES_MOUSE_VERSION
    int ret = 0;

    for (; *args; args++) {
	if (!strcmp(*args, "delay")) {
	    char *eptr;
	    zlong delay;

	    if (!*++args ||
		((delay = zstrtol(*args, &eptr, 10)), eptr != NULL)) {
		zwarnnam(nam, "mouse delay requires an integer argument");
		return 1;
	    }
	    if (mouseinterval((int)delay) != OK)
		ret = 1;
	} else {
	    char *arg = *args;
	    int onoff = 1;
	    if (*arg == '+')
		arg++;
	    else if (*arg == '-') {
		arg++;
		onoff = 0;
	    }
	    if (!strcmp(arg, "motion")) {
		mmask_t old_mask = zcurses_mouse_mask;
		if (onoff)
		    zcurses_mouse_mask |= REPORT_MOUSE_POSITION;
		else
		    zcurses_mouse_mask &= ~REPORT_MOUSE_POSITION;
		if (old_mask != zcurses_mouse_mask)
		    zcurses_flags |= ZCF_MOUSE_MASK_CHANGED;
	    } else {
		zwarnnam(nam, "unrecognised mouse command: %s", *arg);
		return 1;
	    }
	}
    }

    return ret;
#else
    return 1;
#endif
}


static int
zccmd_position(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    int i, intarr[6];
    char **array, dbuf[DIGBUFSIZE];

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

    /* Look no pointers:  these are macros. */
    getyx(w->win, intarr[0], intarr[1]);
    if (intarr[0] == -1)
	return 1;
    getbegyx(w->win, intarr[2], intarr[3]);
    if (intarr[2] == -1)
	return 1;
    getmaxyx(w->win, intarr[4], intarr[5]);
    if (intarr[4] == -1)
	return 1;

    array = (char **)zalloc(7*sizeof(char *));
    for (i = 0; i < 6; i++) {
	sprintf(dbuf, "%d", intarr[i]);
	array[i] = ztrdup(dbuf);
    }
    array[6] = NULL;

    setaparam(args[1], array);
    return 0;
}


static int
zccmd_querychar(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    short cp;
    Colorpairnode cpn;
    const struct zcurses_namenumberpair *zattrp;
    LinkList clist;
#if defined(HAVE_WIN_WCH) && defined(HAVE_GETCCHAR)
    attr_t attrs;
    wchar_t c;
    cchar_t cc;
    int count;
    VARARR(char, instr, 2*MB_CUR_MAX+1);
#else
    chtype inc, attrs;
    char instr[3];
#endif

    node = zcurses_validate_window(args[0], ZCURSES_USED);
    if (node == NULL) {
	zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	return 1;
    }

    w = (ZCWin)getdata(node);

#if defined(HAVE_WIN_WCH) && defined(HAVE_GETCCHAR)
    if (win_wch(w->win, &cc) == ERR)
	return 1;

    if (getcchar(&cc, &c, &attrs, &cp, NULL) == ERR)
	return 1;
    /* Hmmm... I always get 0 for cp, whereas the following works... */
    cp = PAIR_NUMBER(winch(w->win));

    count = wctomb(instr, c);
    if (count == -1)
	return 1;
    (void)metafy(instr, count, META_NOALLOC);
#else
    inc = winch(w->win);
    /* I think the following is correct, the manual is a little terse */
    cp = PAIR_NUMBER(inc);
    inc &= A_CHARTEXT;
    if (imeta(inc)) {
	instr[0] = Meta;
	instr[1] = STOUC(inc ^ 32);
	instr[2] = '\0';
    } else {
	instr[0] = STOUC(inc);
	instr[1] = '\0';
    }
    attrs = inc;
#endif

    /*
     * Attribute numbers vary, so make a linked list.
     * This also saves us from doing the permanent allocation till
     * the end.
     */
    clist = newlinklist();
    /* First the (possibly multibyte) character itself. */
    addlinknode(clist, instr);
    /*
     * Next the colo[u]r.
     * We should be able to match it in the colorpair list, but
     * if some reason we can't, fail safe and output the number.
     */
    cpn = zcurses_colorget_reverse(cp);
    if (cpn) {
	addlinknode(clist, cpn->node.nam);
    } else {
	/* report color pair number */
	char digits[DIGBUFSIZE];
	sprintf(digits, "%d", (int)cp);
	addlinknode(clist, digits);
    }
    /* Now see what attributes are present. */
    for (zattrp = zcurses_attributes; zattrp->name; zattrp++) {
	if (attrs & zattrp->number)
	    addlinknode(clist, zattrp->name);
    }

    /* Turn this into an array and store it. */
    return !setaparam(args[1] ? args[1] : "reply", zlinklist2array(clist));
}


static int
zccmd_touch(const char *nam, char **args)
{
    LinkNode node;
    ZCWin w;
    int ret = 0;

    for (; *args; args++) {
	node = zcurses_validate_window(args[0], ZCURSES_USED);
	if (node == NULL) {
	    zwarnnam(nam, "%s: %s", zcurses_strerror(zc_errno), args[0]);
	    return 1;
	}

	w = (ZCWin)getdata(node);
	if (touchwin(w->win) != OK)
	    ret = 1;
    }

    return ret;
}


/*********************
  Main builtin handler
 *********************/

/**/
static int
bin_zcurses(char *nam, char **args, Options ops, UNUSED(int func))
{
    char **saargs;
    struct zcurses_subcommand *zcsc;
    int num_args;

    struct zcurses_subcommand scs[] = {
	{"init", zccmd_init, 0, 0},
	{"addwin", zccmd_addwin, 5, 6},
	{"delwin", zccmd_delwin, 1, 1},
	{"refresh", zccmd_refresh, 0, -1},
	{"move", zccmd_move, 3, 3},
	{"clear", zccmd_clear, 1, 2},
	{"position", zccmd_position, 2, 2},
	{"char", zccmd_char, 2, 2},
	{"string", zccmd_string, 2, 2},
	{"border", zccmd_border, 1, 1},
	{"end", zccmd_endwin, 0, 0},
	{"attr", zccmd_attr, 2, -1},
	{"bg", zccmd_bg, 2, -1},
	{"scroll", zccmd_scroll, 2, 2},
	{"input", zccmd_input, 1, 4},
	{"timeout", zccmd_timeout, 2, 2},
	{"mouse", zccmd_mouse, 0, -1},
	{"querychar", zccmd_querychar, 1, 2},
	{"touch", zccmd_touch, 1, -1},
	{NULL, (zccmd_t)0, 0, 0}
    };

    for(zcsc = scs; zcsc->name; zcsc++) {
	if(!strcmp(args[0], zcsc->name))
	    break;
    }

    if (zcsc->name == NULL) {
	zwarnnam(nam, "unknown subcommand: %s", args[0]);
	return 1;
    }

    saargs = args;
    while (*saargs++);
    num_args = saargs - (args + 2);

    if (num_args < zcsc->minargs) {
	zwarnnam(nam, "too few arguments for subcommand: %s", args[0]);
	return 1;
    } else if (zcsc->maxargs >= 0 && num_args > zcsc->maxargs) {
	zwarnnam(nam, "too many arguments for subcommand: %s", args[0]);
	return 1;
    }

    if (zcsc->cmd != zccmd_init && zcsc->cmd != zccmd_endwin &&
	!zcurses_getwindowbyname("stdscr")) {
	zwarnnam(nam, "command `%s' can't be used before `zcurses init'",
		 zcsc->name);
	return 1;
    }

    return zcsc->cmd(nam, args+1);
}


static struct builtin bintab[] = {
    BUILTIN("zcurses", 0, bin_zcurses, 1, -1, 0, "", NULL),
};


/*******************
 * Special variables
 *******************/

static char **
zcurses_colorsarrgetfn(UNUSED(Param pm))
{
    return zcurses_pairs_to_array(zcurses_colors);
}

static const struct gsu_array zcurses_colorsarr_gsu =
{ zcurses_colorsarrgetfn, arrsetfn, stdunsetfn };


static char **
zcurses_attrgetfn(UNUSED(Param pm))
{
    return zcurses_pairs_to_array(zcurses_attributes);
}

static const struct gsu_array zcurses_attrs_gsu =
{ zcurses_attrgetfn, arrsetfn, stdunsetfn };


static char **
zcurses_keycodesgetfn(UNUSED(Param pm))
{
    return zcurses_pairs_to_array(keypad_names);
}

static const struct gsu_array zcurses_keycodes_gsu =
{ zcurses_keycodesgetfn, arrsetfn, stdunsetfn };


static char **
zcurses_windowsgetfn(UNUSED(Param pm))
{
    LinkNode node;
    char **arr, **arrptr;
    int count = countlinknodes(zcurses_windows);

    arrptr = arr = (char **)zhalloc((count+1) * sizeof(char *));

    for (node = firstnode(zcurses_windows); node; incnode(node))
	*arrptr++ = dupstring(((ZCWin)getdata(node))->name);
    *arrptr = NULL;

    return arr;
}

static const struct gsu_array zcurses_windows_gsu =
{ zcurses_windowsgetfn, arrsetfn, stdunsetfn };


static zlong
zcurses_colorsintgetfn(UNUSED(Param pm))
{
    return COLORS;
}

static const struct gsu_integer zcurses_colorsint_gsu =
{ zcurses_colorsintgetfn, nullintsetfn, stdunsetfn };


static zlong
zcurses_colorpairsintgetfn(UNUSED(Param pm))
{
    return COLOR_PAIRS;
}

static const struct gsu_integer zcurses_colorpairsint_gsu =
{ zcurses_colorpairsintgetfn, nullintsetfn, stdunsetfn };


static struct paramdef partab[] = {
    SPECIALPMDEF("zcurses_colors", PM_ARRAY|PM_READONLY,
		 &zcurses_colorsarr_gsu, NULL, NULL),
    SPECIALPMDEF("zcurses_attrs", PM_ARRAY|PM_READONLY,
		 &zcurses_attrs_gsu, NULL, NULL),
    SPECIALPMDEF("zcurses_keycodes", PM_ARRAY|PM_READONLY,
		 &zcurses_keycodes_gsu, NULL, NULL),
    SPECIALPMDEF("zcurses_windows", PM_ARRAY|PM_READONLY,
		 &zcurses_windows_gsu, NULL, NULL),
    SPECIALPMDEF("ZCURSES_COLORS", PM_INTEGER|PM_READONLY,
		 &zcurses_colorsint_gsu, NULL, NULL),
    SPECIALPMDEF("ZCURSES_COLOR_PAIRS", PM_INTEGER|PM_READONLY,
		 &zcurses_colorpairsint_gsu, NULL, NULL)
};

/***************************
 * Standard module interface
 ***************************/


/*
 * boot_ is executed when the module is loaded.
 */

static struct features module_features = {
    bintab, sizeof(bintab)/sizeof(*bintab),
    NULL, 0,
    NULL, 0,
    partab, sizeof(partab)/sizeof(*partab),
    0
};

/**/
int
setup_(UNUSED(Module m))
{
    return 0;
}

/**/
int
features_(Module m, char ***features)
{
    *features = featuresarray(m, &module_features);
    return 0;
}

/**/
int
enables_(Module m, int **enables)
{
    return handlefeatures(m, &module_features, enables);
}

/**/
int
boot_(Module m)
{
    zcurses_windows = znewlinklist();

    return 0;
}

/**/
int
cleanup_(Module m)
{
    freelinklist(zcurses_windows, (FreeFunc) zcurses_free_window);
    if (zcurses_colorpairs)
	deletehashtable(zcurses_colorpairs);
    return setfeatureenables(m, &module_features, NULL);
}

/**/
int
finish_(UNUSED(Module m))
{
    return 0;
}

[-- Attachment #3: hashtable.c --]
[-- Type: text/x-csrc, Size: 34713 bytes --]

/*
 * hashtable.c - hash tables
 *
 * This file is part of zsh, the Z shell.
 *
 * Copyright (c) 1992-1997 Paul Falstad
 * All rights reserved.
 *
 * Permission is hereby granted, without written agreement and without
 * license or royalty fees, to use, copy, modify, and distribute this
 * software and to distribute modified versions of this software for any
 * purpose, provided that the above copyright notice and the following
 * two paragraphs appear in all copies of this software.
 *
 * In no event shall Paul Falstad or the Zsh Development Group be liable
 * to any party for direct, indirect, special, incidental, or consequential
 * damages arising out of the use of this software and its documentation,
 * even if Paul Falstad and the Zsh Development Group have been advised of
 * the possibility of such damage.
 *
 * Paul Falstad and the Zsh Development Group specifically disclaim any
 * warranties, including, but not limited to, the implied warranties of
 * merchantability and fitness for a particular purpose.  The software
 * provided hereunder is on an "as is" basis, and Paul Falstad and the
 * Zsh Development Group have no obligation to provide maintenance,
 * support, updates, enhancements, or modifications.
 *
 */

#include "../config.h"

#ifdef ZSH_HASH_DEBUG
# define HASHTABLE_DEBUG_MEMBERS \
    /* Members of struct hashtable used for debugging hash tables */ \
    HashTable next, last;	/* linked list of all hash tables           */ \
    char *tablename;		/* string containing name of the hash table */ \
    PrintTableStats printinfo;	/* pointer to function to print table stats */
#else /* !ZSH_HASH_DEBUG */
# define HASHTABLE_DEBUG_MEMBERS
#endif /* !ZSH_HASH_DEBUG */

#define HASHTABLE_INTERNAL_MEMBERS \
    ScanStatus scan;		/* status of a scan over this hashtable     */ \
    HASHTABLE_DEBUG_MEMBERS

typedef struct scanstatus *ScanStatus;

#include "zsh.mdh"
#include "hashtable.pro"

/* Structure for recording status of a hashtable scan in progress.  When a *
 * scan starts, the .scan member of the hashtable structure points to one  *
 * of these.  That member being non-NULL disables resizing of the          *
 * hashtable (when adding elements).  When elements are deleted, the       *
 * contents of this structure is used to make sure the scan won't stumble  *
 * into the deleted element.                                               */

struct scanstatus {
    int sorted;
    union {
	struct {
	    HashNode *hashtab;
	    int ct;
	} s;
	HashNode u;
    } u;
};

/********************************/
/* Generic Hash Table functions */
/********************************/

#ifdef ZSH_HASH_DEBUG
static HashTable firstht, lastht;
#endif /* ZSH_HASH_DEBUG */

/* Generic hash function */

/**/
mod_export unsigned
hasher(const char *str)
{
    unsigned hashval = 0, c;

    while ((c = *((unsigned char *) str++)))
	hashval += (hashval << 5) + c;

    return hashval;
}

/* Get a new hash table */

/**/
mod_export HashTable
newhashtable(int size, UNUSED(char const *name), UNUSED(PrintTableStats printinfo))
{
    HashTable ht;

    ht = (HashTable) zshcalloc(sizeof *ht);
#ifdef ZSH_HASH_DEBUG
    ht->next = NULL;
    if(!firstht)
	firstht = ht;
    ht->last = lastht;
    if(lastht)
	lastht->next = ht;
    lastht = ht;
    ht->printinfo = printinfo ? printinfo : printhashtabinfo;
    ht->tablename = ztrdup(name);
#endif /* ZSH_HASH_DEBUG */
    ht->nodes = (HashNode *) zshcalloc(size * sizeof(HashNode));
    ht->hsize = size;
    ht->ct = 0;
    ht->scan = NULL;
    ht->scantab = NULL;
    return ht;
}

/* Delete a hash table.  After this function has been used, any *
 * existing pointers to the hash table are invalid.             */

/**/
mod_export void
deletehashtable(HashTable ht)
{
    ht->emptytable(ht);
#ifdef ZSH_HASH_DEBUG
    if(ht->next)
	ht->next->last = ht->last;
    else
	lastht = ht->last;
    if(ht->last)
	ht->last->next = ht->next;
    else
	firstht = ht->next;
    zsfree(ht->tablename);
#endif /* ZSH_HASH_DEBUG */
    zfree(ht->nodes, ht->hsize * sizeof(HashNode));
    zfree(ht, sizeof(*ht));
}

/* Add a node to a hash table.                          *
 * nam is the key to use in hashing.  nodeptr points    *
 * to the node to add.  If there is already a node in   *
 * the table with the same key, it is first freed, and  *
 * then the new node is added.  If the number of nodes  *
 * is now greater than twice the number of hash values, *
 * the table is then expanded.                          */

/**/
mod_export void
addhashnode(HashTable ht, char *nam, void *nodeptr)
{
    HashNode oldnode = addhashnode2(ht, nam, nodeptr);
    if (oldnode)
	ht->freenode(oldnode);
}

/* Add a node to a hash table, returning the old node on replacement. */

/**/
HashNode
addhashnode2(HashTable ht, char *nam, void *nodeptr)
{
    unsigned hashval;
    HashNode hn, hp, hq;

    hn = (HashNode) nodeptr;
    hn->nam = nam;

    hashval = ht->hash(hn->nam) % ht->hsize;
    hp = ht->nodes[hashval];

    /* check if this is the first node for this hash value */
    if (!hp) {
	hn->next = NULL;
	ht->nodes[hashval] = hn;
	if (++ht->ct >= ht->hsize * 2 && !ht->scan)
	    expandhashtable(ht);
	return NULL;
    }

    /* else check if the first node contains the same key */
    if (ht->cmpnodes(hp->nam, hn->nam) == 0) {
	ht->nodes[hashval] = hn;
	replacing:
	hn->next = hp->next;
	if(ht->scan) {
	    if(ht->scan->sorted) {
		HashNode *hashtab = ht->scan->u.s.hashtab;
		int i;
		for(i = ht->scan->u.s.ct; i--; )
		    if(hashtab[i] == hp)
			hashtab[i] = hn;
	    } else if(ht->scan->u.u == hp)
		ht->scan->u.u = hn;
	}
	return hp;
    }

    /* else run through the list and check all the keys */
    hq = hp;
    hp = hp->next;
    for (; hp; hq = hp, hp = hp->next) {
	if (ht->cmpnodes(hp->nam, hn->nam) == 0) {
	    hq->next = hn;
	    goto replacing;
	}
    }

    /* else just add it at the front of the list */
    hn->next = ht->nodes[hashval];
    ht->nodes[hashval] = hn;
    if (++ht->ct >= ht->hsize * 2 && !ht->scan)
        expandhashtable(ht);
    return NULL;
}

/* Get an enabled entry in a hash table.  *
 * If successful, it returns a pointer to *
 * the hashnode.  If the node is DISABLED *
 * or isn't found, it returns NULL        */

/**/
mod_export HashNode
gethashnode(HashTable ht, const char *nam)
{
    unsigned hashval;
    HashNode hp;

    hashval = ht->hash(nam) % ht->hsize;

    if( (int)ht->printnode == -1 ) {
        zwarnnam("==", "nam:%s, hashval:%d", nam, hashval);
    }
    for (hp = ht->nodes[hashval]; hp; hp = hp->next) {
	if (ht->cmpnodes(hp->nam, nam) == 0) {
	    if (hp->flags & DISABLED) {
                if( (int)ht->printnode == -1 ) {
                    zwarnnam("==", "nam:%s, hashval:%d DISABLED return", nam, hashval);
                }
		return NULL;
	    } else {
                if( (int)ht->printnode == -1 ) {
                    zwarnnam("==", "nam:%s, hashval:%d RETURN", nam, hashval);
                }
		return hp;
            }
	}
    }
    return NULL;
}

/* Get an entry in a hash table.  It will *
 * ignore the DISABLED flag and return a  *
 * pointer to the hashnode if found, else *
 * it returns NULL.                       */

/**/
mod_export HashNode
gethashnode2(HashTable ht, const char *nam)
{
    unsigned hashval;
    HashNode hp;

    hashval = ht->hash(nam) % ht->hsize;
    for (hp = ht->nodes[hashval]; hp; hp = hp->next) {
	if (ht->cmpnodes(hp->nam, nam) == 0)
	    return hp;
    }
    return NULL;
}

/* Remove an entry from a hash table.           *
 * If successful, it removes the node from the  *
 * table and returns a pointer to it.  If there *
 * is no such node, then it returns NULL        */

/**/
mod_export HashNode
removehashnode(HashTable ht, const char *nam)
{
    unsigned hashval;
    HashNode hp, hq;

    hashval = ht->hash(nam) % ht->hsize;
    hp = ht->nodes[hashval];

    /* if no nodes at this hash value, return NULL */
    if (!hp)
	return NULL;

    /* else check if the key in the first one matches */
    if (ht->cmpnodes(hp->nam, nam) == 0) {
	ht->nodes[hashval] = hp->next;
	gotit:
	ht->ct--;
	if(ht->scan) {
	    if(ht->scan->sorted) {
		HashNode *hashtab = ht->scan->u.s.hashtab;
		int i;
		for(i = ht->scan->u.s.ct; i--; )
		    if(hashtab[i] == hp)
			hashtab[i] = NULL;
	    } else if(ht->scan->u.u == hp)
		ht->scan->u.u = hp->next;
	}
	return hp;
    }

    /* else run through the list and check the rest of the keys */
    hq = hp;
    hp = hp->next;
    for (; hp; hq = hp, hp = hp->next) {
	if (ht->cmpnodes(hp->nam, nam) == 0) {
	    hq->next = hp->next;
	    goto gotit;
	}
    }

    /* else it is not in the list, so return NULL */
    return NULL;
}

/* Disable a node in a hash table */

/**/
void
disablehashnode(HashNode hn, UNUSED(int flags))
{
    hn->flags |= DISABLED;
}

/* Enable a node in a hash table */

/**/
void
enablehashnode(HashNode hn, UNUSED(int flags))
{
    hn->flags &= ~DISABLED;
}

/* Compare two hash table entries by name */

/**/
static int
hnamcmp(const void *ap, const void *bp)
{
    HashNode a = *(HashNode *)ap;
    HashNode b = *(HashNode *)bp;
    return ztrcmp(a->nam, b->nam);
}

/* Scan the nodes in a hash table and execute scanfunc on nodes based on
 * the flags that are set/unset.  scanflags is passed unchanged to
 * scanfunc (if executed).
 *
 * If sorted != 0, then sort entries of hash table before scanning.
 * If flags1 > 0, then execute scanfunc on a node only if at least one of
 *                these flags is set.
 * If flags2 > 0, then execute scanfunc on a node only if all of
 *                these flags are NOT set.
 * The conditions above for flags1/flags2 must both be true.
 *
 * It is safe to add, remove or replace hash table elements from within
 * the scanfunc.  Replaced elements will appear in the scan exactly once,
 * the new version if it was not scanned before the replacement was made.
 * Added elements might or might not appear in the scan.
 *
 * pprog, if non-NULL, is a pattern that must match the name
 * of the node.
 *
 * The function returns the number of matches, as reduced by pprog, flags1
 * and flags2.
 */

/**/
mod_export int
scanmatchtable(HashTable ht, Patprog pprog, int sorted,
	       int flags1, int flags2, ScanFunc scanfunc, int scanflags)
{
    int match = 0;
    struct scanstatus st;

    /*
     * scantab is currently only used by modules to scan
     * tables where the contents are generated on the fly from
     * other objects.  Note the fact that in this case pprog,
     * sorted, flags1 and flags2 are ignore.
     */
    if (!pprog && ht->scantab) {
	ht->scantab(ht, scanfunc, scanflags);
	return ht->ct;
    }
    if (sorted) {
	int i, ct = ht->ct;
	VARARR(HashNode, hnsorttab, ct);
	HashNode *htp, hn;

	/*
	 * Because the structure might change under our feet,
	 * we can't apply the flags and the pattern before sorting,
	 * tempting though that is.
	 */
	for (htp = hnsorttab, i = 0; i < ht->hsize; i++)
	    for (hn = ht->nodes[i]; hn; hn = hn->next)
		*htp++ = hn;
	qsort((void *)hnsorttab, ct, sizeof(HashNode), hnamcmp);

	st.sorted = 1;
	st.u.s.hashtab = hnsorttab;
	st.u.s.ct = ct;
	ht->scan = &st;

	for (htp = hnsorttab, i = 0; i < ct; i++, htp++) {
	    if ((!flags1 || ((*htp)->flags & flags1)) &&
		!((*htp)->flags & flags2) &&
		(!pprog || pattry(pprog, (*htp)->nam))) {
		match++;
		scanfunc(*htp, scanflags);
	    }
	}

	ht->scan = NULL;
    } else {
	int i, hsize = ht->hsize;
	HashNode *nodes = ht->nodes;

	st.sorted = 0;
	ht->scan = &st;

	for (i = 0; i < hsize; i++)
	    for (st.u.u = nodes[i]; st.u.u; ) {
		HashNode hn = st.u.u;
		st.u.u = st.u.u->next;
		if ((!flags1 || (hn->flags & flags1)) && !(hn->flags & flags2)
		    && (!pprog || pattry(pprog, hn->nam))) {
		    match++;
		    scanfunc(hn, scanflags);
		}
	    }

	ht->scan = NULL;
    }

    return match;
}


/**/
mod_export int
scanhashtable(HashTable ht, int sorted, int flags1, int flags2,
	      ScanFunc scanfunc, int scanflags)
{
    return scanmatchtable(ht, NULL, sorted, flags1, flags2,
			  scanfunc, scanflags);
}

/* Expand hash tables when they get too many entries. *
 * The new size is 4 times the previous size.         */

/**/
static void
expandhashtable(HashTable ht)
{
    struct hashnode **onodes, **ha, *hn, *hp;
    int i, osize;

    osize = ht->hsize;
    onodes = ht->nodes;

    ht->hsize = osize * 4;
    ht->nodes = (HashNode *) zshcalloc(ht->hsize * sizeof(HashNode));
    ht->ct = 0;

    /* scan through the old list of nodes, and *
     * rehash them into the new list of nodes  */
    for (i = 0, ha = onodes; i < osize; i++, ha++) {
	for (hn = *ha; hn;) {
	    hp = hn->next;
	    ht->addnode(ht, hn->nam, hn);
	    hn = hp;
	}
    }
    zfree(onodes, osize * sizeof(HashNode));
}

/* Empty the hash table and resize it if necessary */

/**/
static void
resizehashtable(HashTable ht, int newsize)
{
    struct hashnode **ha, *hn, *hp;
    int i;

    /* free all the hash nodes */
    ha = ht->nodes;
    for (i = 0; i < ht->hsize; i++, ha++) {
	for (hn = *ha; hn;) {
	    hp = hn->next;
	    ht->freenode(hn);
	    hn = hp;
	}
    }

    /* If new size desired is different from current size, *
     * we free it and allocate a new nodes array.          */
    if (ht->hsize != newsize) {
	zfree(ht->nodes, ht->hsize * sizeof(HashNode));
	ht->nodes = (HashNode *) zshcalloc(newsize * sizeof(HashNode));
	ht->hsize = newsize;
    } else {
	/* else we just re-zero the current nodes array */
	memset(ht->nodes, 0, newsize * sizeof(HashNode));
    }

    ht->ct = 0;
}

/* Generic method to empty a hash table */

/**/
mod_export void
emptyhashtable(HashTable ht)
{
    resizehashtable(ht, ht->hsize);
}

/**/
#ifdef ZSH_HASH_DEBUG

/* Print info about hash table */

#define MAXDEPTH 7

/**/
void
printhashtabinfo(HashTable ht)
{
    HashNode hn;
    int chainlen[MAXDEPTH + 1];
    int i, tmpcount, total;

    printf("name of table   : %s\n",   ht->tablename);
    printf("size of nodes[] : %d\n",   ht->hsize);
    printf("number of nodes : %d\n\n", ht->ct);

    memset(chainlen, 0, sizeof(chainlen));

    /* count the number of nodes just to be sure */
    total = 0;
    for (i = 0; i < ht->hsize; i++) {
	tmpcount = 0;
	for (hn = ht->nodes[i]; hn; hn = hn->next)
	    tmpcount++;
	if (tmpcount >= MAXDEPTH)
	    chainlen[MAXDEPTH]++;
	else
	    chainlen[tmpcount]++;
	total += tmpcount;
    }

    for (i = 0; i < MAXDEPTH; i++)
	printf("number of hash values with chain of length %d  : %4d\n", i, chainlen[i]);
    printf("number of hash values with chain of length %d+ : %4d\n", MAXDEPTH, chainlen[MAXDEPTH]);
    printf("total number of nodes                         : %4d\n", total);
}

/**/
int
bin_hashinfo(char *nam, char **args, Options ops, int func)
{
    HashTable ht;

    printf("----------------------------------------------------\n");
    queue_signals();
    for(ht = firstht; ht; ht = ht->next) {
	ht->printinfo(ht);
	printf("----------------------------------------------------\n");
    }
    unqueue_signals();
    return 0;
}

/**/
#endif /* ZSH_HASH_DEBUG */

/********************************/
/* Command Hash Table Functions */
/********************************/

/* hash table containing external commands */
 
/**/
mod_export HashTable cmdnamtab;
 
/* how far we've hashed the PATH so far */
 
/**/
mod_export char **pathchecked;

/* Create a new command hash table */
 
/**/
void
createcmdnamtable(void)
{
    cmdnamtab = newhashtable(201, "cmdnamtab", NULL);

    cmdnamtab->hash        = hasher;
    cmdnamtab->emptytable  = emptycmdnamtable;
    cmdnamtab->filltable   = fillcmdnamtable;
    cmdnamtab->cmpnodes    = strcmp;
    cmdnamtab->addnode     = addhashnode;
    cmdnamtab->getnode     = gethashnode2;
    cmdnamtab->getnode2    = gethashnode2;
    cmdnamtab->removenode  = removehashnode;
    cmdnamtab->disablenode = NULL;
    cmdnamtab->enablenode  = NULL;
    cmdnamtab->freenode    = freecmdnamnode;
    cmdnamtab->printnode   = printcmdnamnode;

    pathchecked = path;
}

/**/
static void
emptycmdnamtable(HashTable ht)
{
    emptyhashtable(ht);
    pathchecked = path;
}

/* Add all commands in a given directory *
 * to the command hashtable.             */

/**/
void
hashdir(char **dirp)
{
    Cmdnam cn;
    DIR *dir;
    char *fn, *unmetadir, *pathbuf, *pathptr;
    int dirlen;
#if defined(_WIN32) || defined(__CYGWIN__)
    char *exe;
#endif /* _WIN32 || _CYGWIN__ */

    if (isrelative(*dirp))
	return;
    unmetadir = unmeta(*dirp);
    if (!(dir = opendir(unmetadir)))
	return;

    dirlen = strlen(unmetadir);
    pathbuf = (char *)zalloc(dirlen + PATH_MAX + 2);
    sprintf(pathbuf, "%s/", unmetadir);
    pathptr = pathbuf + dirlen + 1;

    while ((fn = zreaddir(dir, 1))) {
	if (!cmdnamtab->getnode(cmdnamtab, fn)) {
	    char *fname = ztrdup(fn);
	    struct stat statbuf;
	    int add = 0, dummylen;

	    unmetafy(fn, &dummylen);
	    if (strlen(fn) > PATH_MAX) {
		/* Too heavy to do all the allocation */
		add = 1;
	    } else {
		strcpy(pathptr, fn);
		/*
		 * This is the same test as for the glob qualifier for
		 * executable plain files.
		 */
		if (unset(HASHEXECUTABLESONLY) ||
		    (access(pathbuf, X_OK) == 0 &&
		     stat(pathbuf, &statbuf) == 0 &&
		     S_ISREG(statbuf.st_mode) && (statbuf.st_mode & S_IXUGO)))
		    add = 1;
	    }
	    if (add) {
		cn = (Cmdnam) zshcalloc(sizeof *cn);
		cn->node.flags = 0;
		cn->u.name = dirp;
		cmdnamtab->addnode(cmdnamtab, fname, cn);
	    } else
		zsfree(fname);
	}
#if defined(_WIN32) || defined(__CYGWIN__)
	/* Hash foo.exe as foo, since when no real foo exists, foo.exe
	   will get executed by DOS automatically.  This quiets
	   spurious corrections when CORRECT or CORRECT_ALL is set. */
	if ((exe = strrchr(fn, '.')) &&
	    (exe[1] == 'E' || exe[1] == 'e') &&
	    (exe[2] == 'X' || exe[2] == 'x') &&
	    (exe[3] == 'E' || exe[3] == 'e') && exe[4] == 0) {
	    *exe = 0;
	    if (!cmdnamtab->getnode(cmdnamtab, fn)) {
		cn = (Cmdnam) zshcalloc(sizeof *cn);
		cn->node.flags = 0;
		cn->u.name = dirp;
		cmdnamtab->addnode(cmdnamtab, ztrdup(fn), cn);
	    }
	}
#endif /* _WIN32 || __CYGWIN__ */
    }
    closedir(dir);
    zfree(pathbuf, dirlen + PATH_MAX + 2);
}

/* Go through user's PATH and add everything to *
 * the command hashtable.                       */

/**/
static void
fillcmdnamtable(UNUSED(HashTable ht))
{
    char **pq;
 
    for (pq = pathchecked; *pq; pq++)
	hashdir(pq);

    pathchecked = pq;
}

/**/
static void
freecmdnamnode(HashNode hn)
{
    Cmdnam cn = (Cmdnam) hn;
 
    zsfree(cn->node.nam);
    if (cn->node.flags & HASHED)
	zsfree(cn->u.cmd);
 
    zfree(cn, sizeof(struct cmdnam));
}

/* Print an element of the cmdnamtab hash table (external command) */
 
/**/
static void
printcmdnamnode(HashNode hn, int printflags)
{
    Cmdnam cn = (Cmdnam) hn;

    if (printflags & PRINT_WHENCE_WORD) {
	printf("%s: %s\n", cn->node.nam, (cn->node.flags & HASHED) ? 
	       "hashed" : "command");
	return;
    }

    if ((printflags & PRINT_WHENCE_CSH) || (printflags & PRINT_WHENCE_SIMPLE)) {
	if (cn->node.flags & HASHED) {
	    zputs(cn->u.cmd, stdout);
	    putchar('\n');
	} else {
	    zputs(*(cn->u.name), stdout);
	    putchar('/');
	    zputs(cn->node.nam, stdout);
	    putchar('\n');
	}
	return;
    }

    if (printflags & PRINT_WHENCE_VERBOSE) {
	if (cn->node.flags & HASHED) {
	    nicezputs(cn->node.nam, stdout);
	    printf(" is hashed to ");
	    nicezputs(cn->u.cmd, stdout);
	    putchar('\n');
	} else {
	    nicezputs(cn->node.nam, stdout);
	    printf(" is ");
	    nicezputs(*(cn->u.name), stdout);
	    putchar('/');
	    nicezputs(cn->node.nam, stdout);
	    putchar('\n');
	}
	return;
    }

    if (printflags & PRINT_LIST) {
	printf("hash ");

	if(cn->node.nam[0] == '-')
	    printf("-- ");
    }

    if (cn->node.flags & HASHED) {
	quotedzputs(cn->node.nam, stdout);
	putchar('=');
	quotedzputs(cn->u.cmd, stdout);
	putchar('\n');
    } else {
	quotedzputs(cn->node.nam, stdout);
	putchar('=');
	quotedzputs(*(cn->u.name), stdout);
	putchar('/');
	quotedzputs(cn->node.nam, stdout);
	putchar('\n');
    }
}

/***************************************/
/* Shell Function Hash Table Functions */
/***************************************/

/* hash table containing the shell functions */

/**/
mod_export HashTable shfunctab;

/**/
void
createshfunctable(void)
{
    shfunctab = newhashtable(7, "shfunctab", NULL);

    shfunctab->hash        = hasher;
    shfunctab->emptytable  = NULL;
    shfunctab->filltable   = NULL;
    shfunctab->cmpnodes    = strcmp;
    shfunctab->addnode     = addhashnode;
    shfunctab->getnode     = gethashnode;
    shfunctab->getnode2    = gethashnode2;
    shfunctab->removenode  = removeshfuncnode;
    shfunctab->disablenode = disableshfuncnode;
    shfunctab->enablenode  = enableshfuncnode;
    shfunctab->freenode    = freeshfuncnode;
    shfunctab->printnode   = printshfuncnode;
}

/* Remove an entry from the shell function hash table.   *
 * It checks if the function is a signal trap and if so, *
 * it will disable the trapping of that signal.          */

/**/
static HashNode
removeshfuncnode(UNUSED(HashTable ht), const char *nam)
{
    HashNode hn;
    int signum;

    if (!strncmp(nam, "TRAP", 4) && (signum = getsignum(nam + 4)) != -1)
	hn = removetrap(signum);
    else
	hn = removehashnode(shfunctab, nam);

    return hn;
}

/* Disable an entry in the shell function hash table.    *
 * It checks if the function is a signal trap and if so, *
 * it will disable the trapping of that signal.          */

/**/
static void
disableshfuncnode(HashNode hn, UNUSED(int flags))
{
    hn->flags |= DISABLED;
    if (!strncmp(hn->nam, "TRAP", 4)) {
	int signum = getsignum(hn->nam + 4);
	if (signum != -1) {
	    sigtrapped[signum] &= ~ZSIG_FUNC;
	    unsettrap(signum);
	}
    }
}

/* Re-enable an entry in the shell function hash table.  *
 * It checks if the function is a signal trap and if so, *
 * it will re-enable the trapping of that signal.        */

/**/
static void
enableshfuncnode(HashNode hn, UNUSED(int flags))
{
    Shfunc shf = (Shfunc) hn;

    shf->node.flags &= ~DISABLED;
    if (!strncmp(shf->node.nam, "TRAP", 4)) {
	int signum = getsignum(shf->node.nam + 4);
	if (signum != -1) {
	    settrap(signum, NULL, ZSIG_FUNC);
	}
    }
}

/**/
static void
freeshfuncnode(HashNode hn)
{
    Shfunc shf = (Shfunc) hn;

    zsfree(shf->node.nam);
    if (shf->funcdef)
	freeeprog(shf->funcdef);
    if (shf->redir)
	freeeprog(shf->redir);
    zsfree(shf->filename);
    if (shf->sticky) {
	if (shf->sticky->n_on_opts)
	    zfree(shf->sticky->on_opts,
		  shf->sticky->n_on_opts * sizeof(*shf->sticky->on_opts));
	if (shf->sticky->n_off_opts)
	    zfree(shf->sticky->off_opts,
		  shf->sticky->n_off_opts * sizeof(*shf->sticky->off_opts));
	zfree(shf->sticky, sizeof(*shf->sticky));
    }
    zfree(shf, sizeof(struct shfunc));
}

/* Print a shell function */
 
/**/
static void
printshfuncnode(HashNode hn, int printflags)
{
    Shfunc f = (Shfunc) hn;
    char *t = 0;

    if ((printflags & PRINT_NAMEONLY) ||
	((printflags & PRINT_WHENCE_SIMPLE) &&
	!(printflags & PRINT_WHENCE_FUNCDEF))) {
	zputs(f->node.nam, stdout);
	putchar('\n');
	return;
    }
 
    if ((printflags & (PRINT_WHENCE_VERBOSE|PRINT_WHENCE_WORD)) &&
	!(printflags & PRINT_WHENCE_FUNCDEF)) {
	nicezputs(f->node.nam, stdout);
	printf((printflags & PRINT_WHENCE_WORD) ? ": function" :
	       (f->node.flags & PM_UNDEFINED) ?
	       " is an autoload shell function" :
	       " is a shell function");
	if (f->filename && (printflags & PRINT_WHENCE_VERBOSE) &&
	    strcmp(f->filename, f->node.nam) != 0) {
	    printf(" from ");
	    quotedzputs(f->filename, stdout);
	}
	putchar('\n');
	return;
    }
 
    quotedzputs(f->node.nam, stdout);
    if (f->funcdef || f->node.flags & PM_UNDEFINED) {
	printf(" () {\n");
	zoutputtab(stdout);
	if (f->node.flags & PM_UNDEFINED) {
	    printf("%c undefined\n", hashchar);
	    zoutputtab(stdout);
	} else
	    t = getpermtext(f->funcdef, NULL, 1);
	if (f->node.flags & (PM_TAGGED|PM_TAGGED_LOCAL)) {
	    printf("%c traced\n", hashchar);
	    zoutputtab(stdout);
	}
	if (!t) {
	    char *fopt = "UtTkz";
	    int flgs[] = {
		PM_UNALIASED, PM_TAGGED, PM_TAGGED_LOCAL,
		PM_KSHSTORED, PM_ZSHSTORED, 0
	    };
	    int fl;;

	    zputs("builtin autoload -X", stdout);
	    for (fl=0;fopt[fl];fl++)
		if (f->node.flags & flgs[fl]) putchar(fopt[fl]);
	} else {
	    zputs(t, stdout);
	    zsfree(t);
	    if (f->funcdef->flags & EF_RUN) {
		printf("\n");
		zoutputtab(stdout);
		quotedzputs(f->node.nam, stdout);
		printf(" \"$@\"");
	    }
	}
	printf("\n}");
    } else {
	printf(" () { }");
    }
    if (f->redir) {
	t = getpermtext(f->redir, NULL, 1);
	if (t) {
	    zputs(t, stdout);
	    zsfree(t);
	}
    }

    putchar('\n');
}

/*
 * Wrap scanmatchtable for shell functions with optional
 * expansion of leading tabs.
 * expand = 0 is standard: use hard tabs.
 * expand > 0 uses that many spaces.
 * expand < 0 uses no identation.
 *
 * Note this function and the following two are called with
 * interrupts queued, so saving and restoring text_expand_tabs
 * is safe.
 */

/**/
mod_export int
scanmatchshfunc(Patprog pprog, int sorted, int flags1, int flags2,
		ScanFunc scanfunc, int scanflags, int expand)
{
    int ret, save_expand;

    save_expand = text_expand_tabs;
    text_expand_tabs = expand;
    ret = scanmatchtable(shfunctab, pprog, sorted, flags1, flags2,
			scanfunc, scanflags);
    text_expand_tabs = save_expand;

    return ret;
}

/* Wrap scanhashtable to expand tabs for shell functions */

/**/
mod_export int
scanshfunc(int sorted, int flags1, int flags2,
	      ScanFunc scanfunc, int scanflags, int expand)
{
    return scanmatchshfunc(NULL, sorted, flags1, flags2,
			   scanfunc, scanflags, expand);
}

/* Wrap shfunctab->printnode to expand tabs */

/**/
mod_export void
printshfuncexpand(HashNode hn, int printflags, int expand)
{
    int save_expand;

    save_expand = text_expand_tabs;
    text_expand_tabs = expand;
    shfunctab->printnode(hn, printflags);
    text_expand_tabs = save_expand;
}

/**************************************/
/* Reserved Word Hash Table Functions */
/**************************************/

/* Nodes for reserved word hash table */

static struct reswd reswds[] = {
    {{NULL, "!", 0}, BANG},
    {{NULL, "[[", 0}, DINBRACK},
    {{NULL, "{", 0}, INBRACE},
    {{NULL, "}", 0}, OUTBRACE},
    {{NULL, "case", 0}, CASE},
    {{NULL, "coproc", 0}, COPROC},
    {{NULL, "declare", 0}, TYPESET},
    {{NULL, "do", 0}, DOLOOP},
    {{NULL, "done", 0}, DONE},
    {{NULL, "elif", 0}, ELIF},
    {{NULL, "else", 0}, ELSE},
    {{NULL, "end", 0}, ZEND},
    {{NULL, "esac", 0}, ESAC},
    {{NULL, "export", 0}, TYPESET},
    {{NULL, "fi", 0}, FI},
    {{NULL, "float", 0}, TYPESET},
    {{NULL, "for", 0}, FOR},
    {{NULL, "foreach", 0}, FOREACH},
    {{NULL, "function", 0}, FUNC},
    {{NULL, "if", 0}, IF},
    {{NULL, "integer", 0}, TYPESET},
    {{NULL, "local", 0}, TYPESET},
    {{NULL, "nocorrect", 0}, NOCORRECT},
    {{NULL, "readonly", 0}, TYPESET},
    {{NULL, "repeat", 0}, REPEAT},
    {{NULL, "select", 0}, SELECT},
    {{NULL, "then", 0}, THEN},
    {{NULL, "time", 0}, TIME},
    {{NULL, "typeset", 0}, TYPESET},
    {{NULL, "until", 0}, UNTIL},
    {{NULL, "while", 0}, WHILE},
    {{NULL, NULL, 0}, 0}
};

/* hash table containing the reserved words */

/**/
mod_export HashTable reswdtab;

/* Build the hash table containing zsh's reserved words. */

/**/
void
createreswdtable(void)
{
    Reswd rw;

    reswdtab = newhashtable(23, "reswdtab", NULL);

    reswdtab->hash        = hasher;
    reswdtab->emptytable  = NULL;
    reswdtab->filltable   = NULL;
    reswdtab->cmpnodes    = strcmp;
    reswdtab->addnode     = addhashnode;
    reswdtab->getnode     = gethashnode;
    reswdtab->getnode2    = gethashnode2;
    reswdtab->removenode  = NULL;
    reswdtab->disablenode = disablehashnode;
    reswdtab->enablenode  = enablehashnode;
    reswdtab->freenode    = NULL;
    reswdtab->printnode   = printreswdnode;

    for (rw = reswds; rw->node.nam; rw++)
	reswdtab->addnode(reswdtab, rw->node.nam, rw);
}

/* Print a reserved word */

/**/
static void
printreswdnode(HashNode hn, int printflags)
{
    Reswd rw = (Reswd) hn;

    if (printflags & PRINT_WHENCE_WORD) {
	printf("%s: reserved\n", rw->node.nam);
	return;
    }

    if (printflags & PRINT_WHENCE_CSH) {
	printf("%s: shell reserved word\n", rw->node.nam);
	return;
    }

    if (printflags & PRINT_WHENCE_VERBOSE) {
	printf("%s is a reserved word\n", rw->node.nam);
	return;
    }

    /* default is name only */
    printf("%s\n", rw->node.nam);
}

/********************************/
/* Aliases Hash Table Functions */
/********************************/

/* hash table containing the aliases */
 
/**/
mod_export HashTable aliastab;
 
/* has table containing suffix aliases */

/**/
mod_export HashTable sufaliastab;
 
/* Create new hash tables for aliases */

/**/
void
createaliastable(HashTable ht)
{
    ht->hash        = hasher;
    ht->emptytable  = NULL;
    ht->filltable   = NULL;
    ht->cmpnodes    = strcmp;
    ht->addnode     = addhashnode;
    ht->getnode     = gethashnode;
    ht->getnode2    = gethashnode2;
    ht->removenode  = removehashnode;
    ht->disablenode = disablehashnode;
    ht->enablenode  = enablehashnode;
    ht->freenode    = freealiasnode;
    ht->printnode   = printaliasnode;
}

/**/
void
createaliastables(void)
{
    /* Table for regular and global aliases */

    aliastab = newhashtable(23, "aliastab", NULL);

    createaliastable(aliastab);

    /* add the default aliases */
    aliastab->addnode(aliastab, ztrdup("run-help"), createaliasnode(ztrdup("man"), 0));
    aliastab->addnode(aliastab, ztrdup("which-command"), createaliasnode(ztrdup("whence"), 0));


    /* Table for suffix aliases --- make this smaller */

    sufaliastab = newhashtable(11, "sufaliastab", NULL);

    createaliastable(sufaliastab);
}

/* Create a new alias node */

/**/
mod_export Alias
createaliasnode(char *txt, int flags)
{
    Alias al;

    al = (Alias) zshcalloc(sizeof *al);
    al->node.flags = flags;
    al->text = txt;
    al->inuse = 0;
    return al;
}

/**/
static void
freealiasnode(HashNode hn)
{
    Alias al = (Alias) hn;
 
    zsfree(al->node.nam);
    zsfree(al->text);
    zfree(al, sizeof(struct alias));
}

/* Print an alias */

/**/
static void
printaliasnode(HashNode hn, int printflags)
{
    Alias a = (Alias) hn;

    if (printflags & PRINT_NAMEONLY) {
	zputs(a->node.nam, stdout);
	putchar('\n');
	return;
    }

    if (printflags & PRINT_WHENCE_WORD) {
	if (a->node.flags & ALIAS_SUFFIX)
	    printf("%s: suffix alias\n", a->node.nam);
	else
	    printf("%s: alias\n", a->node.nam);
	return;
    }

    if (printflags & PRINT_WHENCE_SIMPLE) {
	zputs(a->text, stdout);
	putchar('\n');
	return;
    }

    if (printflags & PRINT_WHENCE_CSH) {
	nicezputs(a->node.nam, stdout);
	printf(": ");
	if (a->node.flags & ALIAS_SUFFIX)
	    printf("suffix ");
	else if (a->node.flags & ALIAS_GLOBAL)
	    printf("globally ");
	printf ("aliased to ");
	nicezputs(a->text, stdout);
	putchar('\n');
	return;
    }

    if (printflags & PRINT_WHENCE_VERBOSE) {
	nicezputs(a->node.nam, stdout);
	printf(" is a");
	if (a->node.flags & ALIAS_SUFFIX)
	    printf(" suffix");
	else if (a->node.flags & ALIAS_GLOBAL)
	    printf(" global");
	else
	    printf("n");
	printf(" alias for ");
	nicezputs(a->text, stdout);
	putchar('\n');
	return;
    }

    if (printflags & PRINT_LIST) {
	printf("alias ");
	if (a->node.flags & ALIAS_SUFFIX)
	    printf("-s ");
	else if (a->node.flags & ALIAS_GLOBAL)
	    printf("-g ");

	/* If an alias begins with `-', then we must output `-- ' *
	 * first, so that it is not interpreted as an option.     */
	if(a->node.nam[0] == '-')
	    printf("-- ");
    }

    quotedzputs(a->node.nam, stdout);
    putchar('=');
    quotedzputs(a->text, stdout);

    putchar('\n');
}

/*************************************/
/* History Line Hash Table Functions */
/*************************************/

/**/
void
createhisttable(void)
{
    histtab = newhashtable(599, "histtab", NULL);

    histtab->hash        = histhasher;
    histtab->emptytable  = emptyhisttable;
    histtab->filltable   = NULL;
    histtab->cmpnodes    = histstrcmp;
    histtab->addnode     = addhistnode;
    histtab->getnode     = gethashnode2;
    histtab->getnode2    = gethashnode2;
    histtab->removenode  = removehashnode;
    histtab->disablenode = NULL;
    histtab->enablenode  = NULL;
    histtab->freenode    = freehistnode;
    histtab->printnode   = NULL;
}

/**/
unsigned
histhasher(const char *str)
{
    unsigned hashval = 0;

    while (inblank(*str)) str++;

    while (*str) {
	if (inblank(*str)) {
	    do str++; while (inblank(*str));
	    if (*str)
		hashval += (hashval << 5) + ' ';
	}
	else
	    hashval += (hashval << 5) + *(unsigned char *)str++;
    }
    return hashval;
}

/**/
void
emptyhisttable(HashTable ht)
{
    emptyhashtable(ht);
    if (hist_ring)
	histremovedups();
}

/* Compare two strings with normalized white-space */

/**/
int
histstrcmp(const char *str1, const char *str2)
{
    while (inblank(*str1)) str1++;
    while (inblank(*str2)) str2++;
    while (*str1 && *str2) {
	if (inblank(*str1)) {
	    if (!inblank(*str2))
		break;
	    do str1++; while (inblank(*str1));
	    do str2++; while (inblank(*str2));
	}
	else {
	    if (*str1 != *str2)
		break;
	    str1++;
	    str2++;
	}
    }
    return *str1 - *str2;
}

/**/
void
addhistnode(HashTable ht, char *nam, void *nodeptr)
{
    HashNode oldnode = addhashnode2(ht, nam, nodeptr);
    Histent he = (Histent)nodeptr;
    if (oldnode && oldnode != (HashNode)nodeptr) {
	if (he->node.flags & HIST_MAKEUNIQUE
	 || (he->node.flags & HIST_FOREIGN && (Histent)oldnode == he->up)) {
	    (void) addhashnode2(ht, oldnode->nam, oldnode); /* restore hash */
	    he->node.flags |= HIST_DUP;
	    he->node.flags &= ~HIST_MAKEUNIQUE;
	}
	else {
	    oldnode->flags |= HIST_DUP;
	    if (hist_ignore_all_dups)
		freehistnode(oldnode); /* Remove the old dup */
	}
    }
    else
	he->node.flags &= ~HIST_MAKEUNIQUE;
}

/**/
void
freehistnode(HashNode nodeptr)
{
    freehistdata((Histent)nodeptr, 1);
    zfree(nodeptr, sizeof (struct histent));
}

/**/
void
freehistdata(Histent he, int unlink)
{
    if (!he)
	return;

    if (!(he->node.flags & (HIST_DUP | HIST_TMPSTORE)))
	removehashnode(histtab, he->node.nam);

    zsfree(he->node.nam);
    if (he->nwords)
	zfree(he->words, he->nwords*2*sizeof(short));

    if (unlink) {
	if (!--histlinect)
	    hist_ring = NULL;
	else {
	    if (he == hist_ring)
		hist_ring = hist_ring->up;
	    he->up->down = he->down;
	    he->down->up = he->up;
	}
    }
}

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 14:44 ` Sebastian Gniazdowski
@ 2015-09-12 14:57   ` Mikael Magnusson
  2015-09-12 15:28     ` Bart Schaefer
  2015-09-12 16:08   ` Bart Schaefer
  1 sibling, 1 reply; 13+ messages in thread
From: Mikael Magnusson @ 2015-09-12 14:57 UTC (permalink / raw)
  To: Sebastian Gniazdowski; +Cc: zsh workers

On Sat, Sep 12, 2015 at 4:44 PM, Sebastian Gniazdowski
<sgniazdowski@gmail.com> wrote:
> Hello,
> I was looking for a way to be able to use colors on zsh < 5.1.1. Did
> some debugging:

> As it can be seen, hash misses are caused by the DISABLED flag ( "if
> (hp->flags & DISABLED) {" in function gethashnode). I attach two files
> with debug prints that produced the output. --- lines are hash table
> dumps, == are gethashnode() interiors.

It's preferable to send unified diff patches, inline in the email,
rather than full files attached to the message.

-- 
Mikael Magnusson


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 14:57   ` Mikael Magnusson
@ 2015-09-12 15:28     ` Bart Schaefer
  0 siblings, 0 replies; 13+ messages in thread
From: Bart Schaefer @ 2015-09-12 15:28 UTC (permalink / raw)
  To: zsh workers

On Sep 12,  4:57pm, Mikael Magnusson wrote:
} Subject: Re: Patch for curses module
}
} > I attach two files
} > with debug prints that produced the output.
} 
} It's preferable to send unified diff patches, inline in the email,
} rather than full files attached to the message.

True, but these aren't intended as patches, so I don't think having
them in-line would have been helpful.

-- 
Barton E. Schaefer


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 14:44 ` Sebastian Gniazdowski
  2015-09-12 14:57   ` Mikael Magnusson
@ 2015-09-12 16:08   ` Bart Schaefer
  2015-09-12 16:45     ` Sebastian Gniazdowski
  1 sibling, 1 reply; 13+ messages in thread
From: Bart Schaefer @ 2015-09-12 16:08 UTC (permalink / raw)
  To: Sebastian Gniazdowski, zsh-workers

On Sep 12,  4:44pm, Sebastian Gniazdowski wrote:
}
} As it can be seen, hash misses are caused by the DISABLED flag ( "if
} (hp->flags & DISABLED) {" in function gethashnode). I attach two files
} with debug prints that produced the output. --- lines are hash table
} dumps, == are gethashnode() interiors. Can someone explain why
} sometimes a node has the DISABLED flag?

Try this?


diff --git a/Src/Modules/curses.c b/Src/Modules/curses.c
index 0054aef..64329f6 100644
--- a/Src/Modules/curses.c
+++ b/Src/Modules/curses.c
@@ -370,7 +370,7 @@ zcurses_colorget(const char *nam, char *colorpair)
 	    return NULL;
 	}
 
-	cpn = (Colorpairnode)zalloc(sizeof(struct colorpairnode));
+	cpn = (Colorpairnode)zshcalloc(sizeof(struct colorpairnode));
 	
 	if (!cpn) {
 	    zsfree(cp);
@@ -462,7 +462,7 @@ zccmd_init(const char *nam, char **args)
 	    use_default_colors();
 #endif
 	    /* Initialise the default color pair, always 0 */
-	    cpn = (Colorpairnode)zalloc(sizeof(struct colorpairnode));
+	    cpn = (Colorpairnode)zshcalloc(sizeof(struct colorpairnode));
 	    if (cpn) {
 		cpn->colorpair = 0;
 		addhashnode(zcurses_colorpairs,


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 16:08   ` Bart Schaefer
@ 2015-09-12 16:45     ` Sebastian Gniazdowski
  2015-09-12 17:02       ` Bart Schaefer
  2015-09-12 17:40       ` Sebastian Gniazdowski
  0 siblings, 2 replies; 13+ messages in thread
From: Sebastian Gniazdowski @ 2015-09-12 16:45 UTC (permalink / raw)
  To: zsh-workers

With this patch color pairs are created at first use and then found
and returned with no errors. So the patch helps.

Wonder if there is a workaround to have memory from zalloc() cleared
e.g. by proper sequence of calls in zsh script or by an environment
variable, etc.

Best regards,
Sebastian Gniazdowski


On 12 September 2015 at 18:08, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 12,  4:44pm, Sebastian Gniazdowski wrote:
> }
> } As it can be seen, hash misses are caused by the DISABLED flag ( "if
> } (hp->flags & DISABLED) {" in function gethashnode). I attach two files
> } with debug prints that produced the output. --- lines are hash table
> } dumps, == are gethashnode() interiors. Can someone explain why
> } sometimes a node has the DISABLED flag?
>
> Try this?
>
>
> diff --git a/Src/Modules/curses.c b/Src/Modules/curses.c
> index 0054aef..64329f6 100644
> --- a/Src/Modules/curses.c
> +++ b/Src/Modules/curses.c
> @@ -370,7 +370,7 @@ zcurses_colorget(const char *nam, char *colorpair)
>             return NULL;
>         }
>
> -       cpn = (Colorpairnode)zalloc(sizeof(struct colorpairnode));
> +       cpn = (Colorpairnode)zshcalloc(sizeof(struct colorpairnode));
>
>         if (!cpn) {
>             zsfree(cp);
> @@ -462,7 +462,7 @@ zccmd_init(const char *nam, char **args)
>             use_default_colors();
>  #endif
>             /* Initialise the default color pair, always 0 */
> -           cpn = (Colorpairnode)zalloc(sizeof(struct colorpairnode));
> +           cpn = (Colorpairnode)zshcalloc(sizeof(struct colorpairnode));
>             if (cpn) {
>                 cpn->colorpair = 0;
>                 addhashnode(zcurses_colorpairs,


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 16:45     ` Sebastian Gniazdowski
@ 2015-09-12 17:02       ` Bart Schaefer
  2015-09-12 17:40       ` Sebastian Gniazdowski
  1 sibling, 0 replies; 13+ messages in thread
From: Bart Schaefer @ 2015-09-12 17:02 UTC (permalink / raw)
  To: zsh-workers

On Sep 12,  6:45pm, Sebastian Gniazdowski wrote:
}
} Wonder if there is a workaround to have memory from zalloc() cleared
} e.g. by proper sequence of calls in zsh script or by an environment
} variable, etc.

Very unlikely.  It would depend on the system malloc library
implementation, if there were any possibility of doing it with an
environment setting.  If it were possible to create arbitrary memory
state on the malloc heap by proper sequence of script calls, that
would be a potential security issue.

There's probably some way to write a script loop that keeps creating
the same color entry repeatedly until it is successfully looked up,
but I don't know what that code would look like.


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 16:45     ` Sebastian Gniazdowski
  2015-09-12 17:02       ` Bart Schaefer
@ 2015-09-12 17:40       ` Sebastian Gniazdowski
  2015-09-12 18:33         ` Bart Schaefer
  1 sibling, 1 reply; 13+ messages in thread
From: Sebastian Gniazdowski @ 2015-09-12 17:40 UTC (permalink / raw)
  To: zsh-workers

Bart Schaefer <schaefer@brasslantern.com> wrote:

} Very unlikely.  It would depend on the system malloc library
} implementation, if there were any possibility of doing it with an
} environment setting.  If it were possible to create arbitrary memory
} state on the malloc heap by proper sequence of script calls, that
} would be a potential security issue.

} There's probably some way to write a script loop that keeps creating
} the same color entry repeatedly until it is successfully looked up,
} but I don't know what that code would look like.

I wonder how can creating subsequent color pair make previous pairs
non-disabled. It happens in random manner, sometimes it helps and
sometimes not. In first reply I attached a log where red/black fixed
white/black

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 17:40       ` Sebastian Gniazdowski
@ 2015-09-12 18:33         ` Bart Schaefer
  2015-09-13 11:58           ` Sebastian Gniazdowski
  0 siblings, 1 reply; 13+ messages in thread
From: Bart Schaefer @ 2015-09-12 18:33 UTC (permalink / raw)
  To: zsh-workers

On Sep 12,  7:40pm, Sebastian Gniazdowski wrote:
}
} I wonder how can creating subsequent color pair make previous pairs
} non-disabled. It happens in random manner, sometimes it helps and
} sometimes not. In first reply I attached a log where red/black fixed
} white/black

Looking at your trace, I see white/black being created twice, the
second time after it was cache-missed for DISABLED but also before
the red/black is created.  The second time white/black is created
is when I believe it "fixes" the DISABLED bit.

Am I misreading the trace, or misunderstanding the sequence of events
that created the trace?


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-12 18:33         ` Bart Schaefer
@ 2015-09-13 11:58           ` Sebastian Gniazdowski
  2015-09-13 16:47             ` Bart Schaefer
  0 siblings, 1 reply; 13+ messages in thread
From: Sebastian Gniazdowski @ 2015-09-13 11:58 UTC (permalink / raw)
  To: zsh-workers; +Cc: Bart Schaefer

You are apparently right, it wasn't the next color pair that fixed
white/black, but new creation of white/black that at one time
succeeded.

I'm now trying to provide a clean memory block for the zalloc call by
doing preceding 16 byte allocations and frees. The 16 bytes comes from
size of colorpairnode struct. It's 16 on x86 and 32 on 64 bit machine.
I am hoping that zalloc will reuse the memory blocks. Doing simple
test in C showed that it's reasonable to expect this:

#include <stdlib.h>
#include <stdio.h>

int main() {
    char * a = (char*) malloc(16);
    free(a);
    char * b = (char*) malloc(16);
    free(b);

    printf(" %p vs %p ", a, b );
    return 0;
}

The test worked on FreeBSD, OSX and Linux. I also added traces to
zalloc() to ensure that it will call malloc(16), and obtained that
this will be done for 15 chars long string assignment (but not for 15
zeros '$\0...', what's interesting).

So I do the following before calling zcurses:

_provide_clean_memory_to_old_zsh() {
    local i=100
    while (( i-- )); do
        local a b
        a="               "
        b="               "
        unset a b
    done
}

The disabled flag is defined as follows:

#define DISABLED        (1<<0)

Hash node which is part of colorpairnode as:
struct hashnode {
    HashNode next;          /* next in hash chain */
    char *nam;                  /* hash key           */
    int flags;                      /* various flags      */
};

Therefore filling a 16/32 byte area with zeros or e.g. spaces and then
freeing it should provide proper memory for zcurses. But this doesn't
happen. Does unset free memory? What can be done to allocate and free
16/32 bytes block?

Best regards,
Sebastian Gniazdowski

On 12 September 2015 at 20:33, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 12,  7:40pm, Sebastian Gniazdowski wrote:
> }
> } I wonder how can creating subsequent color pair make previous pairs
> } non-disabled. It happens in random manner, sometimes it helps and
> } sometimes not. In first reply I attached a log where red/black fixed
> } white/black
>
> Looking at your trace, I see white/black being created twice, the
> second time after it was cache-missed for DISABLED but also before
> the red/black is created.  The second time white/black is created
> is when I believe it "fixes" the DISABLED bit.
>
> Am I misreading the trace, or misunderstanding the sequence of events
> that created the trace?


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-13 11:58           ` Sebastian Gniazdowski
@ 2015-09-13 16:47             ` Bart Schaefer
  2015-09-13 19:35               ` Sebastian Gniazdowski
  0 siblings, 1 reply; 13+ messages in thread
From: Bart Schaefer @ 2015-09-13 16:47 UTC (permalink / raw)
  To: zsh-workers

On Sep 13,  1:58pm, Sebastian Gniazdowski wrote:
}
} _provide_clean_memory_to_old_zsh() {
}     local i=100
}     while (( i-- )); do
}         local a b
}         a="               "
}         b="               "
}         unset a b
}     done
} }

If malloc works the way you think, that loop is just going to allocate
and free the same two blocks over and over.  Declaring "local" inside a
loop doesn't have any special meaning.  You would at least need e.g.

  while (( i-- )); do
    local a$i="                 " b$i="                 "
  done

Then because declared local, all the a$i and b$i will be freed at the end
of the function scope, you don't need an explicit unset.

} Hash node which is part of colorpairnode as:
} struct hashnode {
}     HashNode next;          /* next in hash chain */
}     char *nam;                  /* hash key           */
}     int flags;                      /* various flags      */
} };

Yes, but you need to clear single blocks the size of the entire
colorpairnode struct (18+ bytes, a hashnode plus a short) rather
than just the size of the prefix hashnode.

} Therefore filling a 16/32 byte area with zeros or e.g. spaces and then
} freeing it should provide proper memory for zcurses. But this doesn't
} happen. Does unset free memory? What can be done to allocate and free
} 16/32 bytes block?

Even if you get the size right, the malloc library isn't guaranteed to
re-use freed memory in LIFO order.  It may (re)use it in the order
that it can access it the fastest, whether or not that's best-fit.
It might also aggressively release freed blocks back to the OS, in
which case any value you write there may be clobbered by another
thread even if you do get back the same block you previously freed.

You can try to thwart this by allocating something that is NOT freed
every so often, so that there's less chance of a contiguous block for
malloc to release, but ultimately trying to control memory layout
from the shell script level is never going to produce a consistent
result.


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-13 16:47             ` Bart Schaefer
@ 2015-09-13 19:35               ` Sebastian Gniazdowski
  2015-09-13 21:02                 ` Bart Schaefer
  0 siblings, 1 reply; 13+ messages in thread
From: Sebastian Gniazdowski @ 2015-09-13 19:35 UTC (permalink / raw)
  To: zsh-workers

On 13 September 2015 at 18:47, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 13,  1:58pm, Sebastian Gniazdowski wrote:
> }
> } _provide_clean_memory_to_old_zsh() {
> }     local i=100
> }     while (( i-- )); do
> }         local a b
> }         a="               "
> }         b="               "
> }         unset a b
> }     done
> } }
>
> If malloc works the way you think, that loop is just going to allocate
> and free the same two blocks over and over.  Declaring "local" inside a
> loop doesn't have any special meaning.  You would at least need e.g.
>
>   while (( i-- )); do
>     local a$i="                 " b$i="                 "
>   done

Maybe local means that the memory comes from stack, while zalloc()
works on heap? I tried with global variables but still no luck though.

> Then because declared local, all the a$i and b$i will be freed at the end
> of the function scope, you don't need an explicit unset.
>
> } Hash node which is part of colorpairnode as:
> } struct hashnode {
> }     HashNode next;          /* next in hash chain */
> }     char *nam;                  /* hash key           */
> }     int flags;                      /* various flags      */
> } };
>
> Yes, but you need to clear single blocks the size of the entire
> colorpairnode struct (18+ bytes, a hashnode plus a short) rather
> than just the size of the prefix hashnode.

I have printed sizeof of the struct and it's 16, and 32 on 64 bit machine.

> } Therefore filling a 16/32 byte area with zeros or e.g. spaces and then
> } freeing it should provide proper memory for zcurses. But this doesn't
> } happen. Does unset free memory? What can be done to allocate and free
> } 16/32 bytes block?
>
> Even if you get the size right, the malloc library isn't guaranteed to
> re-use freed memory in LIFO order.  It may (re)use it in the order
> that it can access it the fastest, whether or not that's best-fit.
> It might also aggressively release freed blocks back to the OS, in
> which case any value you write there may be clobbered by another
> thread even if you do get back the same block you previously freed.

Yes it's hard to say about guarantees with such trick, good that it's
only about colors and older zsh versions.

Best regards,
Sebastian Gniazdowski


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: Patch for curses module
  2015-09-13 19:35               ` Sebastian Gniazdowski
@ 2015-09-13 21:02                 ` Bart Schaefer
  0 siblings, 0 replies; 13+ messages in thread
From: Bart Schaefer @ 2015-09-13 21:02 UTC (permalink / raw)
  To: zsh-workers

On Sep 13,  9:35pm, Sebastian Gniazdowski wrote:
} 
} Maybe local means that the memory comes from stack, while zalloc()
} works on heap?

No, parameters are always allocated from the global malloc arena, even
if declared local.  Zsh's internal heap (which is itself allocated
from malloc) is is used only when expanding $param expressions.

} I tried with global variables but still no luck though.
} 
} I have printed sizeof of the struct and it's 16, and 32 on 64 bit machine.

Then your best bet is to create a single parameter 32 bytes in size.
But note that an assignment like that is three allocations, a struct
param plus the name and value, so when you get around to allocating
the colorpairnode you're just as likely to get the freed param as the
space used for the value; but scalars have a 0 in the flags (whereas
PM_ARRAY and DISABLED use the same bits) so that may be OK.


^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2015-09-13 21:02 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-09-11 17:35 Patch for curses module Sebastian Gniazdowski
2015-09-12 14:44 ` Sebastian Gniazdowski
2015-09-12 14:57   ` Mikael Magnusson
2015-09-12 15:28     ` Bart Schaefer
2015-09-12 16:08   ` Bart Schaefer
2015-09-12 16:45     ` Sebastian Gniazdowski
2015-09-12 17:02       ` Bart Schaefer
2015-09-12 17:40       ` Sebastian Gniazdowski
2015-09-12 18:33         ` Bart Schaefer
2015-09-13 11:58           ` Sebastian Gniazdowski
2015-09-13 16:47             ` Bart Schaefer
2015-09-13 19:35               ` Sebastian Gniazdowski
2015-09-13 21:02                 ` Bart Schaefer

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).