Install run-help and *.zwc files system wide in build system

Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

Hello,

I am maintaining for Gentoo linux a user overlay which installs
files for "run-help" system wide. In my opinion this is very
important since new zsh users (who nowadays typically come from bash)
are used to bash's "help", and IMHO something similar should be
possible out-of-the-box for them.

The "documented way" to produce the files for run-help is only
suboptimal, because:

1. The files are only installed per-user and not system-wide which
would be clearly preferrable.

2. It does not work out-of-the box: With current man/groff versions,
one must export (an empty) GROFF_NO_SGR, MANWIDTH=80, possibly
unset MANPL, MANROFFSEQ, set utf8-aware locales etc.

In a recent discussion with the gentoo zsh maintainer,
the conclusion was to make the following suggestions to you:

(a) It would be nice if zsh would be able to install
system-wide help files in the build process.

(b) Similarly, it would be nice if zsh would be able to
install *.zwc files for the functions it provides in the
build process.

Of course, both should happen only if appropriate ./configure
options are passed (e.g. compiling *.zwc files is only possible
in the build process if no cross-compilation is happening,
and building help-files requires tools like man, groff, and perl
which one should not require just to _build_ a zsh).

If there is interest, I will write and post a script (and perhaps
also a patch to the build system to call this script) which
generates the files for run-help into a specified directory
and tries to take care off all the issues mentioned in 2.
This would take care of (a).

I am not sure what is the best strategy to (b), since package
managers tend to change timestamps of directories/filse when
copying installed files from a building sandbox or binary archive
to the live filesystem:
Somehow one must guarantee that the *.zwc files have the newest
filestamps, since otherwise they are ignored by zsh (AFAIK).
Should one perhaps produce a script (or at least a list of files)
in the building process which will touch these files so that this
script can then just be called in a post-install hook?

Regards
Martin Väth

Re: Install run-help and *.zwc files system wide in build system

[Peter Stephenson]

On Sun, 10 Nov 2013 08:57:11 +0000 (UTC)
Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
> (a) It would be nice if zsh would be able to install
> system-wide help files in the build process.
> 
> If there is interest, I will write and post a script (and perhaps
> also a patch to the build system to call this script) which
> generates the files for run-help into a specified directory
> and tries to take care off all the issues mentioned in 2.
> This would take care of (a).

I'd be very happy to put this in the distribution if you were able to do
it.

I don't use .zwc files, so I'll leave others to discuss that.

pws

Re: Install run-help and *.zwc files system wide in build system

[Bart Schaefer]

On Nov 10,  8:57am, Martin Vaeth wrote:
}
} The "documented way" to produce the files for run-help is only
} suboptimal, because:
} 
} 2. It does not work out-of-the box: With current man/groff versions,
} one must export (an empty) GROFF_NO_SGR, MANWIDTH=80, possibly
} unset MANPL, MANROFFSEQ, set utf8-aware locales etc.

Of course the "right" way to go about this would be to generate the
run-help files directly from the yodl source, or at least to generate
formatted output from the yodl that could more easly be split up by
Util/helpfiles.

That would mean creating a "zhelp.yo" file patterned after Doc/zman.yo
and Doc/ztexi.yo plus Makefile.in rules using "yodl -w zhelp.yo ..."
 
} (b) Similarly, it would be nice if zsh would be able to
} install *.zwc files for the functions it provides in the
} build process.

Someone should do (again? I wonder if it ever was done) benchmarking
of whether .zwc files really save any time/CPU.  I'm sure they do for
sets of functions that are all going to be used during the session;
but I'm doubtful that packing a bunch of autoloads, most of which are
never actually loaded, into one big library file, is more effective
than reading the individual files of only the subset that is used.

It certainly doesn't save disk space; in nearly all cases ZWC files
are bigger than their text counterparts, because they have to retain
the original text for XTRACE purposes and no compression is applied.

} Of course, both should happen only if appropriate ./configure
} options are passed (e.g. compiling *.zwc files is only possible
} in the build process if no cross-compilation is happening,
} and building help-files requires tools like man, groff, and perl
} which one should not require just to _build_ a zsh).

This is why the precompiled doc is distributed in a separate tarball
from the shell sources.  If we're going to start building run-help
files (or special Util/helpfiles input) as well, then those will need
to be added to the doc tarball.

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1667 bytes --]

Please find attached the announced patches for
run-help files generation, for simplicity all bundled into
one patch against current git master.
(This is my first attempt to post a patch to a mailing list;
I hope attachments are allowed).

Main part is a new (Bourne) shell script "generate-help.sh" which
uses man, clcrt, and Util/helpfiles (hence perl) to generate the
run-help files.

There are further patches to the build system, to run-help itself
and, since I was at it, I also added a _run-help completion file
to the patch.

The purpose of the patched build system is to call
generate-help.sh and to install/uninstall the help directoy
unless
./configure --disable-runhelpdir
or
./configure --enable-runhelpdir=
is used. (To make the patch active, autotools must be used
to generate the new configure script after the patch, of course).

run-help itself is patched to default to the default directory
HELPDIR=/usr/share/zsh/$ZSH_VERSION/help

The meaning of the new _run-help completion file should be clear...
(note that this file is not used, by default, since by default,
run-help is an alias to "man", and so, of course, usually the
"man" completion is used instead).


I agree, of course, that "generate-help.sh" should better
generate the run-help files directly from yodl, but I am not (yet)
familiar enough with yodl to do this.
If eventually somebody writes a script which can do this, this can
simply replace the current "generate-help.sh".

In the moment, I have not attempted to put the generated
run-help directory into the tarball, since this can lead
to other problems: This directory contains symlinks which
are not available on all systems...

[-- Attachment #2: Type: TEXT/PLAIN, Size: 1168 bytes --]

--- eix-0.29.6.ebuild
+++ eix-0.30.0.ebuild
@@ -14,7 +14,7 @@
 LICENSE="GPL-2"
 SLOT="0"
 KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~ppc-aix ~amd64-fbsd ~sparc-fbsd ~x86-fbsd ~x86-freebsd ~ia64-hpux ~x86-interix ~amd64-linux ~ia64-linux ~x86-linux ~ppc-macos ~x64-macos ~x86-macos ~sparc-solaris ~sparc64-solaris ~x86-solaris"
-IUSE="clang debug +dep doc nls optimization security strong-optimization strong-security sqlite swap-remote tools"
+IUSE="debug +dep doc nls optimization security strong-optimization strong-security sqlite swap-remote tools"
 
 BOTHDEPEND="sqlite? ( >=dev-db/sqlite-3 )
 	nls? ( virtual/libintl )"
@@ -22,7 +22,6 @@
 	app-shells/push"
 DEPEND="${BOTHDEPEND}
 	app-arch/xz-utils
-	clang? ( sys-devel/clang )
 	nls? ( sys-devel/gettext )"
 
 pkg_setup() {
@@ -46,7 +45,6 @@
 		$(use_enable swap-remote) \
 		$(use_with prefix always-accept-keywords) \
 		$(use_with dep dep-default) \
-		$(use_with clang nongnu-cxx clang++) \
 		--with-zsh-completion \
 		--with-ebuild-sh-default="/usr/$(get_libdir)/portage/bin/ebuild.sh" \
 		--with-portage-rootpath="${ROOTPATH}" \

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 61 bytes --]

Sorry, I attached the wrong patch. The correct one is here...

[-- Attachment #2: Type: TEXT/PLAIN, Size: 6791 bytes --]

--- 1/Completion/Zsh/Command/_run-help
+++ 1/Completion/Zsh/Command/_run-help
@@ -0,0 +1,7 @@
+#compdef run-help
+local d expl
+local HELPDIR=${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}
+[[ -d $HELPDIR ]] && {
+	d=($HELPDIR/*(:t))
+	(($#d)) && _wanted commands expl 'command' compadd -a d
+} || _man
--- 1/Doc/Makefile.in
+++ 1/Doc/Makefile.in
@@ -33,6 +33,7 @@
 VPATH           = @srcdir@
 sdir            = @srcdir@
 sdir_top        = @top_srcdir@
+runhelpdir      = @runhelpdir@
 INSTALL         = @INSTALL@
 
 @DEFS_MK@
@@ -82,7 +83,7 @@
 
 # ========== DEPENDENCIES FOR BUILDING ==========
 
-all: man texi ../META-FAQ
+all: man runhelp texi ../META-FAQ
 .PHONY: all
 
 everything: all dvi html pdf info
@@ -183,6 +184,11 @@
 man: $(MAN)
 .PHONY: man
 
+runhelp: man
+	test x"$(runhelpdir)" = x"" || \
+	  ${SHELL} $(sdir)/generate-help.sh $(sdir_top) $(sdir)/runhelp $(lc_ctype)
+.PHONY: runhelp
+
 $(MAN): zmacros.yo zman.yo
 
 zsh.1 zshall.1: Zsh/intro.yo Zsh/metafaq.yo Zsh/invoke.yo Zsh/files.yo \
@@ -285,12 +291,12 @@
 
 # ========== DEPENDENCIES FOR INSTALLING ==========
 
-# install just installs the manual pages
-install: install.man
+# install just installs the manual and runhelp pages
+install: install.man install.runhelp
 .PHONY: install
 
-# uninstall just unistalls the manual pages
-uninstall: uninstall.man
+# uninstall just uninstalls the manual and runhelp pages
+uninstall: uninstall.man uninstall.runhelp
 .PHONY: uninstall
 
 # install man pages, creating install directory if necessary
@@ -302,6 +308,17 @@
 	done
 .PHONY: install.man
 
+# install runhelp pages, creating install directory if necessary
+install.runhelp: runhelp
+	if test x"$(runhelpdir)" != x""; then \
+	    ${SHELL} $(sdir_top)/mkinstalldirs $(DESTDIR)$(runhelpdir); \
+	    for file in $(sdir)/runhelp/*; do \
+	        $(INSTALL_DATA) $$file $(DESTDIR)$(runhelpdir); \
+	        test -h $$file && cp -P $$file $(DESTDIR)$(runhelpdir) || :; \
+	  done; \
+	fi
+.PHONY: install.runhelp
+
 # install info pages, creating install directory if necessary
 install.info: texi
 	${SHELL} $(sdir_top)/mkinstalldirs $(DESTDIR)$(infodir)
@@ -334,6 +349,11 @@
 	done
 .PHONY: uninstall.man
 
+# uninstall runhelp pages
+uninstall.runhelp:
+	test x"$(runhelpdir)" = x"" || rm -rf -- $(DESTDIR)$(runhelpdir)
+.PHONY: uninstall.runhelp
+
 # uninstall info pages
 uninstall.info:
 	rm -f $(DESTDIR)$(infodir)/$(tzsh).info
--- 1/Doc/generate-help.sh
+++ 1/Doc/generate-help.sh
@@ -0,0 +1,83 @@
+#!/usr/bin/env sh
+
+Echo() {
+	printf '%s\n' "${*}"
+}
+
+[ ${#} -ge 2 ] && test -f "${1}/Doc/zshbuiltins.1" || { Echo \
+'Usage: generate-help.sh zsh-sources-directory destination-directory [LC_CTYPE]
+If you pass LC_CTYPE it must be utf8 aware.
+If you do not pass LC_CTYPE a heuristics is used to guess some for your system'
+	exit 1
+}
+
+case ${1} in
+/*) S=${1};;
+*) S=${PWD}/${1};;
+esac
+D=${2}
+
+Die() {
+	Echo "generate-help.sh: fatal: ${*}" >&2
+	exit 1
+}
+
+Info() {
+	Echo "generate-help.sh: ${*}"
+}
+
+test -d "${D}" || mkdir -- "${D}" || Die "failed to create ${D}"
+
+# We need GROFF_NO_SGR to produce "classical" formatting:
+export GROFF_NO_SGR=
+
+# This is self-explaining
+export MANWIDTH=80
+
+# In case the user has exported this, we restore the default:
+unset MANPL MANROFFSEQ
+
+# Now we must prepare locales:
+# Problem is LC_CTYPE since this must be utf8-aware for man to work properly.
+# Since there is no standard utf8-aware locale, we must use heuristics.
+unset LC_ALL LC_CTYPE
+export LANG=C
+if [ ${#} -eq 3 ]
+then	LC_CTYPE=${3}
+else	# Prefer en*utf8* (case ignored) over *utf8* over en*|*.* over rest
+	unset i j
+	for i in `locale -a 2>/dev/null`
+	do	case ${i} in
+		en*[uU][tT][fF]8*)
+			LC_CTYPE=${i}
+			break;;
+		*[uU][tT][fF]8*)
+			[ -n "${LC_CTYPE}" ] || LC_CTYPE=${i};;
+		en*|*.*)
+			j=${i};;
+		esac
+	done
+	[ -n "${LC_CTYPE}" ] || [ -z "${j}" ] || LC_CTYPE=${j}
+fi
+if [ -z "${LC_CTYPE}" ]
+then	Info "LC_CTYPE unset"
+else	Info "LC_CTYPE=${LC_CTYPE}"
+	export LC_CTYPE
+fi
+
+Info "generating files into ${D}"
+
+# It is necessary to be paranoid about the success of the following pipe,
+# since any change in locale or environment can break it completely.
+# In the lack of pipestatus, we better use temporary files
+rm -f -- "${D}/tmp1" "${D}/tmp2"
+man "${S}/Doc/zshbuiltins.1" >"${D}/tmp1" || Die 'man zshbuiltins.1 failed'
+colcrt "${D}/tmp1" >"${D}/tmp2" || Die 'colcrt failed'
+( cd -- "${D}" >/dev/null 2>&1 || Die "cannot cd to ${D}"
+perl "${S}/Util/helpfiles" <tmp2 || Die 'perl Util/helpfiles failed' )
+rm -f -- "${D}/tmp1" "${D}/tmp2"
+
+# Finally, we make a sanity check:
+
+test -r "${D}/zmodload" || Die 'did not produce all required files'
+Info 'success'
--- 1/Makefile.in
+++ 1/Makefile.in
@@ -63,8 +63,8 @@
 	$(MAKE) install STRIPFLAGS="-s"
 
 # install/uninstall most things
-install: install.bin install.modules install.fns install.man
-uninstall: uninstall.bin uninstall.modules uninstall.fns uninstall.man
+install: install.bin install.modules install.fns install.man install.runhelp
+uninstall: uninstall.bin uninstall.modules uninstall.fns uninstall.man uninstall.runhelp
 
 # install/uninstall just the binary
 install.bin uninstall.bin:
@@ -78,6 +78,10 @@
 install.man uninstall.man:
 	@cd Doc && $(MAKE) $(MAKEDEFS) $@
 
+# install/uninstall just the runhelp files
+install.runhelp uninstall.runhelp:
+	@cd Doc && $(MAKE) $(MAKEDEFS) $@
+
 # install/uninstall just the shell functions
 install.fns:
 	if test x$(fndir) != x && test x$(fndir) != xno; then \
--- 1/StartupFiles/zshrc
+++ 1/StartupFiles/zshrc
@@ -75,7 +75,7 @@
 # Some environment variables
 export MAIL=/var/spool/mail/$USERNAME
 export LESS=-cex3M
-export HELPDIR=/usr/local/lib/zsh/help  # directory for run-help function to find docs
+export HELPDIR=/usr/share/zsh/$ZSH_VERSION/help  # directory for run-help function to find docs
 
 MAILCHECK=300
 HISTSIZE=200
--- 1/configure.ac
+++ 1/configure.ac
@@ -216,11 +216,22 @@
   AC_DEFINE_UNQUOTED(GLOBAL_ZLOGOUT, "$zlogout")
 fi
 
+ifdef([runhelpdir],[undefine([runhelpdir])])dnl
+AC_ARG_ENABLE(runhelpdir,
+AC_HELP_STRING([--enable-runhelpdir=DIR], [the default directory for the run-help files]),
+[if test x"$enableval" = xno; then
+  runhelpdir=
+else
+  runhelpdir="$enableval"
+fi], [runhelpdir="/usr/share/zsh/$VERSION/help"])
+
+
 AC_SUBST(zshenv)dnl
 AC_SUBST(zshrc)dnl
 AC_SUBST(zprofile)dnl
 AC_SUBST(zlogin)dnl
 AC_SUBST(zlogout)dnl
+AC_SUBST(runhelpdir)dnl
 
 dnl Do you want dynamically loaded binary modules.
 ifdef([dynamic],[undefine([dynamic])])dnl

Re: Install run-help and *.zwc files system wide in build system

[Bart Schaefer]

On Nov 10, 11:28pm, Martin Vaeth wrote:
} 
} (This is my first attempt to post a patch to a mailing list;
} I hope attachments are allowed).

We generally prefer the patches in-line unless that might result in
their being garbled by your email client.  Attaching them text/plain
as you did (rather than as text/x-diff or application/x-patch or
some such) is the next best thing.

Thanks for this, particularly for the locale and groff tweaks.  I'll
leave it to PWS to decide how to fold this in.  Some comments:

} since I was at it, I also added a _run-help completion file
} to the patch.

Why?  Does anybody ever actually type out "run-help" as a command and
then attempt to complete after it?  Normally run-help is auto-executed
from the corresponding widget, there's no opportunity for completion.

} I agree, of course, that "generate-help.sh" should better
} generate the run-help files directly from yodl, but I am not (yet)
} familiar enough with yodl to do this.

Another suggestion (I've probably made this one before in past years)
would be to do all of this in perl directly in Util/helpfiles, instead
of needing other external processes piped into it.

} In the moment, I have not attempted to put the generated
} run-help directory into the tarball, since this can lead
} to other problems: This directory contains symlinks which
} are not available on all systems...

For tarball purposes, the symlinks could be followed when packing ... or
we could actually use hard links, I'm not sure what benefit there is to
using symlinks in this particular case.

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> } since I was at it, I also added a _run-help completion file
> } to the patch.
>
> Why?  Does anybody ever actually type out "run-help" as a command and
> then attempt to complete after it?

After 'alias help="run-help"' (and using the preload run-help function
instead of the alias) you can use "help" as in bash:
I find it rather useful to get as completion all the zsh commands.

This is exactly the reminder which I expect from "help"
(you can use it for zsh commands even if you do not know
or do not remember the command you are looking for).
It also corresponds in a sense to what you get if you type
just "help" in bash.

> } generate the run-help files directly from yodl, but I am not (yet)
> } familiar enough with yodl to do this.
>
> Another suggestion (I've probably made this one before in past years)
> would be to do all of this in perl directly in Util/helpfiles, instead
> of needing other external processes piped into it.

This sounds rather cumbersome since it would mean that you need
parts of groff (line-wrapping and several special code) implemented
in perl.

> For tarball purposes, the symlinks could be followed when packing ... or
> we could actually use hard links, I'm not sure what benefit there is to
> using symlinks in this particular case.

At least, in the installation, it would be reasonable to keep symlinks
(if supported by the system): This saves space compared to copying, and,
in contrast to hardlinks, this information does not easily get lost
when making backups, binary packages, etc.
But perhaps one could just patch the perl file to produce a *list* of
symlinks instead which is then installed manually (depending on whether
symbolic links are supported).
When I find time (no promises yet), I will do this.

Re: Install run-help and *.zwc files system wide in build system

[Bart Schaefer]

On Nov 11,  9:09am, Martin Vaeth wrote:
}
} > Another suggestion (I've probably made this one before in past years)
} > would be to do all of this in perl directly in Util/helpfiles, instead
} > of needing other external processes piped into it.
} 
} This sounds rather cumbersome since it would mean that you need
} parts of groff (line-wrapping and several special code) implemented
} in perl.

No, you misunderstand.  Perl would still run and read the man|colcrt
pipeline, but all the environment setup and locale testing would be
done in perl instead of in shell.  Convert generate-help.sh into perl,
but not the external commands generate-help.sh runs.

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
> Convert generate-help.sh into perl

This makes sense.

In order to avoid problems on systems without symlinks
and to solve problems with relative paths, I think it is
most useful to patch the helpfiles script directly.
I have done so in the appended patch.

Now the generated directory Doc/help and files Doc/help.txt can
just be included into the source tarball - they will not be
regenerated if they already exist: Perl, man, and colcrt/col
are not required in this case.

Since I did not completely understand how the tarball is
being created, the actual inclusion into the tarball is
still missing.

The appended patch should be used *instead* of my previous one.

--- 1/Completion/Zsh/Command/_run-help
+++ 1/Completion/Zsh/Command/_run-help
@@ -0,0 +1,7 @@
+#compdef run-help
+local d expl
+local HELPDIR=${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}
+[[ -d $HELPDIR ]] && {
+	d=($HELPDIR/*(:t))
+	(($#d)) && _wanted commands expl 'command' compadd -a d
+} || _man
--- 1/Doc/Makefile.in
+++ 1/Doc/Makefile.in
@@ -33,7 +33,9 @@
 VPATH           = @srcdir@
 sdir            = @srcdir@
 sdir_top        = @top_srcdir@
+runhelpdir      = @runhelpdir@
 INSTALL         = @INSTALL@
+LN_S            = @LN_S@
 
 @DEFS_MK@
 
@@ -82,7 +84,7 @@
 
 # ========== DEPENDENCIES FOR BUILDING ==========
 
-all: man texi ../META-FAQ
+all: man runhelp texi ../META-FAQ
 .PHONY: all
 
 everything: all dvi html pdf info
@@ -183,6 +185,14 @@
 man: $(MAN)
 .PHONY: man
 
+runhelp: man
+	test x"$(runhelpdir)" = x"" || { \
+	    test -r $(sdir)/help.txt && test -d $(sdir)/help; \
+	} || perl $(sdir_top)/Util/helpfiles $(lc_ctype) -- \
+	    $(sdir)/zshbuiltins.1 $(sdir)/help $(sdir)/help.txt \
+	  || { rm -f $(sdir)/help.txt ; rm -rf $(sdir)/help ; false; }
+.PHONY: runhelp
+
 $(MAN): zmacros.yo zman.yo
 
 zsh.1 zshall.1: Zsh/intro.yo Zsh/metafaq.yo Zsh/invoke.yo Zsh/files.yo \
@@ -285,12 +295,12 @@
 
 # ========== DEPENDENCIES FOR INSTALLING ==========
 
-# install just installs the manual pages
-install: install.man
+# install just installs the manual and runhelp pages
+install: install.man install.runhelp
 .PHONY: install
 
-# uninstall just unistalls the manual pages
-uninstall: uninstall.man
+# uninstall just uninstalls the manual and runhelp pages
+uninstall: uninstall.man uninstall.runhelp
 .PHONY: uninstall
 
 # install man pages, creating install directory if necessary
@@ -302,6 +312,18 @@
 	done
 .PHONY: install.man
 
+# install runhelp pages, creating install directory if necessary
+install.runhelp: runhelp
+	if test x"$(runhelpdir)" != x""; then \
+	    ${SHELL} $(sdir_top)/mkinstalldirs $(DESTDIR)$(runhelpdir); \
+	    $(INSTALL_DATA) $(sdir)/help/* $(DESTDIR)$(runhelpdir); \
+	    while read from to; do \
+	        rm -f $(DESTDIR)$(runhelpdir)/$$to ; \
+	        $(LN_S) $$from $(DESTDIR)$(runhelpdir)/$$to; \
+	    done < $(sdir)/help.txt; \
+	fi
+.PHONY: install.runhelp
+
 # install info pages, creating install directory if necessary
 install.info: texi
 	${SHELL} $(sdir_top)/mkinstalldirs $(DESTDIR)$(infodir)
@@ -334,6 +356,11 @@
 	done
 .PHONY: uninstall.man
 
+# uninstall runhelp pages
+uninstall.runhelp:
+	test x"$(runhelpdir)" = x"" || rm -rf -- $(DESTDIR)$(runhelpdir)
+.PHONY: uninstall.runhelp
+
 # uninstall info pages
 uninstall.info:
 	rm -f $(DESTDIR)$(infodir)/$(tzsh).info
--- 1/Makefile.in
+++ 1/Makefile.in
@@ -63,8 +63,8 @@
 	$(MAKE) install STRIPFLAGS="-s"
 
 # install/uninstall most things
-install: install.bin install.modules install.fns install.man
-uninstall: uninstall.bin uninstall.modules uninstall.fns uninstall.man
+install: install.bin install.modules install.fns install.man install.runhelp
+uninstall: uninstall.bin uninstall.modules uninstall.fns uninstall.man uninstall.runhelp
 
 # install/uninstall just the binary
 install.bin uninstall.bin:
@@ -78,6 +78,10 @@
 install.man uninstall.man:
 	@cd Doc && $(MAKE) $(MAKEDEFS) $@
 
+# install/uninstall just the runhelp files
+install.runhelp uninstall.runhelp:
+	@cd Doc && $(MAKE) $(MAKEDEFS) $@
+
 # install/uninstall just the shell functions
 install.fns:
 	if test x$(fndir) != x && test x$(fndir) != xno; then \
--- 1/StartupFiles/zshrc
+++ 1/StartupFiles/zshrc
@@ -75,7 +75,7 @@
 # Some environment variables
 export MAIL=/var/spool/mail/$USERNAME
 export LESS=-cex3M
-export HELPDIR=/usr/local/lib/zsh/help  # directory for run-help function to find docs
+export HELPDIR=/usr/share/zsh/$ZSH_VERSION/help  # directory for run-help function to find docs
 
 MAILCHECK=300
 HISTSIZE=200
--- 1/Util/helpfiles
+++ 1/Util/helpfiles
@@ -2,8 +2,10 @@
 
 # helpfiles:  make help files for Z-shell builtins from the manual entries.
 
-# Create help files for zsh commands in the current directory;
-# assumes no other files are present.
+# Create help files for zsh commands for the manpage file of zshbuildins.1
+# passed as the first arg.
+# The second arg is the directory in which the help files will be created.
+# Assumes no other files are present.
 # No overwriting check;  `.' becomes `dot', `:' becomes `colon'.
 
 # Any command claiming to be `same as <foo>' or `equivalent to <foo>'
@@ -11,14 +13,11 @@
 # link to <foo>.  (Arguably the help file should be put at the start
 # instead.)
 
-# Takes one filename argument, or stdin: the zsh manual page as a plain
-# ascii file: `man zshbuiltins | colcrt -' (remember the -) should do
-# the trick.
-
-# If you don't have colcrt, try 'col -bx'.  The x is necessary so that
-# spaces don't turn into tabs, which messes up the calculations of
-# indentation on machines which randomly wrap lines round to the
-# previous line (so you see what we're up against).
+# If a third arg is given, the symlink is not created, but a
+# list of symlinks is put into the file specified by that arg.
+
+# Optionally, you can pass -fLC_CTYPE as an additional first argument.
+# If do that, this LC_CTYPE is used instead of one specified by a heuristic.
 
 # Example usage:
 #    cd ~/zsh-4.0.1				# or wherever
@@ -39,7 +38,82 @@
 #    }
 # now <Esc>-h works for shell builtins.
 
-while (<>) {
+sub Usage {
+    print(STDERR "Usage: helpfiles [-fLC_CTYPE] zshbuiltins.1 dest-dir [link-file]\n");
+    exit(1);
+}
+
+sub Info {
+    print('helpfiles: ', @_, "\n");
+}
+
+sub Die {
+    print(STDERR 'helpfiles: ', @_, "\n");
+    exit(1);
+}
+
+&Usage() unless(@ARGV);
+delete($ENV{'LC_ALL'});
+$ENV{'LANG'} = 'C';
+if($ARGV[0] =~ /-f(.*)/) {
+    $lc_type = $1;
+    shift(@ARGV);
+    &Usage() unless(@ARGV);
+    if($lc_type eq '') {
+        $lc_type = shift(@ARGV);
+        &Usage() unless(@ARGV);
+    }
+} else {
+    open(LOCALE, '-|', 'locale', '-a') || &Die('cannot execute locale -a');
+    $lc_ctype = '';
+    $choice = 0;
+    while(<LOCALE>) {
+        if(/en.*utf8/i) {
+            $lc_ctype = $_;
+            last;
+        } elsif(/utf8/i) {
+            $lc_ctype = $_;
+            $choice = 2;
+        } elsif(($choice < 1) && (/(en)|\./i)) {
+            $lc_ctype = $_;
+            $choice = 1;
+        }
+    }
+    close(LOCALE);
+}
+&Info("using LC_CTYPE=$lc_ctype");
+$ENV{'LC_CTYPE'} = $lc_ctype unless($lc_ctype eq '');
+shift(@ARGV) if($ARGV[0] eq '--');
+&Usage() unless(@ARGV);
+$manfile = shift(@ARGV);
+&Usage() unless(@ARGV);
+$destdir = shift(@ARGV);
+$linkfile = ((@ARGV) ? shift(@ARGV) : '');
+unless(-d $destdir) {
+    mkdir($destdir) || &Die("$destdir is not a directory and cannot be created");
+}
+
+delete($ENV{'MANPL'});
+delete($ENV{'MANROFFSEQ'});
+$ENV{'MANWIDTH'} = '80';
+$ENV{'GROFF_NO_SGR'} = ''; # We need "classical" formatting of man pages.
+
+unless(open(MANPAGE, '-|', "man $manfile | colcrt -")) {
+    close(MANPAGE);
+    open(MANPAGE, '-|', "man $manfile | col -bx")
+# The x is necessary so that spaces don't turn into tabs, which messes
+# up the calculations of indentation on machines which randomly wrap lines
+# round to the previous line (so you see what we're up against).
+     || &Die('can run neither "man | colcrt -" nor "man | col -bx"');
+}
+
+unless($linkfile eq '') {
+    open(LINKFILE, '>', $linkfile) || &Die("cannot open $linkfile for writing")
+}
+
+chdir($destdir) || &Die("cannot cd to $destdir");
+
+while (<MANPAGE>) {
     last if /^\s*SHELL BUILTIN COMMANDS/;
     /zshbuiltins/ && $zb++;
     last if ($zb && /^\s*DESCRIPTIONS/);
@@ -93,7 +167,7 @@
     print;
 }
 
-while (<>) { last unless /^\s*$/; }
+while (<MANPAGE>) { last unless /^\s*$/; }
 
 /^(\s+)(\S+)/;
 $indentstr = $1;
@@ -101,13 +175,13 @@
 &newcmd($2);
 print if $print;
 
-BUILTINS: while (<>) {
+BUILTINS: while (<MANPAGE>) {
     next if /^\w/;
 
     undef($undented);
     if (/^\s*$/ || ($undented = (/^(\s*)/  && length($1) < $indent))) {
 	$undented && &doprint($_);
-	while (defined($_ = <>) && /(^\w)|(^\s*$)/) {
+	while (defined($_ = <MANPAGE>) && /(^\w)|(^\s*$)/) {
 	    # NAME is the start of the next section when in zshall.
 	    # (Historical note: we used to exit on the page header,
 	    # but text from the old section can continue to the
@@ -118,11 +192,11 @@
 	}
         if (/^\s*Page/ || /^zsh.*\s\d$/) {
 	    do {
-		$_ = <>;
+		$_ = <MANPAGE>;
 	    } while (defined($_) && /^\s*$/);
 	    if (/^\s*ZSHBUILTINS/) {
 		do {
-		    $_ = <>;
+		    $_ = <MANPAGE>;
 		} while (defined($_) && /^\s*$/);
 	    }
 	}
@@ -132,7 +206,7 @@
 	    s/^\s*/ /;
 	    $defer = $_;
 	    do {
-		$_ = <>;
+		$_ = <MANPAGE>;
 	    } while (defined($_) && /^\s*$/);
 	    last unless defined($_);
 	}
@@ -157,6 +231,7 @@
 
 select STDOUT;
 close OUT;
+close MANPAGE;
 
 foreach $file (<*>) {
     open (IN, $file);
@@ -176,8 +251,16 @@
 
 	# Make this a link to that.
 	unlink $file;
-	symlink ($sameas, $file);
+	if($linkfile eq '') {
+	    symlink ($sameas, $file);
+	} else {
+	    print(LINKFILE "$sameas $file\n");
+	}
     }
 }
+close(LINKFILE) unless($linkfile eq '');
+
+# Make one sanity check
+&Die('not all files were properly generated') unless(-r 'zmodload');
 
 __END__
--- 1/configure.ac
+++ 1/configure.ac
@@ -216,11 +216,22 @@
   AC_DEFINE_UNQUOTED(GLOBAL_ZLOGOUT, "$zlogout")
 fi
 
+ifdef([runhelpdir],[undefine([runhelpdir])])dnl
+AC_ARG_ENABLE(runhelpdir,
+AC_HELP_STRING([--enable-runhelpdir=DIR], [the default directory for the run-help files]),
+[if test x"$enableval" = xno; then
+  runhelpdir=
+else
+  runhelpdir="$enableval"
+fi], [runhelpdir="/usr/share/zsh/$VERSION/help"])
+
+
 AC_SUBST(zshenv)dnl
 AC_SUBST(zshrc)dnl
 AC_SUBST(zprofile)dnl
 AC_SUBST(zlogin)dnl
 AC_SUBST(zlogout)dnl
+AC_SUBST(runhelpdir)dnl
 
 dnl Do you want dynamically loaded binary modules.
 ifdef([dynamic],[undefine([dynamic])])dnl
@@ -559,6 +570,7 @@
 AC_PROG_INSTALL             dnl Check for BSD compatible `install'
 AC_PROG_AWK                 dnl Check for mawk,gawk,nawk, then awk.
 AC_PROG_LN                  dnl Check for working ln, for "make install"
+AC_PROG_LN_S                dnl Use ln -s or ln for "make install.runhelp"
 AC_PROG_EGREP               dnl sets $EGREP to grep -E or egrep
 AC_CHECK_PROGS([YODL], [yodl], [: yodl])
 

Re: Install run-help and *.zwc files system wide in build system

[Peter Stephenson]

On Mon, 11 Nov 2013 22:59:42 +0000 (UTC)
Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
> The appended patch should be used *instead* of my previous one.

Thanks.

> +AC_HELP_STRING([--enable-runhelpdir=DIR], [the default directory for the run-help files]),
> +[if test x"$enableval" = xno; then
> +  runhelpdir=
> +else
> +  runhelpdir="$enableval"
> +fi], [runhelpdir="/usr/share/zsh/$VERSION/help"])

It would make more sense for this to default to a similar location to
the function install path:

${datadir}/${tzsh_name}/'${VERSION}'/help

(I think it's defined that way so ${VERSION} is substituted in the
Makefile) which is already by default in a /share/ area.  This will
keep the help files in the same directory structure as the functions.

-- 
Peter Stephenson <p.stephenson@samsung.com>  Principal Software Engineer
Tel: +44 (0)1223 434724                Samsung Cambridge Solution Centre
St John's House, St John's Innovation Park, Cowley Road,
Cambridge, CB4 0DS, UK

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

Peter Stephenson <p.stephenson@samsung.com> wrote:
>
> It would make more sense for this to default to a similar location to
> the function install path

Thanks; apparently I did not read carefully enough through the
original configure.am. Of course, also the position of the new code
inside configure.am should probably be closer to the code you cited.

Meanwhile, I think that I understood the mechanism of zsh's tarball
creation. My understanding is that it is best to keep the new
Doc/help directory under version control (git/cvs), but of course
neither the Doc/help/* files themselves (except for some .* files)
nor Doc/help.txt

So my suggestion is to extend .gitignore, .cvsignore, and
.distfiles data correspondingly. Of course, the previous Makefile
rule for checking the existence of Doc/help makes no sense now if
this directory is under version control, and instead the existence
of a representative file inside it should be checked.

I append again the (now hopefully final) version of the full patch.
(Although the difference to the previous patch is small, I do not
consider it readable to post a patch for a patch.)

--- 1/.gitignore
+++ 1/.gitignore
@@ -30,6 +30,9 @@
 CVS
 .#*
 
+Doc/help.txt
+Doc/help/[_a-zA-Z0-9]*
+
 Doc/intro.a4.pdf
 Doc/intro.a4.ps
 Doc/intro.us.pdf
--- 1/Completion/Zsh/Command/_run-help
+++ 1/Completion/Zsh/Command/_run-help
@@ -0,0 +1,7 @@
+#compdef run-help
+local d expl
+local HELPDIR=${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}
+[[ -d $HELPDIR ]] && {
+	d=($HELPDIR/*(:t))
+	(($#d)) && _wanted commands expl 'command' compadd -a d
+} || _man
--- 1/Doc/.cvsignore
+++ 1/Doc/.cvsignore
@@ -1,4 +1,5 @@
 Makefile
+help.txt
 version.yo
 zsh*.1
 zsh.texi
--- 1/Doc/.distfiles
+++ 1/Doc/.distfiles
@@ -11,6 +11,7 @@
     zshcompctl.1 zshcompsys.1 zshcompwid.1 zshexpn.1
     zshmisc.1 zshmodules.1 zshoptions.1 zshparam.1 zshroadmap.1
     zshzle.1 zshall.1 zshzftpsys.1 zshcontrib.1 zshtcpsys.1
+    help.txt
 '
 
 DISTFILES_DOC='
--- 1/Doc/Makefile.in
+++ 1/Doc/Makefile.in
@@ -33,7 +33,9 @@
 VPATH           = @srcdir@
 sdir            = @srcdir@
 sdir_top        = @top_srcdir@
+runhelpdir      = @runhelpdir@
 INSTALL         = @INSTALL@
+LN_S            = @LN_S@
 
 @DEFS_MK@
 
@@ -82,7 +84,7 @@
 
 # ========== DEPENDENCIES FOR BUILDING ==========
 
-all: man texi ../META-FAQ
+all: man runhelp texi ../META-FAQ
 .PHONY: all
 
 everything: all dvi html pdf info
@@ -183,6 +185,14 @@
 man: $(MAN)
 .PHONY: man
 
+runhelp: man
+	test x"$(runhelpdir)" = x"" || { \
+	    test -r $(sdir)/help.txt && test -r $(sdir)/help/zmodload; \
+	} || perl $(sdir_top)/Util/helpfiles $(lc_ctype) -- \
+	    $(sdir)/zshbuiltins.1 $(sdir)/help $(sdir)/help.txt \
+	    || { rm -f $(sdir)/help.txt $(sdir)/help/zmodload; false; }
+.PHONY: runhelp
+
 $(MAN): zmacros.yo zman.yo
 
 zsh.1 zshall.1: Zsh/intro.yo Zsh/metafaq.yo Zsh/invoke.yo Zsh/files.yo \
@@ -285,12 +295,12 @@
 
 # ========== DEPENDENCIES FOR INSTALLING ==========
 
-# install just installs the manual pages
-install: install.man
+# install just installs the manual and runhelp pages
+install: install.man install.runhelp
 .PHONY: install
 
-# uninstall just unistalls the manual pages
-uninstall: uninstall.man
+# uninstall just uninstalls the manual and runhelp pages
+uninstall: uninstall.man uninstall.runhelp
 .PHONY: uninstall
 
 # install man pages, creating install directory if necessary
@@ -302,6 +312,18 @@
 	done
 .PHONY: install.man
 
+# install runhelp pages, creating install directory if necessary
+install.runhelp: runhelp
+	if test x"$(runhelpdir)" != x""; then \
+	    ${SHELL} $(sdir_top)/mkinstalldirs $(DESTDIR)$(runhelpdir); \
+	    $(INSTALL_DATA) $(sdir)/help/* $(DESTDIR)$(runhelpdir); \
+	    while read from to; do \
+	        rm -f $(DESTDIR)$(runhelpdir)/$$to || : ; \
+	        $(LN_S) $$from $(DESTDIR)$(runhelpdir)/$$to; \
+	    done < $(sdir)/help.txt; \
+	fi
+.PHONY: install.runhelp
+
 # install info pages, creating install directory if necessary
 install.info: texi
 	${SHELL} $(sdir_top)/mkinstalldirs $(DESTDIR)$(infodir)
@@ -334,6 +356,11 @@
 	done
 .PHONY: uninstall.man
 
+# uninstall runhelp pages
+uninstall.runhelp:
+	test x"$(runhelpdir)" = x"" || rm -rf -- $(DESTDIR)$(runhelpdir)
+.PHONY: uninstall.runhelp
+
 # uninstall info pages
 uninstall.info:
 	rm -f $(DESTDIR)$(infodir)/$(tzsh).info
--- 1/Doc/help/.cvsignore
+++ 1/Doc/help/.cvsignore
@@ -0,0 +1 @@
+[_a-zA-Z0-9]*
--- 1/Doc/help/.distfiles
+++ 1/Doc/help/.distfiles
@@ -0,0 +1,5 @@
+DISTFILES_SRC='
+    .cvsignore
+    .distfiles
+    [_a-zA-Z0-9]*
+'
--- 1/Makefile.in
+++ 1/Makefile.in
@@ -63,8 +63,8 @@
 	$(MAKE) install STRIPFLAGS="-s"
 
 # install/uninstall most things
-install: install.bin install.modules install.fns install.man
-uninstall: uninstall.bin uninstall.modules uninstall.fns uninstall.man
+install: install.bin install.modules install.fns install.man install.runhelp
+uninstall: uninstall.bin uninstall.modules uninstall.fns uninstall.man uninstall.runhelp
 
 # install/uninstall just the binary
 install.bin uninstall.bin:
@@ -78,6 +78,10 @@
 install.man uninstall.man:
 	@cd Doc && $(MAKE) $(MAKEDEFS) $@
 
+# install/uninstall just the runhelp files
+install.runhelp uninstall.runhelp:
+	@cd Doc && $(MAKE) $(MAKEDEFS) $@
+
 # install/uninstall just the shell functions
 install.fns:
 	if test x$(fndir) != x && test x$(fndir) != xno; then \
--- 1/StartupFiles/zshrc
+++ 1/StartupFiles/zshrc
@@ -75,7 +75,7 @@
 # Some environment variables
 export MAIL=/var/spool/mail/$USERNAME
 export LESS=-cex3M
-export HELPDIR=/usr/local/lib/zsh/help  # directory for run-help function to find docs
+export HELPDIR=/usr/share/zsh/$ZSH_VERSION/help  # directory for run-help function to find docs
 
 MAILCHECK=300
 HISTSIZE=200
--- 1/Util/helpfiles
+++ 1/Util/helpfiles
@@ -2,8 +2,10 @@
 
 # helpfiles:  make help files for Z-shell builtins from the manual entries.
 
-# Create help files for zsh commands in the current directory;
-# assumes no other files are present.
+# Create help files for zsh commands for the manpage file of zshbuildins.1
+# passed as the first arg.
+# The second arg is the directory in which the help files will be created.
+# Assumes no other files are present.
 # No overwriting check;  `.' becomes `dot', `:' becomes `colon'.
 
 # Any command claiming to be `same as <foo>' or `equivalent to <foo>'
@@ -11,14 +13,11 @@
 # link to <foo>.  (Arguably the help file should be put at the start
 # instead.)
 
-# Takes one filename argument, or stdin: the zsh manual page as a plain
-# ascii file: `man zshbuiltins | colcrt -' (remember the -) should do
-# the trick.
-
-# If you don't have colcrt, try 'col -bx'.  The x is necessary so that
-# spaces don't turn into tabs, which messes up the calculations of
-# indentation on machines which randomly wrap lines round to the
-# previous line (so you see what we're up against).
+# If a third arg is given, the symlink is not created, but a
+# list of symlinks is put into the file specified by that arg.
+
+# Optionally, you can pass -fLC_CTYPE as an additional first argument.
+# If do that, this LC_CTYPE is used instead of one specified by a heuristic.
 
 # Example usage:
 #    cd ~/zsh-4.0.1				# or wherever
@@ -39,7 +38,82 @@
 #    }
 # now <Esc>-h works for shell builtins.
 
-while (<>) {
+sub Usage {
+    print(STDERR "Usage: helpfiles [-fLC_CTYPE] zshbuiltins.1 dest-dir [link-file]\n");
+    exit(1);
+}
+
+sub Info {
+    print('helpfiles: ', @_, "\n");
+}
+
+sub Die {
+    print(STDERR 'helpfiles: ', @_, "\n");
+    exit(1);
+}
+
+&Usage() unless(@ARGV);
+delete($ENV{'LC_ALL'});
+$ENV{'LANG'} = 'C';
+if($ARGV[0] =~ /-f(.*)/) {
+    $lc_type = $1;
+    shift(@ARGV);
+    &Usage() unless(@ARGV);
+    if($lc_type eq '') {
+        $lc_type = shift(@ARGV);
+        &Usage() unless(@ARGV);
+    }
+} else {
+    open(LOCALE, '-|', 'locale', '-a') || &Die('cannot execute locale -a');
+    $lc_ctype = '';
+    $choice = 0;
+    while(<LOCALE>) {
+        if(/en.*utf8/i) {
+            $lc_ctype = $_;
+            last;
+        } elsif(/utf8/i) {
+            $lc_ctype = $_;
+            $choice = 2;
+        } elsif(($choice < 1) && (/(en)|\./i)) {
+            $lc_ctype = $_;
+            $choice = 1;
+        }
+    }
+    close(LOCALE);
+}
+&Info("using LC_CTYPE=$lc_ctype");
+$ENV{'LC_CTYPE'} = $lc_ctype unless($lc_ctype eq '');
+shift(@ARGV) if($ARGV[0] eq '--');
+&Usage() unless(@ARGV);
+$manfile = shift(@ARGV);
+&Usage() unless(@ARGV);
+$destdir = shift(@ARGV);
+$linkfile = ((@ARGV) ? shift(@ARGV) : '');
+unless(-d $destdir) {
+    mkdir($destdir) || &Die("$destdir is not a directory and cannot be created");
+}
+
+delete($ENV{'MANPL'});
+delete($ENV{'MANROFFSEQ'});
+$ENV{'MANWIDTH'} = '80';
+$ENV{'GROFF_NO_SGR'} = ''; # We need "classical" formatting of man pages.
+
+unless(open(MANPAGE, '-|', "man $manfile | colcrt -")) {
+    close(MANPAGE);
+    open(MANPAGE, '-|', "man $manfile | col -bx")
+# The x is necessary so that spaces don't turn into tabs, which messes
+# up the calculations of indentation on machines which randomly wrap lines
+# round to the previous line (so you see what we're up against).
+     || &Die('can run neither "man | colcrt -" nor "man | col -bx"');
+}
+
+unless($linkfile eq '') {
+    open(LINKFILE, '>', $linkfile) || &Die("cannot open $linkfile for writing")
+}
+
+chdir($destdir) || &Die("cannot cd to $destdir");
+
+while (<MANPAGE>) {
     last if /^\s*SHELL BUILTIN COMMANDS/;
     /zshbuiltins/ && $zb++;
     last if ($zb && /^\s*DESCRIPTIONS/);
@@ -93,7 +167,7 @@
     print;
 }
 
-while (<>) { last unless /^\s*$/; }
+while (<MANPAGE>) { last unless /^\s*$/; }
 
 /^(\s+)(\S+)/;
 $indentstr = $1;
@@ -101,13 +175,13 @@
 &newcmd($2);
 print if $print;
 
-BUILTINS: while (<>) {
+BUILTINS: while (<MANPAGE>) {
     next if /^\w/;
 
     undef($undented);
     if (/^\s*$/ || ($undented = (/^(\s*)/  && length($1) < $indent))) {
 	$undented && &doprint($_);
-	while (defined($_ = <>) && /(^\w)|(^\s*$)/) {
+	while (defined($_ = <MANPAGE>) && /(^\w)|(^\s*$)/) {
 	    # NAME is the start of the next section when in zshall.
 	    # (Historical note: we used to exit on the page header,
 	    # but text from the old section can continue to the
@@ -118,11 +192,11 @@
 	}
         if (/^\s*Page/ || /^zsh.*\s\d$/) {
 	    do {
-		$_ = <>;
+		$_ = <MANPAGE>;
 	    } while (defined($_) && /^\s*$/);
 	    if (/^\s*ZSHBUILTINS/) {
 		do {
-		    $_ = <>;
+		    $_ = <MANPAGE>;
 		} while (defined($_) && /^\s*$/);
 	    }
 	}
@@ -132,7 +206,7 @@
 	    s/^\s*/ /;
 	    $defer = $_;
 	    do {
-		$_ = <>;
+		$_ = <MANPAGE>;
 	    } while (defined($_) && /^\s*$/);
 	    last unless defined($_);
 	}
@@ -157,6 +231,7 @@
 
 select STDOUT;
 close OUT;
+close MANPAGE;
 
 foreach $file (<*>) {
     open (IN, $file);
@@ -176,8 +251,16 @@
 
 	# Make this a link to that.
 	unlink $file;
-	symlink ($sameas, $file);
+	if($linkfile eq '') {
+	    symlink ($sameas, $file);
+	} else {
+	    print(LINKFILE "$sameas $file\n");
+	}
     }
 }
+close(LINKFILE) unless($linkfile eq '');
+
+# Make one sanity check
+&Die('not all files were properly generated') unless(-r 'zmodload');
 
 __END__
--- 1/configure.ac
+++ 1/configure.ac
@@ -258,6 +258,18 @@
 AC_HELP_STRING([--enable-ansi2knr], [translate source to K&R C before compiling]),
 [ansi2knr="$enableval"], [ansi2knr=default])
 
+ifdef([runhelpdir],[undefine([runhelpdir])])dnl
+AC_ARG_ENABLE(runhelpdir,
+AC_HELP_STRING([--enable-runhelpdir=DIR], [the directory in which to install run-help files]),
+[if test x"$enableval" = xno; then
+  runhelpdir=
+else
+  runhelpdir="$enableval"
+fi], [runhelpdir=yes])
+if test x"$runhelpdir" = xyes; then
+  runhelpdir=${datadir}/${tzsh_name}/'${VERSION}'/help
+fi
+
 ifdef([fndir],[undefine([fndir])])dnl
 AC_ARG_ENABLE(fndir,
 AC_HELP_STRING([--enable-fndir=DIR], [the directory in which to install functions]),
@@ -297,6 +309,7 @@
   additionalfpath="${enableval}"
 fi], [additionalfpath=""])
 
+AC_SUBST(runhelpdir)dnl
 AC_SUBST(additionalfpath)dnl
 AC_SUBST(fndir)dnl
 AC_SUBST(sitefndir)dnl
@@ -559,6 +572,7 @@
 AC_PROG_INSTALL             dnl Check for BSD compatible `install'
 AC_PROG_AWK                 dnl Check for mawk,gawk,nawk, then awk.
 AC_PROG_LN                  dnl Check for working ln, for "make install"
+AC_PROG_LN_S                dnl Use ln -s/ln/cp for "make install.runhelp"
 AC_PROG_EGREP               dnl sets $EGREP to grep -E or egrep
 AC_CHECK_PROGS([YODL], [yodl], [: yodl])
 

Re: Install run-help and *.zwc files system wide in build system

[Peter Stephenson]

On Tue, 12 Nov 2013 17:41:40 +0000 (UTC)
Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
> I append again the (now hopefully final) version of the full patch.
> (Although the difference to the previous patch is small, I do not
> consider it readable to post a patch for a patch.)

Yes, I've been there...

Thanks, I hope to get around dealing with this later on.

pws

Re: Install run-help and *.zwc files system wide in build system

[Peter Stephenson]

It's committed...  I expect it'll take some shaking down with people
running it and making sure it fails safe if it can't run perl, and so
on.  I will draw attention to it for 5.0.2-test-2.

In particular, I'm not convinced by these...

    open(LOCALE, '-|', 'locale', '-a') || &Die('cannot execute locale -a');

etc...  I don't think you find out if locale even failed to start (never
mind the exit status) until you wait for the forked process when you
close the file handle.  You get a success from the open as soon as it's
succesfully forked.

It would be good to make run-help into run-help.in, substitute the help
directory in it when config.status gets run, and install the result (and
also not install run-help.in).  Then nobody needs to set an environment
variable (but the environment variable will override the value in the
file).  I'll probably get around to it.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
>
> In particular, I'm not convinced by these...
>
>     open(LOCALE, '-|', 'locale', '-a') || &Die('cannot execute locale -a');

I just realize that this should better be 'locale -a' instead 'locale', '-a':
According to perl's manpage, the "list form" is only supported if the
underlying system has a real "fork()".

> etc...  I don't think you find out if locale even failed to start

You get an error if there is no "locale" in $PATH (tested on linux,
no matter whether list form is used or not). But you are right that
one should better also check the return value of "close".

However, evenif an error does not get catched, it is not a big problem:
Just the heuristics cannot be used, i.e. LC_CTYPE would not be modified.
(Maybe one should print a clearer message in this case than just
"Using LC_CTYPE=", e.g. the current value of LC_CTYPE which is actually used).

> It would be good to make run-help into run-help.in, substitute the help
> directory in it when config.status gets run, and install the result

If the script should be provided to the user (as it is currently) so
that he can create the help-directory wherever he wants later on,
it is not good to force a fixed destination directory.
Fixing environment variables like e.g. LC_CTYPE has the same problem,
because the available locales on the machine where zsh is compiled
typically differ from the available locales on the machine where
zsh is executed/installed.

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
> It's committed...

Thanks a lot. It seems that the new files
Doc/help/.cvsignore
Doc/help/.distfiles
are not under version control yet...

Re: Install run-help and *.zwc files system wide in build system

[Bart Schaefer]

On Nov 12,  9:52pm, Martin Vaeth wrote:
} Subject: Re: Install run-help and *.zwc files system wide in build system
}
} Peter Stephenson <p.w.stephenson@ntlworld.com> wrote:
} > It's committed...
} 
} Thanks a lot.

Was it intentional that this creates files Doc/help/{All,Options,Some},
or is that an accident of slightly-obsolete parsing in the old helpfiles
script?

Re: Install run-help and *.zwc files system wide in build system

[Bart Schaefer]

On Nov 12,  5:13pm, Bart Schaefer wrote:
}
} Was it intentional that this creates files Doc/help/{All,Options,Some},
} or is that an accident of slightly-obsolete parsing in the old helpfiles
} script?

The latter.  Util/helpfiles wasn't expecting the introductory text at the
beginning of the section, which was only added about a month ago.

The simplest thing is just to discard the introductory text (patch below).
The only other thing to do would be to add another heading after those
introductory paragraphs, and stash the text in between in some made-up
file name; already, Util/helpfiles anticipates a header "DESCRIPTIONS"
even though there isn't one.


diff --git a/Util/helpfiles b/Util/helpfiles
index 530bb15..ba1c50a 100755
--- a/Util/helpfiles
+++ b/Util/helpfiles
@@ -123,7 +123,7 @@ $print = 0;
 
 sub namesub {
     local($cmd) = shift;
-    if ($cmd =~ /^\w+$/) {
+    if ($cmd =~ /^\w*$/ && lc($cmd) eq $cmd) {
 	$cmd;
     } elsif ($cmd eq '.') {
 	'dot';
@@ -261,6 +261,6 @@ foreach $file (<*>) {
 close(LINKFILE) unless($linkfile eq '');
 
 # Make one sanity check
-&Die('not all files were properly generated') unless(-r 'zmodload');
+&Die('not all files were properly generated') unless(-r 'ztcp');
 
 __END__

Re: Install run-help and *.zwc files system wide in build system

[Peter Stephenson]

On Tue, 12 Nov 2013 21:42:55 +0000 (UTC)
Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
> > It would be good to make run-help into run-help.in, substitute the help
> > directory in it when config.status gets run, and install the result
> 
> If the script should be provided to the user (as it is currently) so
> that he can create the help-directory wherever he wants later on,
> it is not good to force a fixed destination directory.

That's not the point --- there's already a path hardwired into it as a
default if the environment variable isn't set:

local HELPDIR=${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}

That default should be the right one, not some arbitrary string that
might not point at anything at all.

pws

Re: Install run-help and *.zwc files system wide in build system

[Martin Vaeth]

Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Tue, 12 Nov 2013 21:42:55 +0000 (UTC)
> Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
>> > It would be good to make run-help into run-help.in, substitute the help
>> > directory in it when config.status gets run, and install the result

Now I understand what you mean (I had misread "run-help" for
"helpfiles", since you were talking about the latter just before).

Yes, indeed, I thought about doing this when writing the patch,
and IMHO the same thing should also happen to the _run-help
command completion.

However, I did not find out how _run-help.in could be
prevented from being installed: As far as I understood, just
anything in the Completion hierarchy gets installed, and
nothing else seems to use such a mechanism. So I thought
that it is some sort of philosophy that these files should
be identical on every system.

Re: Install run-help and *.zwc files system wide in build system

[Peter Stephenson]

On Wed, 13 Nov 2013 10:10:18 +0000 (UTC)
Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
> However, I did not find out how _run-help.in could be
> prevented from being installed: As far as I understood, just
> anything in the Completion hierarchy gets installed, and
> nothing else seems to use such a mechanism. So I thought
> that it is some sort of philosophy that these files should
> be identical on every system.

Yes, all functions get installed at the moment.  However, that's simply
a limitation: there's no intention that installed functions should be
identical on all systems if that's not useful, any more than
documentation, though we avoid any unnecessary variation.  I'll try to
find some time to see how easy it is to exclude files --- possibly just
doing ad hoc exclusions of suffixes like .in if that's the path of least
resistance.

pws

[PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Jun T.]

---
 Util/helpfiles | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

Locale names may be either xx_YY.UTF8 or xx_YY.UTF-8.

Also fixed a typo: lc_type --> lc_ctype


diff --git a/Util/helpfiles b/Util/helpfiles
index ba1c50a..3e3125e 100755
--- a/Util/helpfiles
+++ b/Util/helpfiles
@@ -56,11 +56,11 @@ sub Die {
 delete($ENV{'LC_ALL'});
 $ENV{'LANG'} = 'C';
 if($ARGV[0] =~ /-f(.*)/) {
-    $lc_type = $1;
+    $lc_ctype = $1;
     shift(@ARGV);
     &Usage() unless(@ARGV);
-    if($lc_type eq '') {
-        $lc_type = shift(@ARGV);
+    if($lc_ctype eq '') {
+        $lc_ctype = shift(@ARGV);
         &Usage() unless(@ARGV);
     }
 } else {
@@ -68,10 +68,10 @@ if($ARGV[0] =~ /-f(.*)/) {
     $lc_ctype = '';
     $choice = 0;
     while(<LOCALE>) {
-        if(/en.*utf8/i) {
+        if(/en.*utf-?8/i) {
             $lc_ctype = $_;
             last;
-        } elsif(/utf8/i) {
+        } elsif(/utf-?8/i) {
             $lc_ctype = $_;
             $choice = 2;
         } elsif(($choice < 1) && (/(en)|\./i)) {
-- 
1.8.3.4 (Apple Git-47)

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Jun T.]

In helpfiles, line 77 (either with or without my previous patch):

        } elsif(($choice < 1) && (/(en)|\./i)) {

the pattern matches everything containing either 'en' or '.'.
What is intended here?

Re: Install run-help and *.zwc files system wide in build system

[Bart Schaefer]

On Nov 13, 11:21am, Peter Stephenson wrote:
}
} On Wed, 13 Nov 2013 10:10:18 +0000 (UTC)
} Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
} > However, I did not find out how _run-help.in could be
} > prevented from being installed ...
} 
} ... I'll try to
} find some time to see how easy it is to exclude files --- possibly just
} doing ad hoc exclusions of suffixes like .in if that's the path of least
} resistance.

Wouldn't the path of least resistence be to put _run_help.in in another
directory and simply write it into the Completion tree from configure or
Makefile?

I'm having a bit of angst about how this interacts with having separate
build and source trees, a build feature which I use frequently.

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Martin Vaeth]

Jun T. <takimoto-j@kba.biglobe.ne.jp> wrote:
> In helpfiles, line 77 (either with or without my previous patch):
>
>         } elsif(($choice < 1) && (/(en)|\./i)) {
>
> the pattern matches everything containing either 'en' or '.'.
> What is intended here?

Yes, this is the last fallback if nothing with utf8 can be found.

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Bart Schaefer]

On Nov 13,  4:56pm, Martin Vaeth wrote:
} Subject: Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.
}
} Jun T. <takimoto-j@kba.biglobe.ne.jp> wrote:
} > In helpfiles, line 77 (either with or without my previous patch):
} >
} >         } elsif(($choice < 1) && (/(en)|\./i)) {
} >
} > the pattern matches everything containing either 'en' or '.'.
} > What is intended here?
} 
} Yes, this is the last fallback if nothing with utf8 can be found.

Are you sure you didn't mean (/(^en)|\./i) ?  It's really OK for the
"en" to appear anywhere?  And wouldn't you want to prefer something
that contains "en" to somethimg that was found previously but has "."?

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> Are you sure you didn't mean (/(^en)|\./i) ?  It's really OK for the
> "en" to appear anywhere?

There are so many "standards" for locale names that I really do not know
what is the best: For instance, for the similar "de" (which might
perhaps also be checked) there are besides de_* also fy_DE and hsb_DE on
a Debian installation at my institute (though I do not know what
they mean).
On the other hand, you are right that e.g. ca_ES@valencia
should perhaps not be matching... Perhaps something like
/((^|[^a-z])(en|de))|\./i
is better?
Anyhow, all these fallbacks are a poor choice anyway.

> And wouldn't you want to prefer something that contains "en"
> to somethimg that was found previously but has "."?

It is easy to add further distinctions with

} elsif(($choice < 2) && (/.../)) { # second-to-worst fallback
  $lc_ctype = $_;
  $choice = 2;
} elsif(($choice < 1) && (/.../)) { # worst fallback
  $lc_ctype = $_;
  $choice = 1;
}

(and increasing the previous assignments of $choice correspondingly:
The idea is that $choice is measuring the quality of the "best"
match so far).
However, I am not sure whether this really makes sense and what
is reasonable to put into /.../:
As mentioned, probably none of the fallbacks is ideal, anyway.

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Phil Pennock]

On 2013-11-13 at 18:28 +0000, Martin Vaeth wrote:
> There are so many "standards" for locale names that I really do not know
> what is the best: For instance, for the similar "de" (which might
> perhaps also be checked) there are besides de_* also fy_DE and hsb_DE on
> a Debian installation at my institute (though I do not know what
> they mean).

ISO 639 language code, followed by ISO 3166 region tag identifying a
regional dialect.

"en_US" is "English as spoken in the USA", "en_GB" is "English as spoken
in Britain".

"fy" is Frisian, as it spoken in the north-east of The Netherlands and
apparently also in Germany, per "fy_DE".

"hsb" appears to be "Upper Sorbian", per
<http://www.ethnologue.com/language/hsb>.

The POSIX locale stuff is not using the same separators as RFC5646 but
the same loose principles apply.

-Phil

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Martin Vaeth]

Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
>> perhaps also be checked) there are besides de_* also fy_DE and hsb_DE on
>> a Debian installation at my institute (though I do not know what
>> they mean).
>
> ISO 639 language code, followed by ISO 3166 region tag identifying a
> regional dialect.

Thanks for the information. That it is some sort of German dialect
spoken somewhere else was clear to me, that's why I think it *might*
be a possible fallback.
The more important question concerning the patch is whether it will
use utf8 or other character sets. I do not know how to get this
information (even less how to get it in a compatible way).
It is really a pity that the only "standard" locales (C and POSIX)
which a guaranteed do not produce correct files...

Anyway, I think this discussion becomes rather academic: It is hard
to believe that somebody generates e.g. fy_DE as utf8 on his system
but no other *.utf8 locales, even if theoretically possible.
So, concerning that patch, I still think it makes no sense to be
too careful about fallbacks. After all, if the wrong selection is
made for whatever reason, it can be overridden by the option resp. in
the makefile by setting a variable.
And whoever is generating the distribution tarball hopefully does
this on a system which has some *.utf8 locale.

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Phil Pennock]

On 2013-11-14 at 07:50 +0000, Martin Vaeth wrote:
> Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
> >> perhaps also be checked) there are besides de_* also fy_DE and hsb_DE on
> >> a Debian installation at my institute (though I do not know what
> >> they mean).
> >
> > ISO 639 language code, followed by ISO 3166 region tag identifying a
> > regional dialect.
> 
> Thanks for the information. That it is some sort of German dialect
> spoken somewhere else was clear to me, that's why I think it *might*
> be a possible fallback.

Back to the point being raised, which I was addressing without being too
direct, in an attempt to be a little tactful: Bart is right, the
language code does need to be anchored ("^en").  The fact that "de" can
appear in two places in the examples you cite does not mean it should be
_accepted_ in either; "de_" is "German", "_DE" is "Germany".

Looking more closely, I'm confused: why do we care which character sets
are available, and why are we not just setting `LC_ALL=C` during the
generation?  We're making plain-text files, not stored in a
charset-dependent hierarchy, from a source under our control which has a
limited set of possible outputs.  It seems the only changes likely to be
made, depending upon character set, relate to the various hyphen/dash
options (where it's highly likely we want to force HYPHEN-MINUS from
traditional ASCII, to be sure that options shown can be copy/pasted
without worrying about man/groff macro variants and what dash might have
been chosen).

The commit message and ChangeLog additions are rather short on details,
since they only talk about the change causing the helpfiles to be
generated and don't mention anything about the addition of locale
fiddling.

-Phil

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Martin Vaeth]

Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
>
> Looking more closely, I'm confused: why do we care which character sets
> are available, and why are we not just setting `LC_ALL=C` during the
> generation?

It does not work. (At least on my gentoo box here it is reproducible):

After repairing the $lc_type -> $lc_ctype typos, I get:

perl ./helpfiles -fC zshbuiltins 11
helpfiles: using LC_CTYPE=C
Error executing formatting or display command.
System command (cd "/usr/share/man" && (echo ".pl 11i"; \
/usr/bin/unxz -c '/usr/share/man/man1/zshbuiltins.1.xz') | \
/usr/bin/gtbl | /usr/bin/nroff -mandoc | \
/usr/bin/less -+F -LiMXsR --shift 5) exited with status 36096.
No manual entry for zshbuiltins
helpfiles: not all files were properly generated

Similarly with LC_ALL=C, of course.

Just changing LC_CTYPE to some utf8 aware locale fixed it.
Since this is inherent to man/colcrt/less/whatever is implicitly
involved in this pipe, there is not much we can do except setting
a working LC_CTYPE.

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Jun T.]

I tested several locales and found that the only difference in the
generated help docs is the character used for hyphenation (split a word
at the end of a line); ASCII minus (0x2D) if C locale is used, and 
Unicode HYPHEN U+2010 (UTF-8: E2 80 90) if any of UTF-8 locales is used.

LC_CTYPE=C works at least on my Fedora/Debian/Mac OS X, and I think it
is the best and simplest way to go. Having Unicode HYPHEN in otherwise
pure ASCII files may just cause trouble rather than benefit.


2013/11/14 20:11、Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
> System command (cd "/usr/share/man" && (echo ".pl 11i"; \
> /usr/bin/unxz -c '/usr/share/man/man1/zshbuiltins.1.xz') | \
> /usr/bin/gtbl | /usr/bin/nroff -mandoc | \
> /usr/bin/less -+F -LiMXsR --shift 5) exited with status 36096.

Maybe colcrt died unexpectedly, and less got SIGPIPE?

Could you please try modifying the line 101 of helpfiles

unless(open(MANPAGE, '-|', "man $manfile | colcrt -")) {

to

..., "man $manfile 2>man.log | colcrt - 2>colcrt.log")) {

or even to

…, "man $manfile 2>man.log | tee man.out | colcrt - 2>colcrt.log")) {

and see whether the logfiles (and man.out) contain any useful info.

The charset used by less can come from various places; environment
variables LESSCHASET, LESSCHARDEF and LC_CTYPE, and your $HOME/.lesskey
file (if any).

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Bart Schaefer]

On Nov 15, 11:58pm, Jun T. wrote:
}
} I tested several locales and found that the only difference in the
} generated help docs is the character used for hyphenation (split a word
} at the end of a line); ASCII minus (0x2D) if C locale is used, and 
} Unicode HYPHEN U+2010 (UTF-8: E2 80 90) if any of UTF-8 locales is used.

I get Unicode double-quotes (E2 80 98 and E2 80 99) as well, in my
generated help files with the new Util/helpfiles (CentOS).

Incidentally, Martin, what's the logic/mnemonic behind using "-f" as the
command line switch for setting the character set?

# Optionally, you can pass -fLC_CTYPE as an additional first argument.
# If do that, this LC_CTYPE is used instead of one specified by a heuristic.

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Martin Vaeth]

Sorry for the delay; I am rather busy these days.

Jun T. <takimoto-j@kba.biglobe.ne.jp> wrote:
>
> variables LESSCHASET, LESSCHARDEF and LC_CTYPE, and your $HOME/.lesskey

Got it!  LESSCHARSET=utf-8 was set here which was the culprit.
So LANG=C seems to work if all LESS* and LC_* variables
are reset.  I attach the patch with the simplification
(note that also the call in the makefile must be changed).
Also, I added a check when closing the pipeline.

--- 1/Doc/Makefile.in
+++ 1/Doc/Makefile.in
@@ -188,7 +188,7 @@
 runhelp: man
 	test x"$(runhelpdir)" = x"" || { \
 	    test -r $(sdir)/help.txt && test -r $(sdir)/help/zmodload; \
-	} || perl $(sdir_top)/Util/helpfiles $(lc_ctype) -- \
+	} || perl $(sdir_top)/Util/helpfiles \
 	    $(sdir)/zshbuiltins.1 $(sdir)/help $(sdir)/help.txt \
 	    || { rm -f $(sdir)/help.txt $(sdir)/help/zmodload; false; }
 .PHONY: runhelp
--- 1/Util/helpfiles
+++ 1/Util/helpfiles
@@ -16,9 +16,6 @@
 # If a third arg is given, the symlink is not created, but a
 # list of symlinks is put into the file specified by that arg.
 
-# Optionally, you can pass -fLC_CTYPE as an additional first argument.
-# If do that, this LC_CTYPE is used instead of one specified by a heuristic.
-
 # Example usage:
 #    cd ~/zsh-4.0.1				# or wherever
 #    mkdir Help
@@ -39,7 +36,7 @@
 # now <Esc>-h works for shell builtins.
 
 sub Usage {
-    print(STDERR "Usage: helpfiles [-fLC_CTYPE] zshbuiltins.1 dest-dir [link-file]\n");
+    print(STDERR "Usage: helpfiles zshbuiltins.1 dest-dir [link-file]\n");
     exit(1);
 }
 
@@ -53,38 +50,6 @@
 }
 
 &Usage() unless(@ARGV);
-delete($ENV{'LC_ALL'});
-$ENV{'LANG'} = 'C';
-if($ARGV[0] =~ /-f(.*)/) {
-    $lc_type = $1;
-    shift(@ARGV);
-    &Usage() unless(@ARGV);
-    if($lc_type eq '') {
-        $lc_type = shift(@ARGV);
-        &Usage() unless(@ARGV);
-    }
-} else {
-    open(LOCALE, '-|', 'locale', '-a') || &Die('cannot execute locale -a');
-    $lc_ctype = '';
-    $choice = 0;
-    while(<LOCALE>) {
-        if(/en.*utf8/i) {
-            $lc_ctype = $_;
-            last;
-        } elsif(/utf8/i) {
-            $lc_ctype = $_;
-            $choice = 2;
-        } elsif(($choice < 1) && (/(en)|\./i)) {
-            $lc_ctype = $_;
-            $choice = 1;
-        }
-    }
-    close(LOCALE);
-}
-&Info("using LC_CTYPE=$lc_ctype");
-$ENV{'LC_CTYPE'} = $lc_ctype unless($lc_ctype eq '');
-shift(@ARGV) if($ARGV[0] eq '--');
-&Usage() unless(@ARGV);
 $manfile = shift(@ARGV);
 &Usage() unless(@ARGV);
 $destdir = shift(@ARGV);
@@ -93,8 +58,10 @@
     mkdir($destdir) || &Die("$destdir is not a directory and cannot be created");
 }
 
-delete($ENV{'MANPL'});
-delete($ENV{'MANROFFSEQ'});
+foreach (keys %ENV) {
+	delete($ENV{$_}) if(/^((LC_)|(LESS)|(MAN))/);
+}
+$ENV{'LANG'} = 'C';
 $ENV{'MANWIDTH'} = '80';
 $ENV{'GROFF_NO_SGR'} = ''; # We need "classical" formatting of man pages.
 
@@ -231,7 +198,7 @@
 
 select STDOUT;
 close OUT;
-close MANPAGE;
+close(MANPAGE) || &Die('piping from man ', $manfile, ' failed')
 
 foreach $file (<*>) {
     open (IN, $file);

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Bart Schaefer]

On Nov 15,  7:58pm, Martin Vaeth wrote:
>
> So LANG=C seems to work if all LESS* and LC_* variables
> are reset.  I attach the patch with the simplification

Committed/pushed, with one tiny change:

> +close(MANPAGE) || &Die('piping from man ', $manfile, ' failed')

Added a semicolon at the end of that line.

modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Martin Vaeth]

This patch implements a mechanism which supports "source/" subdirs
in the function hierarchy:

The "source/" directories are included from installation, and for
source/*.in files corresponding sources are generated after
corresponding substitutions.
Currently, the only substitution made is for @runhelpdir@.

The patch uses the new mechanism to hardcode the configured
default of HELPDIR.

The patch also fixes minor issues related to the previous commits:

1. Since "helpdir" now tests for ztcp, also the Makefile.in
   now checks for the same file for consistency reasons.

2. The special symbols '.' and ':' are added to the _run-help completion

3. In the previous commits, the new files Doc/help/{.distfiles,.cvsignore}
   of the patch were not added to git; the patch produces them again.

Please do not forget the following things after applying the patch
and before commiting:

(a) rm Functions/Misc/run-help Completion/Zsh/Command/_run-help
    because patch will just create 0-size files from them

(b) chmod a+x Config/cleanfuncs.sh Config/createfuncs.sh
    not strictly necessary but for consistency

(c) Something like "git add --all ." so that the new files will really
    be added to git (so that 3. above does not happen again.)


Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> Wouldn't the path of least resistence be to put _run_help.in in another
> directory and simply write it into the Completion tree from configure or
> Makefile?
>
> I'm having a bit of angst about how this interacts with having separate
> build and source trees, a build feature which I use frequently.

I have not tested the latter, but unless I have used false variables
by mistake, it should work as good (or, more precisely: as bad)
as your suggested "path of least resistence":

In any case the Completion tree (which is a "source" directory) is
modified (the new files are created or removed, respectively).
This is not really optimal, since it would be preferrable that
no produced files are written to any "source" directory.
However, "fixing" this would need a completely modified approach
of copying the whole functions hierarchy to the build directory
which is not good, either. I do not want to suggest that a
fundamental change to the build system.

--- 1/.gitignore
+++ 1/.gitignore
@@ -25,6 +25,8 @@
 /stamp-h.in
 /autom4te.cache
 
+Completion/Zsh/Command/_run-help
+
 Config/defs.mk
 
 CVS
@@ -73,6 +75,8 @@
 Etc/FAQ
 Etc/FAQ.html
 
+Functions/Misc/run-help
+
 Src/*.epro
 Src/*.export
 Src/*.mdh
--- 1/Completion/Zsh/Command/.cvsignore
+++ 1/Completion/Zsh/Command/.cvsignore
@@ -0,0 +1 @@
+_run-help
--- 1/Completion/Zsh/Command/_run-help
+++ 1/Completion/Zsh/Command/_run-help
@@ -1,7 +0,0 @@
-#compdef run-help
-local d expl
-local HELPDIR=${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}
-[[ -d $HELPDIR ]] && {
-	d=($HELPDIR/*(:t))
-	(($#d)) && _wanted commands expl 'command' compadd -a d
-} || _man
--- 1/Completion/Zsh/Command/source/.distfiles
+++ 1/Completion/Zsh/Command/source/.distfiles
@@ -0,0 +1,4 @@
+DISTFILES_SRC='
+.distfiles
+_run-help.in
+'
--- 1/Completion/Zsh/Command/source/_run-help.in
+++ 1/Completion/Zsh/Command/source/_run-help.in
@@ -0,0 +1,7 @@
+#compdef run-help
+local d expl
+local HELPDIR=${HELPDIR:-@runhelpdir@}
+[[ -d $HELPDIR ]] && {
+	d=($HELPDIR/*(:t))
+	(($#d)) && d+=('.' ':') && _wanted commands expl 'command' compadd -a d
+} || _man
--- 1/Config/cleanfns.sh
+++ 1/Config/cleanfns.sh
@@ -0,0 +1,22 @@
+#!/bin/sh
+
+allfuncs="`grep ' functions=.' ${dir_top}/config.modules |
+  sed -e '/^#/d' -e '/ link=no/d' -e 's/^.* functions=//'`"
+
+allfuncs="`cd $sdir_top; echo ${allfuncs}`"
+
+for file in $allfuncs; do
+  if test -d $sdir_top/$file; then
+    case "$file" in
+      */CVS/*|*/source/*)
+        continue
+      ;;
+      */source)
+        for source in $sdir_top/$file/*.in; do
+          destfile=$sdir_top/`echo $source | sed -e 's%source/\([^/]*\)\.in$%\1%'`
+          rm -f "$destfile"
+        done
+      ;;
+    esac
+  fi
+done
--- 1/Config/createfns.sh
+++ 1/Config/createfns.sh
@@ -0,0 +1,28 @@
+#!/bin/sh
+
+test "x$runhelpdir" = x && runhelpdir='/usr/share/zsh/$ZSH_VERSION/help'
+
+allfuncs="`grep ' functions=.' ${dir_top}/config.modules |
+  sed -e '/^#/d' -e '/ link=no/d' -e 's/^.* functions=//'`"
+
+allfuncs="`cd $sdir_top; echo ${allfuncs}`"
+
+for file in $allfuncs; do
+  if test -d $sdir_top/$file; then
+    case "$file" in
+      */CVS/*|*/source/*)
+        continue
+      ;;
+      */source)
+        for source in $sdir_top/$file/*.in; do
+          destfile=$sdir_top/`echo $source | sed -e 's%source/\([^/]*\)\.in$%\1%'`
+          rm -f "$destfile"
+          sed -e "s'@runhelpdir@'${runhelpdir}'" "$sdir_top/$source" >"$destfile" || {
+            echo "sed failed for $destfile"
+            exit 1
+          }
+        done
+      ;;
+    esac
+  fi
+done
--- 1/Config/installfns.sh
+++ 1/Config/installfns.sh
@@ -15,7 +15,7 @@
 for file in $allfuncs; do
   if test -f $sdir_top/$file; then
     case "$file" in
-      */CVS/*) continue;;
+      */CVS/*|*/source|*/source/*) continue;;
     esac
     if test x$FUNCTIONS_SUBDIRS != x && test x$FUNCTIONS_SUBDIRS != xno; then
       case "$file" in
--- 1/Doc/Makefile.in
+++ 1/Doc/Makefile.in
@@ -187,10 +187,10 @@
 
 runhelp: man
 	test x"$(runhelpdir)" = x"" || { \
-	    test -r $(sdir)/help.txt && test -r $(sdir)/help/zmodload; \
+	    test -r $(sdir)/help.txt && test -r $(sdir)/help/ztcp; \
 	} || perl $(sdir_top)/Util/helpfiles \
 	    $(sdir)/zshbuiltins.1 $(sdir)/help $(sdir)/help.txt \
-	    || { rm -f $(sdir)/help.txt $(sdir)/help/zmodload; false; }
+	    || { rm -f $(sdir)/help.txt $(sdir)/help/ztcp; false; }
 .PHONY: runhelp
 
 $(MAN): zmacros.yo zman.yo
--- 1/Doc/help/.cvsignore
+++ 1/Doc/help/.cvsignore
@@ -0,0 +1 @@
+[_a-zA-Z0-9]*
--- 1/Doc/help/.distfiles
+++ 1/Doc/help/.distfiles
@@ -0,0 +1,4 @@
+DISTFILES_SRC='
+    .cvsignore .distfiles
+    [_a-zA-Z0-9]*
+'
--- 1/Functions/Misc/.cvsignore
+++ 1/Functions/Misc/.cvsignore
@@ -0,0 +1 @@
+run-help
--- 1/Functions/Misc/.distfiles
+++ 1/Functions/Misc/.distfiles
@@ -1,4 +1,5 @@
 DISTFILES_SRC='
+.cvsignore
 .distfiles
 add-zsh-hook
 allopt
@@ -12,7 +13,6 @@
 promptnl
 regexp-replace
 relative
-run-help
 run-help-git
 run-help-openssl
 run-help-p4
--- 1/Functions/Misc/run-help
+++ 1/Functions/Misc/run-help
@@ -1,126 +0,0 @@
-#!/bin/zsh
-#
-# Figure out where to get the best help, and get it.
-#
-# Install this function by placing it in your FPATH and then
-# adding to your .zshrc the lines:
-#	unalias run-help
-#	autoload -Uz run-help
-#
-
-emulate -RL zsh
-
-local HELPDIR="${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}"
-
-[[ $1 == "." ]] && 1="dot"
-[[ $1 == ":" ]] && 1="colon"
-
-# Check whether Util/helpfiles has been used to generate zsh help
-if [[ $# == 0 || $1 == "-l" ]]
-then
-    if [[ -d $HELPDIR ]]
-    then
-	echo "Here is a list of topics for which special help is available:"
-	echo ""
-	print -rc $HELPDIR/*(:t)
-    else
-	echo "There is no list of special help topics available at this time."
-    fi
-    return 0
-elif [[ -n "${HELPDIR:-}" && -r $HELPDIR/$1 && $1 != compctl ]]
-then
-    ${=PAGER:-more} $HELPDIR/$1
-    return $?
-fi
-
-# No zsh help; use "whence" to figure out where else we might look
-local what places noalias newline='
-'
-integer i=0 didman=0
-
-places=( "${(@f)$(builtin whence -va $1)}" )
-if [[ $places = *"not found"* && $1 != ${(Q)1} ]]; then
-  # Different when unquoted, so try stripping quotes.
-  places=( "${(@f)$(builtin whence -va ${(Q)1})}" )
-  if (( ${#places} )); then
-      set -- "${(Q)@}"
-  fi
-  # Quotation is significant to aliases, so suppress lookup.
-  noalias=1
-fi
-
-{
-while ((i++ < $#places))
-do
-    what=$places[$i]
-    [[ -n $noalias && $what = *" is an alias "* ]] && continue
-    builtin print -r $what
-    case $what in
-    (*( is an alias for (noglob|nocorrect))*)
-	[[ ${what[(w)7]:t} != ${what[(w)1]} ]] &&
-	  run_help_orig_cmd=${what[(w)1]} run-help ${what[(w)7]:t}
-	;;
-    (*( is an alias)*)
-	[[ ${what[(w)6]:t} != ${what[(w)1]} ]] &&
-	  run_help_orig_cmd=${what[(w)1]} run-help ${what[(w)6]:t}
-	;;
-    (*( is a * function))
-	case ${what[(w)1]} in
-	(comp*) man zshcompsys;;
-	(zf*) man zshftpsys;;
-	(run-help) man zshcontrib;;
-	(*) builtin functions ${what[(w)1]} | ${=PAGER:-more};;
-	esac;;
-    (*( is a * builtin))
-	case ${what[(w)1]} in
-	(compctl) man zshcompctl;;
-	(comp*) man zshcompwid;;
-	(bindkey|vared|zle) man zshzle;;
-	(*setopt) man zshoptions;;
-	(cap|getcap|setcap) ;&
-	(clone) ;&
-	(ln|mkdir|mv|rm|rmdir|sync) ;&
-	(sched) ;&
-	(echotc|echoti|sched|stat|zprof|zpty|zsocket|zstyle|ztcp) man zshmodules;;
-	(zftp) man zshftpsys;;
-	(*) man zshbuiltins;;
-	esac
-	;;
-    (*( is hashed to *))
-	man ${what[(w)-1]:t}
-	;;
-    (*( is a reserved word))
-	man zshmisc
-	;;
-    (*)
-	if ((! didman++))
-	then
-	    if whence "run-help-$1:t" >/dev/null
-	    then
-		local cmd_args
-		builtin getln cmd_args
-		builtin print -z "$cmd_args"
-		cmd_args=( ${(z)cmd_args} )
-		# Discard environment assignments, etc.
-		while [[ $cmd_args[1] != ${run_help_orig_cmd:-$1} ]]
-		do
-		    shift cmd_args || return 1
-		done
-		eval "run-help-$1:t ${(q@)cmd_args[2,-1]}"
-	    else
-		POSIXLY_CORRECT=1 man $@:t
-	    fi
-	fi
-	;;
-    esac
-    if ((i < $#places && ! didman))
-    then
-	builtin print -nP "%SPress any key for more help or q to quit%s"
-	builtin read -k what
-	[[ $what != $newline ]] && echo
-	[[ $what == [qQ] ]] && break
-    fi
-done
-} always {
-  unset run_help_orig_cmd
-}
--- 1/Functions/Misc/source/.distfiles
+++ 1/Functions/Misc/source/.distfiles
@@ -0,0 +1,4 @@
+DISTFILES_SRC='
+.distfiles
+run-help.in
+'
--- 1/Functions/Misc/source/run-help.in
+++ 1/Functions/Misc/source/run-help.in
@@ -0,0 +1,126 @@
+#!/bin/zsh
+#
+# Figure out where to get the best help, and get it.
+#
+# Install this function by placing it in your FPATH and then
+# adding to your .zshrc the lines:
+#	unalias run-help
+#	autoload -Uz run-help
+#
+
+emulate -RL zsh
+
+local HELPDIR="${HELPDIR:-@runhelpdir@}"
+
+[[ $1 == "." ]] && 1="dot"
+[[ $1 == ":" ]] && 1="colon"
+
+# Check whether Util/helpfiles has been used to generate zsh help
+if [[ $# == 0 || $1 == "-l" ]]
+then
+    if [[ -d $HELPDIR ]]
+    then
+	echo "Here is a list of topics for which special help is available:"
+	echo ""
+	print -rc $HELPDIR/*(:t)
+    else
+	echo "There is no list of special help topics available at this time."
+    fi
+    return 0
+elif [[ -n "${HELPDIR:-}" && -r $HELPDIR/$1 && $1 != compctl ]]
+then
+    ${=PAGER:-more} $HELPDIR/$1
+    return $?
+fi
+
+# No zsh help; use "whence" to figure out where else we might look
+local what places noalias newline='
+'
+integer i=0 didman=0
+
+places=( "${(@f)$(builtin whence -va $1)}" )
+if [[ $places = *"not found"* && $1 != ${(Q)1} ]]; then
+  # Different when unquoted, so try stripping quotes.
+  places=( "${(@f)$(builtin whence -va ${(Q)1})}" )
+  if (( ${#places} )); then
+      set -- "${(Q)@}"
+  fi
+  # Quotation is significant to aliases, so suppress lookup.
+  noalias=1
+fi
+
+{
+while ((i++ < $#places))
+do
+    what=$places[$i]
+    [[ -n $noalias && $what = *" is an alias "* ]] && continue
+    builtin print -r $what
+    case $what in
+    (*( is an alias for (noglob|nocorrect))*)
+	[[ ${what[(w)7]:t} != ${what[(w)1]} ]] &&
+	  run_help_orig_cmd=${what[(w)1]} run-help ${what[(w)7]:t}
+	;;
+    (*( is an alias)*)
+	[[ ${what[(w)6]:t} != ${what[(w)1]} ]] &&
+	  run_help_orig_cmd=${what[(w)1]} run-help ${what[(w)6]:t}
+	;;
+    (*( is a * function))
+	case ${what[(w)1]} in
+	(comp*) man zshcompsys;;
+	(zf*) man zshftpsys;;
+	(run-help) man zshcontrib;;
+	(*) builtin functions ${what[(w)1]} | ${=PAGER:-more};;
+	esac;;
+    (*( is a * builtin))
+	case ${what[(w)1]} in
+	(compctl) man zshcompctl;;
+	(comp*) man zshcompwid;;
+	(bindkey|vared|zle) man zshzle;;
+	(*setopt) man zshoptions;;
+	(cap|getcap|setcap) ;&
+	(clone) ;&
+	(ln|mkdir|mv|rm|rmdir|sync) ;&
+	(sched) ;&
+	(echotc|echoti|sched|stat|zprof|zpty|zsocket|zstyle|ztcp) man zshmodules;;
+	(zftp) man zshftpsys;;
+	(*) man zshbuiltins;;
+	esac
+	;;
+    (*( is hashed to *))
+	man ${what[(w)-1]:t}
+	;;
+    (*( is a reserved word))
+	man zshmisc
+	;;
+    (*)
+	if ((! didman++))
+	then
+	    if whence "run-help-$1:t" >/dev/null
+	    then
+		local cmd_args
+		builtin getln cmd_args
+		builtin print -z "$cmd_args"
+		cmd_args=( ${(z)cmd_args} )
+		# Discard environment assignments, etc.
+		while [[ $cmd_args[1] != ${run_help_orig_cmd:-$1} ]]
+		do
+		    shift cmd_args || return 1
+		done
+		eval "run-help-$1:t ${(q@)cmd_args[2,-1]}"
+	    else
+		POSIXLY_CORRECT=1 man $@:t
+	    fi
+	fi
+	;;
+    esac
+    if ((i < $#places && ! didman))
+    then
+	builtin print -nP "%SPress any key for more help or q to quit%s"
+	builtin read -k what
+	[[ $what != $newline ]] && echo
+	[[ $what == [qQ] ]] && break
+    fi
+done
+} always {
+  unset run_help_orig_cmd
+}
--- 1/Makefile.in
+++ 1/Makefile.in
@@ -34,6 +34,7 @@
 VPATH           = @srcdir@
 sdir            = @srcdir@
 sdir_top        = @top_srcdir@
+runhelpdir      = @runhelpdir@
 INSTALL         = @INSTALL@
 
 @DEFS_MK@
@@ -41,11 +42,19 @@
 # ========== DEPENDENCIES FOR BUILDING ==========
 
 # default target
-all: config.h config.modules
+all: config.h config.modules create.fns
 	@for subdir in Src Doc; do \
 	  (cd $$subdir && $(MAKE) $(MAKEDEFS) $@) || exit 1; \
 	done
 
+# create recursively the shell functions from source/*.in
+create.fns: config.modules
+	sdir_top="$(sdir_top)" fndir="$(fndir)" dir_top="$(dir_top)" \
+	scriptdir="$(scriptdir)" \
+	runhelpdir="$(runhelpdir)" \
+	$(SHELL) $(sdir_top)/Config/createfns.sh
+.PHONY: create.fns
+
 # prepare module configuration
 prep:
 	@cd Src && $(MAKE) $(MAKEDEFS) $@
@@ -69,18 +78,22 @@
 # install/uninstall just the binary
 install.bin uninstall.bin:
 	@cd Src && $(MAKE) $(MAKEDEFS) $@
+.PHONY: install.bin uninstall.bin
 
 # install/uninstall just the modules
 install.modules uninstall.modules:
 	@cd Src && $(MAKE) $(MAKEDEFS) $@
+.PHONY: install.modules uninstall.modules
 
 # install/uninstall just the man pages
 install.man uninstall.man:
 	@cd Doc && $(MAKE) $(MAKEDEFS) $@
+.PHONY: install.man uninstall.man
 
 # install/uninstall just the runhelp files
 install.runhelp uninstall.runhelp:
 	@cd Doc && $(MAKE) $(MAKEDEFS) $@
+.PHONY: install.runhelp uninstall.runhelp
 
 # install/uninstall just the shell functions
 install.fns:
@@ -96,6 +109,7 @@
 	  $(SHELL) $(sdir_top)/Config/installfns.sh || exit 1; \
 	fi; \
 	exit 0
+.PHONY: install.fns
 
 uninstall.fns:
 	if test x$(fndir) != x && test x$(fndir) != xno; then \
@@ -106,14 +120,17 @@
 	  $(SHELL) $(sdir_top)/Config/uninstallfns.sh || exit 1; \
 	fi; \
 	exit 0
+.PHONY: uninstall.fns
 
 # install/uninstall just the info pages
 install.info uninstall.info:
 	@cd Doc && $(MAKE) $(MAKEDEFS) $@
+.PHONY: install.info uninstall.info
 
 # install/uninstall just the HTML manual
 install.html uninstall.html:
 	@cd Doc && $(MAKE) $(MAKEDEFS) $@
+.PHONY: install.html uninstall.html
 
 # ========== DEPENDENCIES FOR TESTING ==========
 check test:
@@ -123,6 +140,11 @@
 
 @CLEAN_MK@
 
+clean-here: config.modules
+	sdir_top="$(sdir_top)" fndir="$(fndir)" dir_top="$(dir_top)" \
+	scriptdir="$(scriptdir)" \
+	$(SHELL) $(sdir_top)/Config/cleanfns.sh
+
 distclean-here:
 	rm -f Makefile config.h config.status config.log config.cache config.modules config.modules.sh stamp-h Config/defs.mk
 	rm -rf autom4te.cache

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Jun T.]

2013/11/16 14:51、Bart Schaefer <schaefer@brasslantern.com> wrote：

> Committed/pushed, with one tiny change:

The following (unessential) part has escaped from the commit:


diff --git a/Util/helpfiles b/Util/helpfiles
index 26cfb6f..ebd8bd1 100755
--- a/Util/helpfiles
+++ b/Util/helpfiles
@@ -16,9 +16,6 @@
 # If a third arg is given, the symlink is not created, but a
 # list of symlinks is put into the file specified by that arg.
 
-# Optionally, you can pass -fLC_CTYPE as an additional first argument.
-# If do that, this LC_CTYPE is used instead of one specified by a heuristic.
-
 # Example usage:
 #    cd ~/zsh-4.0.1				# or wherever
 #    mkdir Help
@@ -39,7 +36,7 @@
 # now <Esc>-h works for shell builtins.
 
 sub Usage {
-    print(STDERR "Usage: helpfiles [-fLC_CTYPE] zshbuiltins.1 dest-dir [link-file]\n");
+    print(STDERR "Usage: helpfiles zshbuiltins.1 dest-dir [link-file]\n");
     exit(1);
 }
 

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Bart Schaefer]

On Nov 17, 12:17am, Jun T. wrote:
}
} The following (unessential) part has escaped from the commit:

Thanks, I had a lot of trouble with that file for some reason (the patch
kept failing to apply).  Pushed now.

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Bart Schaefer]

On Nov 16, 12:14pm, Martin Vaeth wrote:
}
} This patch implements a mechanism which supports "source/" subdirs
} in the function hierarchy:

I'd rather these were not inside the tree like that.  There should be an
entirely separate location for them.

In fact, in thinking about it, furter, I believe we should maintain the
Completion/ and Functions/ trees as conceptually read-only at "make all"
time.   Any files like _run_help and run-help that need to be generated
from a .in file should be processed by "make install" (install.fns rule)
directly into the $(sitefndir) target, and should never appear in the
Zsh build tree.

Taking that as step further, I'm beginning to think the helpfiles should
also be generated by "make install" rather than being written to a tree
underneath Doc/.

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> Any files like _run_help and run-help that need to be generated
> from a .in file should be processed by "make install" (install.fns rule)
> directly into the $(sitefndir) target, and should never appear in the
> Zsh build tree.

This somehow goes against the concept of "make install" which should
only copy (and at most strip) already generated files: Note that it
can happen that "make install" is run with different privileges than
"make all" and thus should execute as few code as possible.

The really "clean" solution would be to copy the whole functions
hierarchy in the "make all" phase, doing the patches as required.
However, as already mentioned, this would require a rather fundamental
change in the whole build system.

> Taking that as step further, I'm beginning to think the helpfiles should
> also be generated by "make install" rather than being written to a tree
> underneath Doc/.

Running perl, man and friends in the install phase is even worse.
Moreover, I thought the idea is that the generated help-files should
be included in the tarball (like the man-pages). In this case,
the Doc/ directory (in which also the generated man-pages reside)
is certainly the correct location.

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Peter Stephenson]

On Sun, 17 Nov 2013 10:30:47 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Nov 16, 12:14pm, Martin Vaeth wrote:
> }
> } This patch implements a mechanism which supports "source/" subdirs
> } in the function hierarchy:
> 
> I'd rather these were not inside the tree like that.  There should be an
> entirely separate location for them.
> 
> In fact, in thinking about it, furter, I believe we should maintain the
> Completion/ and Functions/ trees as conceptually read-only at "make all"
> time.   Any files like _run_help and run-help that need to be generated
> from a .in file should be processed by "make install" (install.fns rule)
> directly into the $(sitefndir) target, and should never appear in the
> Zsh build tree.

I can see the potential confusion in having files that need processing
in the same tree as files that get installed verbatim, although I didn't
see it as necessarily a problem in practice, but the files should be
generated at make time, not install time, otherwise it's inconsistent
with the rest of the build system.  Up to now, installation has been a
straight copy and I don't see any point in changing that: it adds an
extra point of failure (other than those applicable to copying,
e.g. permissions), for one thing.  I'm not even sure the config.status
processing applicable to substitutions for .in files, which is what we
want here (and I don't see any point in inventing a new form of
processing), is designed to go straight to install directories.
 
> Taking that as step further, I'm beginning to think the helpfiles should
> also be generated by "make install" rather than being written to a tree
> underneath Doc/.

Again, this is inconsistent with other files that get installed,
and also (which is really part of the same thing) means we can't supply
them in the doc bundle with the other pre-generated documentation.

In both cases, where they get written is another matter.

We could have a parallel hierarchy Functions_pre for generated functions.

The doc bundle isn't supposed to be standalone, it's supposed to give
you additional files to install using the files from the normal build,
so I don't think that's a limitation.  So I'm not that fussed where the
generated helpfiles go in the first instance.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Bart Schaefer]

On Nov 17,  8:13pm, Peter Stephenson wrote:
}
} I can see the potential confusion in having files that need processing
} in the same tree as files that get installed verbatim, although I didn't
} see it as necessarily a problem in practice, but the files should be
} generated at make time, not install time, otherwise it's inconsistent
} with the rest of the build system.

Hrm.  OK.  I guess I've just gotten too used to RPM-like systems where
even e.g. building a tarball is a "make install" into a dummy tree which
then gets packed up verbatim.  I don't see modifying a path in run-help
as that much different from skipping the installation of Completion/Cygwin
on a Darwin host [which admittedly we currently do not have a simple
option to accomplish].

} We could have a parallel hierarchy Functions_pre for generated functions.

Do we really expect there to be very many of these?  We've gone twenty
years without having a single one of them until now.  Is an entire
parallel hierarchy necessary?  Completion_pre as well?

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Bart Schaefer]

On Nov 17,  7:42pm, Martin Vaeth wrote:
}
} Moreover, I thought the idea is that the generated help-files should
} be included in the tarball (like the man-pages). In this case,
} the Doc/ directory (in which also the generated man-pages reside)
} is certainly the correct location.

I'm looking at, for example, Doc/Makefile.in "install.info" which
creates a local directory named "infodir" where it builds tzsh.texi
and then installs into the target location and removes infodir again
when it's finished.  Why is generating helfiles from the pre-built
manual pages different than this?

In any case "make clean" should remove Doc/help.

BTW, at some point I've ended up with a "Doc/Doc/help" directory as well
as a "Doc/help" directory.  Doc/Doc/help has .cvsignore and .gitignore,
Doc/help does not ... I'm not sure how this happened.

-- 
Barton E. Schaefer

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> BTW, at some point I've ended up with a "Doc/Doc/help" directory as well
> as a "Doc/help" directory.  Doc/Doc/help has .cvsignore and .gitignore,

.gitignore would be strange; I suppose that you mean .distfiles

> Doc/help does not ... I'm not sure how this happened.

I suppose that you have attempted to apply the patch when you were
in the Doc subdirectory: Since the files are created in the patch
out of nothing in the beginning of the patch, the patch utility would
do this without reporting a failure; only afterwards it would
report back when other files are not found.

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Martin Vaeth]

Independently of how this discussion turns out, the
previous patch I sent also contained some other fixes
which should be applied in any case:

The fix for _run-help, the improved sanity check for help/* creation,
and the 2 missing Doc/help/.* files.

Moreover, since currently _run-help is not treated as an
intermediate file, it should be added to .distfiles, since
otherwise it will not be in the tarball at all.

Finally, as mentioned in another post, code for cleaning Doc/help*
(when cleaning *.1 files) was missing.

The attached patch contains all these minor fixes.
Please be aware to remove the two Doc/help/.* files manually (if you
already created them locally) before applying the patch: otherwise the
content of these files will be duplicated after the patch.

Before committing to git, please do not forget to add these 2 Doc/help/.*
files...

--- 1/Completion/Zsh/Command/.distfiles
+++ 1/Completion/Zsh/Command/.distfiles
@@ -22,6 +22,7 @@
 _print
 _prompt
 _read
+_run-help
 _sched
 _set
 _setopt
--- 1/Completion/Zsh/Command/_run-help
+++ 1/Completion/Zsh/Command/_run-help
@@ -3,5 +3,5 @@
 local HELPDIR=${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}
 [[ -d $HELPDIR ]] && {
 	d=($HELPDIR/*(:t))
-	(($#d)) && _wanted commands expl 'command' compadd -a d
+	(($#d)) && d+=('.' ':') && _wanted commands expl 'command' compadd -a d
 } || _man
--- 1/Doc/Makefile.in
+++ 1/Doc/Makefile.in
@@ -187,10 +187,10 @@
 
 runhelp: man
 	test x"$(runhelpdir)" = x"" || { \
-	    test -r $(sdir)/help.txt && test -r $(sdir)/help/zmodload; \
+	    test -r $(sdir)/help.txt && test -r $(sdir)/help/ztcp; \
 	} || perl $(sdir_top)/Util/helpfiles \
 	    $(sdir)/zshbuiltins.1 $(sdir)/help $(sdir)/help.txt \
-	    || { rm -f $(sdir)/help.txt $(sdir)/help/zmodload; false; }
+	    || { rm -f $(sdir)/help.txt $(sdir)/help/*; false; }
 .PHONY: runhelp
 
 $(MAN): zmacros.yo zman.yo
@@ -396,6 +396,7 @@
 realclean-here: distclean-here
 	cd $(sdir) && rm -f Zsh/modlist.yo Zsh/modmenu.yo Zsh/manmodmenu.yo
 	cd $(sdir) && rm -f version.yo ../META-FAQ zsh.texi $(MAN)
+	cd $(sdir) && rm -f help.txt help/*
 .PHONY: realclean-here
 
 @CLEAN_MK@
--- 1/Doc/help/.cvsignore
+++ 1/Doc/help/.cvsignore
@@ -0,0 +1 @@
+[_a-zA-Z0-9]*
--- 1/Doc/help/.distfiles
+++ 1/Doc/help/.distfiles
@@ -0,0 +1,4 @@
+DISTFILES_SRC='
+    .cvsignore .distfiles
+    [_a-zA-Z0-9]*
+'

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Peter Stephenson]

On Sun, 17 Nov 2013 12:50:32 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> } We could have a parallel hierarchy Functions_pre for generated functions.
> 
> Do we really expect there to be very many of these?  We've gone twenty
> years without having a single one of them until now.  Is an entire
> parallel hierarchy necessary?  Completion_pre as well?

Well, if you don't want it in the normal Functions area, and it needs to
appear in the build tree for later installation, we're running out of
other possibilities.

pws

Doc/Doc/help (was: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system))

[Bart Schaefer]

On Nov 18,  7:00am, Martin Vaeth wrote:
} Subject: Re: modify functions hierarchy (was: Install run-help and *.zwc f
}
} > BTW, at some point I've ended up with a "Doc/Doc/help" directory as well
} > as a "Doc/help" directory.  Doc/Doc/help has .cvsignore and .gitignore,
} 
} .gitignore would be strange; I suppose that you mean .distfiles

Yes.

} > Doc/help does not ... I'm not sure how this happened.
} 
} I suppose that you have attempted to apply the patch when you were
} in the Doc subdirectory

No, I never attempted to apply the patch.  Here's what git says:

commit d3630bcd0c0a9314ff3f1201fc82449ee10be4cb
Author: Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de>
Date:   Tue Nov 12 17:41:40 2013 +0000

    31959: missed files

:100644 100644 6b4026d... 333c58a... M  .gitignore
:100644 100644 02e97b2... 1f5bf6e... M  ChangeLog
:000000 100644 0000000... 0751b25... A  Doc/Doc/help/.cvsignore
:000000 100644 0000000... 60962ad... A  Doc/Doc/help/.distfiles

Re: Doc/Doc/help (was: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system))

[Peter Stephenson]

On Mon, 18 Nov 2013 08:11:27 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> No, I never attempted to apply the patch.  Here's what git says:
> 
> commit d3630bcd0c0a9314ff3f1201fc82449ee10be4cb
> Author: Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de>
> Date:   Tue Nov 12 17:41:40 2013 +0000
> 
>     31959: missed files
> 
> :100644 100644 6b4026d... 333c58a... M  .gitignore
> :100644 100644 02e97b2... 1f5bf6e... M  ChangeLog
> :000000 100644 0000000... 0751b25... A  Doc/Doc/help/.cvsignore
> :000000 100644 0000000... 60962ad... A  Doc/Doc/help/.distfiles

I presume I applied the patch from the wrong directory.

pws

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Bart Schaefer]

On Nov 18,  7:57am, Martin Vaeth wrote:
}
} Independently of how this discussion turns out, the
} previous patch I sent also contained some other fixes

You should go ahead and commit 32008, I think.

Re: Doc/Doc/help (was: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system))

[Bart Schaefer]

On Nov 18,  4:23pm, Peter Stephenson wrote:
}
} > Author: Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de>
} > Date:   Tue Nov 12 17:41:40 2013 +0000
} 
} I presume I applied the patch from the wrong directory.

How did you get git to say the Author was Martin?

Re: Doc/Doc/help (was: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system))

[Peter Stephenson]

On Mon, 18 Nov 2013 08:31:10 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Nov 18,  4:23pm, Peter Stephenson wrote:
> }
> } > Author: Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de>
> } > Date:   Tue Nov 12 17:41:40 2013 +0000
> } 
> } I presume I applied the patch from the wrong directory.
> 
> How did you get git to say the Author was Martin?

--author is a standard option.  To be consistent I also use --date with
the date from the email, though that's arguable (particular since
I don't bother for my own commits).

pws

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Bart Schaefer]

On Nov 18, 10:32am, Peter Stephenson wrote:
}
} On Sun, 17 Nov 2013 12:50:32 -0800
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > } We could have a parallel hierarchy Functions_pre for generated functions.
} > 
} > Do we really expect there to be very many of these?  We've gone twenty
} > years without having a single one of them until now.  Is an entire
} > parallel hierarchy necessary?  Completion_pre as well?
} 
} Well, if you don't want it in the normal Functions area, and it needs to
} appear in the build tree for later installation, we're running out of
} other possibilities.

At the risk of re-opening this can of worms:

Another possibility is to handle it the way Doc/Makefile.in "install.info"
does -- I don't see how that situation is different from this one.

NOT patching these at install time means that you can't "make install"
with a different DESTDIR= than was in place at "make all" time.  *That*
seems wrong to me.

However, I was thinking Config/run-help.in and Config/_run-help.in or the
like when I grumbled about the parallel hierarchy.

Re: Doc/Doc/help (was: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system))

[Bart Schaefer]

On Nov 18,  4:36pm, Peter Stephenson wrote:
} Subject: Re: Doc/Doc/help (was: modify functions hierarchy (was: Install r
}
} > } > Author: Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de>
} > } > Date:   Tue Nov 12 17:41:40 2013 +0000
} 
} --author is a standard option.  To be consistent I also use --date with
} the date from the email, though that's arguable (particular since
} I don't bother for my own commits).

Aha.  I don't think this has been consistently done (I know I haven't
been doing it when I commit on behalf of someone else).

Furthermore it confused me into believing that Martin had been given
permission to do his own commits.  Is there a way to show the person
who actually changed the repository as well as / instead of the
asserted "author"?

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Peter Stephenson]

On Mon, 18 Nov 2013 08:38:28 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> Another possibility is to handle it the way Doc/Makefile.in "install.info"
> does -- I don't see how that situation is different from this one.

It doesn't, fundamentally.  But that means using an ad-hoc sed function
rather than standard config.status substitution.  There's no great
reason why we shouldn't use that here, if we've used it elsewhere.  I
suppose in that case you'd pick something that wasn't likely to be
mistaken and use "sed" on all the functions?

pws

Re: Doc/Doc/help (was: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system))

[Peter Stephenson]

On Mon, 18 Nov 2013 08:45:20 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:

> On Nov 18,  4:36pm, Peter Stephenson wrote:
> } Subject: Re: Doc/Doc/help (was: modify functions hierarchy (was: Install r
> }
> } > } > Author: Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de>
> } > } > Date:   Tue Nov 12 17:41:40 2013 +0000
> } 
> } --author is a standard option.  To be consistent I also use --date with
> } the date from the email, though that's arguable (particular since
> } I don't bother for my own commits).
> 
> Aha.  I don't think this has been consistently done (I know I haven't
> been doing it when I commit on behalf of someone else).
> 
> Furthermore it confused me into believing that Martin had been given
> permission to do his own commits.  Is there a way to show the person
> who actually changed the repository as well as / instead of the
> asserted "author"?

git log --format=full shows stuff like


commit d3630bcd0c0a9314ff3f1201fc82449ee10be4cb
Author: Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de>
Commit: Peter Stephenson <p.w.stephenson@ntlworld.com>

    31959: missed files


Anything mentioning <committer> in the git-log man page will give you
that level of detail, or you can make your own versions of the
formats.

pws

Re: [PATCH] helpfiles: Also accept 'UTF-8' as an encoding name.

[Phil Pennock]

On 2013-11-15 at 19:58 +0000, Martin Vaeth wrote:
> Got it!  LESSCHARSET=utf-8 was set here which was the culprit.
> So LANG=C seems to work if all LESS* and LC_* variables
> are reset.

Side-note: this highlights that we're reading the output after it has
passed through whichever pager was invoked by man(1), which indicates
there are still stability issues here, if the pager happens to be
something else, such as lv(1) influenced by $LV and a config variable.

> +foreach (keys %ENV) {
> +	delete($ENV{$_}) if(/^((LC_)|(LESS)|(MAN))/);
> +}

You might consider adding after this:

    $ENV{'PAGER'} = 'cat';

to bypass a class of problems here.  (I don't know of any man command
using $VIEWER instead of $PAGER, but perhaps better safe than sorry and
set that too.)

-Phil

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Peter Stephenson]

On Sun, 17 Nov 2013 13:01:18 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> I'm looking at, for example, Doc/Makefile.in "install.info" which
> creates a local directory named "infodir" where it builds tzsh.texi
> and then installs into the target location and removes infodir again
> when it's finished.  Why is generating helfiles from the pre-built
> manual pages different than this?

OK, is this good enough?

If it's roughly on lines no one objects to, I'll commit it and let
anyone else tie up the loose ends with other patches.  Otherwise, I'll
leave it to someone else to have yet a third go.

If anyone thinks there's anything else left to do (presumably just local
tweaks at this stage), please also supply a patch.

diff --git a/Config/defs.mk.in b/Config/defs.mk.in
index 58ef110..5c19cc4 100644
--- a/Config/defs.mk.in
+++ b/Config/defs.mk.in
@@ -51,6 +51,7 @@ sitefndir       = @sitefndir@
 scriptdir       = @scriptdir@
 sitescriptdir   = @sitescriptdir@
 htmldir         = @htmldir@
+runhelpdir      = @runhelpdir@
 
 # compilation
 CC              = @CC@
diff --git a/Config/installfns.sh b/Config/installfns.sh
index cf587c4..149f359 100755
--- a/Config/installfns.sh
+++ b/Config/installfns.sh
@@ -10,6 +10,8 @@ allfuncs="`grep ' functions=.' ${dir_top}/config.modules |
 
 allfuncs="`cd $sdir_top; echo ${allfuncs}`"
 
+test -d installfnsdir || mkdir installfnsdir
+
 # We now have a list of files, but we need to use `test -f' to check
 # (1) the glob got expanded (2) we are not looking at directories.
 for file in $allfuncs; do
@@ -44,8 +46,22 @@ for file in $allfuncs; do
         ;;
       esac
     fi
-    test -d $instdir || /bin/sh $sdir_top/mkinstalldirs $instdir || exit 1
-    $INSTALL_DATA $sdir_top/$file $instdir || exit 1
+    basename=`basename $file`
+    ok=0
+    if test -d $instdir || /bin/sh $sdir_top/mkinstalldirs $instdir; then
+      if sed "s|@runhelpdir@|$runhelpdir|" <$sdir_top/$file \
+        >installfnsdir/$basename; then
+	if $INSTALL_DATA installfnsdir/$basename $instdir; then
+	  ok=1
+	fi
+      fi
+    fi
+    case $ok in
+      0)
+      rm -rf installfnsdir
+      exit 1
+      ;;
+    esac
     read line < $sdir_top/$file
     case "$line" in
       '#!'*)
@@ -54,3 +70,5 @@ for file in $allfuncs; do
     esac
   fi
 done
+
+rm -rf installfnsdir
diff --git a/Doc/Makefile.in b/Doc/Makefile.in
index 5893cc8..eae3301 100644
--- a/Doc/Makefile.in
+++ b/Doc/Makefile.in
@@ -33,7 +33,6 @@ dir_top = ..
 VPATH           = @srcdir@
 sdir            = @srcdir@
 sdir_top        = @top_srcdir@
-runhelpdir      = @runhelpdir@
 INSTALL         = @INSTALL@
 LN_S            = @LN_S@
 
diff --git a/Functions/Misc/run-help b/Functions/Misc/run-help
index c817b67..e351dd6 100644
--- a/Functions/Misc/run-help
+++ b/Functions/Misc/run-help
@@ -10,7 +10,7 @@
 
 emulate -RL zsh
 
-local HELPDIR="${HELPDIR:-/usr/share/zsh/$ZSH_VERSION/help}"
+local HELPDIR="${HELPDIR:-@runhelpdir@}"
 
 [[ $1 == "." ]] && 1="dot"
 [[ $1 == ":" ]] && 1="colon"
diff --git a/Makefile.in b/Makefile.in
index d771033..cb74e94 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -88,7 +88,7 @@ install.fns:
 	  test x$(sitefndir) != xno && \
 	    $(SHELL) $(sdir_top)/mkinstalldirs $(DESTDIR)$(sitefndir); \
 	  sdir_top="$(sdir_top)" fndir="$(fndir)" dir_top="$(dir_top)" \
-	  scriptdir="$(scriptdir)" \
+	  scriptdir="$(scriptdir)" runhelpdir="$(runhelpdir)" \
 	  FUNCTIONS_SUBDIRS="$(FUNCTIONS_SUBDIRS)" \
 	  INSTALL_DATA="$(INSTALL_DATA)" \
 	  INSTALL_PROGRAM="$(INSTALL_PROGRAM)" \

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Bart Schaefer]

On Nov 20,  7:26pm, Peter Stephenson wrote:
} Subject: Re: modify functions hierarchy (was: Install run-help and *.zwc f
}
} On Sun, 17 Nov 2013 13:01:18 -0800
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > I'm looking at, for example, Doc/Makefile.in "install.info" which
} > creates a local directory named "infodir" where it builds tzsh.texi
} > and then installs into the target location and removes infodir again
} > when it's finished.  Why is generating helfiles from the pre-built
} > manual pages different than this?
} 
} OK, is this good enough?

Er, this patch doesn't actually have anything to do with generating the
helpfiles ... this is creating run-help and _run-help ...

I meant to reply to "... that means using an ad-hoc sed function ..."
back on Monday but didn't have time right then and never got back to it.
So now I feel a bit as though I'm making new mountains out of molehills.
Anyway ...

This patch is basically what I was thinking of, though it still seems
like overkill to put the entire tree of functions through the sed when
we know there are exactly two files (BTW your patch missed _run-help)
that are going to be affected.

E.g. "make install.info" applies the sed specifically to zsh.texi as
part of the makefile rule.  (On further examination I see this is only
to support AC_ARG_PROGRAM transformations from autoconf, which is not
exactly what I thought was going on, so the parallel may not be as
close as I previously thought.)

Nevertheless I what I was going to say on Monday and didn't get around
to, was:  run-help and _run-help should still get created by configure
(which I admit begs the question of where the .in files go) and then
the paths written by configure should be updated during the install
step, which implies using another file generated from a .in, which
implies rule code in Makefile.in rather than installfns.sh.

However, I'm now getting as tired of this as everyone else, so I am
fine with leaving this part as it unless/until I personally get time
to try to work out something else.

Re: modify functions hierarchy (was: Install run-help and *.zwc files system wide in build system)

[Peter Stephenson]

On Wed, 20 Nov 2013 22:01:00 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> Er, this patch doesn't actually have anything to do with generating the
> helpfiles ... this is creating run-help and _run-help ...

Generating helpfiles doesn't, so far as I can see, involve substituting
anything within the file, or this is there some further issue?

> This patch is basically what I was thinking of, though it still seems
> like overkill to put the entire tree of functions through the sed when
> we know there are exactly two files (BTW your patch missed _run-help)
> that are going to be affected.

OK, I've submitted it with the change to _run-help.  This looks like it
satisfies all the basic requirements.

Now I need to make 5.0.2-test-2 and draw attention to this addition.

pws

Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Bart Schaefer]

On Nov 23,  5:48pm, Peter Stephenson wrote:
} Subject: Re: modify functions hierarchy (was: Install run-help and *.zwc f
}
} On Wed, 20 Nov 2013 22:01:00 -0800
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > Er, this patch doesn't actually have anything to do with generating the
} > helpfiles ... this is creating run-help and _run-help ...
} 
} Generating helpfiles doesn't, so far as I can see, involve substituting
} anything within the file, or this is there some further issue?

No, it was just that the bit of my previous email which you excerpted
(and now removed) was about the helpfiles, so I was momentarily confused
when the patch that was appended was about something else.

Now that you mention it, though, I'm not entirely happy with the way the
dependencies for rebuilding the helpfiles are arranged.  It looks like
it's going to a lot of effort to avoid running Util/helpfiles if the
$(runhelpdir) is not defined, at the expense of possibly not rebuilding
the help files when zshbuiltins.1 has changed.  Why do we care at this
point whether $(runhelpdir) is defined?  Isn't the following better?

diff --git a/Doc/Makefile.in b/Doc/Makefile.in
index eae3301..8c45615 100644
--- a/Doc/Makefile.in
+++ b/Doc/Makefile.in
@@ -184,13 +184,14 @@ $(sdir)/zsh.texi: $(YODLSRC)
 man: $(MAN)
 .PHONY: man
 
-runhelp: man
-	test x"$(runhelpdir)" = x"" || { \
-	    test -r $(sdir)/help.txt && test -r $(sdir)/help/ztcp; \
-	} || perl $(sdir_top)/Util/helpfiles \
+runhelp: help.txt
+.PHONY: runhelp
+
+help.txt: zshbuiltins.1
+	@-rm -f $(sdir)/help.txt $(sdir)/help/*
+	perl $(sdir_top)/Util/helpfiles \
 	    $(sdir)/zshbuiltins.1 $(sdir)/help $(sdir)/help.txt \
 	    || { rm -f $(sdir)/help.txt $(sdir)/help/*; false; }
-.PHONY: runhelp
 
 $(MAN): zmacros.yo zman.yo
 

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> Why do we care at this point whether $(runhelpdir) is defined?

Because if the user does not want to install the helpfiles, he can say

  ./configure --disable-runhelpdir

and then he can build even if he has no tools like perl, man, groff, ...
which are required for the runhelp script.

Care has been taken in configure.am that the above call ends
with an empty $(runhelpdir).

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Bart Schaefer]

On Nov 23,  9:25pm, Martin Vaeth wrote:
} Subject: Re: Helpfiles again (was Re: modify functions hierarchy (was: etc
}
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} >
} > Why do we care at this point whether $(runhelpdir) is defined?
} 
} Because if the user does not want to install the helpfiles, he can say
} 
}   ./configure --disable-runhelpdir

That's not the way this ought to be done.  The absence of tools like
"ansi2knr" and "yodl" is not handled by obscuring the dependencies,
it's handled by checking for their availability in configure and
substituting no-op replacements (which is the reason for the existence
of e.g. the doc tarball), so the "make" targets still have the correct
dependency chain.

How about the following as a compromise.  When --disable-runhelpdir,
assure that $(runhelp) is also empty string, otherwise $(runhelp) is
"runhelp".  Then use $(runhelp) in dependency lists, so e.g. "make all"
has no indirect dependency on running perl etc., but the dependencies
for the "runhelp" target can be accurate.

(Could have done this by substitution of @runhelp@ in Doc/Makefile.in
but it seems more consistent to put it in Config/defs.mk.in and use
$(runhelp) instead.)

This isn't quite as good as what's done with yodl because it means the
doc tarball has to assure help.txt is newer than zshbuiltins.1.  On the
other hand, it can be prevented from griping the way the "texi" target
does when neither the [contents of the] doc tarball nor yodl itself is
available.


diff --git a/Config/defs.mk.in b/Config/defs.mk.in
index 5c19cc4..2c813a3 100644
--- a/Config/defs.mk.in
+++ b/Config/defs.mk.in
@@ -52,6 +52,7 @@ scriptdir       = @scriptdir@
 sitescriptdir   = @sitescriptdir@
 htmldir         = @htmldir@
 runhelpdir      = @runhelpdir@
+runhelp         = @runhelp@
 
 # compilation
 CC              = @CC@
diff --git a/Doc/Makefile.in b/Doc/Makefile.in
index eae3301..50e210f 100644
--- a/Doc/Makefile.in
+++ b/Doc/Makefile.in
@@ -83,7 +83,7 @@ Zsh/seealso.yo Zsh/tcpsys.yo Zsh/zftpsys.yo Zsh/zle.yo
 
 # ========== DEPENDENCIES FOR BUILDING ==========
 
-all: man runhelp texi ../META-FAQ
+all: man $(runhelp) texi ../META-FAQ
 .PHONY: all
 
 everything: all dvi html pdf info
@@ -184,13 +184,14 @@ $(sdir)/zsh.texi: $(YODLSRC)
 man: $(MAN)
 .PHONY: man
 
-runhelp: man
-	test x"$(runhelpdir)" = x"" || { \
-	    test -r $(sdir)/help.txt && test -r $(sdir)/help/ztcp; \
-	} || perl $(sdir_top)/Util/helpfiles \
+runhelp: help.txt
+.PHONY: runhelp
+
+help.txt: zshbuiltins.1
+	@-rm -f $(sdir)/help.txt $(sdir)/help/*
+	perl $(sdir_top)/Util/helpfiles \
 	    $(sdir)/zshbuiltins.1 $(sdir)/help $(sdir)/help.txt \
 	    || { rm -f $(sdir)/help.txt $(sdir)/help/*; false; }
-.PHONY: runhelp
 
 $(MAN): zmacros.yo zman.yo
 
@@ -312,7 +313,7 @@ install.man: man
 .PHONY: install.man
 
 # install runhelp pages, creating install directory if necessary
-install.runhelp: runhelp
+install.runhelp: $(runhelp)
 	if test x"$(runhelpdir)" != x""; then \
 	    ${SHELL} $(sdir_top)/mkinstalldirs $(DESTDIR)$(runhelpdir); \
 	    $(INSTALL_DATA) $(sdir)/help/* $(DESTDIR)$(runhelpdir); \
diff --git a/configure.ac b/configure.ac
index 607c612..5ca364b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -268,6 +268,9 @@ else
 fi], [runhelpdir=yes])
 if test x"$runhelpdir" = xyes; then
   runhelpdir=${datadir}/${tzsh_name}/'${VERSION}'/help
+  runhelp=runhelp
+else
+  runhelp=
 fi
 
 ifdef([fndir],[undefine([fndir])])dnl
@@ -310,6 +313,7 @@ else
 fi], [additionalfpath=""])
 
 AC_SUBST(runhelpdir)dnl
+AC_SUBST(runhelp)dnl
 AC_SUBST(additionalfpath)dnl
 AC_SUBST(fndir)dnl
 AC_SUBST(sitefndir)dnl

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Martin Vaeth]

Bart Schaefer <schaefer@brasslantern.com> wrote:

This will not work as expected:

> --- a/configure.ac
> +++ b/configure.ac
> @@ -268,6 +268,9 @@ else
>  fi], [runhelpdir=yes])
>  if test x"$runhelpdir" = xyes; then
>    runhelpdir=${datadir}/${tzsh_name}/'${VERSION}'/help
> +  runhelp=runhelp
> +else
> +  runhelp=
>  fi

If the user passes

./configure --with-runhelpdir=/prefix/usr/share/zsh/5.0.2/help

then $runhelpdir will not be "yes", but runhelp should
be defined anyway. Probably you want instead something like

if test x"$runhelpdir" = xyes; then
  runhelpdir=${datadir}/${tzsh_name}/'${VERSION}'/help
fi
if test x"$runhelpdir" = ""; then
  runhelp=runhelp
else
  runhelp=
fi

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Martin Vaeth]

Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
> if test x"$runhelpdir" = ""; then
>   runhelp=runhelp
> else
>   runhelp=
> fi

I meant the opposite, of course (and forgot x):

if test x"$runhelpdir" = x; then
  runhelp=
else
  runhelp=runhelp
fi

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Peter Stephenson]

Please let me know when you and Martin think this is all sorted out well enough
for me to make 5.0.3-test-2.  There seems little point in making a test
build for people to confirm the installation is OK if the installation
isn't quite pinned down yet.

Thanks
pws

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Bart Schaefer]

On Nov 24,  5:56pm, Peter Stephenson wrote:
}
} Please let me know when you and Martin think this is all sorted out

Sorry about that.  I've committed 32045 as updated by 32047, and will
forgo further makefile twiddling until after next release.

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Jun T.]

The following two files under Completion/ contain non-UTF-8 character:

(1) Completion/Unix/Command/_systemd: line 3, name of the author.
The character is an 'e' with accent (in latin1 encoding, i.e., ISO8859-1).

(2) Completion/BSD/Command/_portaudit: line 9, just before the '\' at
the end of line. The char is 0xA0 (non-breaking space in latin1 encoding?).

These characters cause the sed command (line 52 of Config/installfns.sh)
to die on Mac OS X if I run 'make install.fns' under UTF-8 locale.

# it seems sed on Mac OS X accepts only valid UTF-8 chars if run under
# UTF-8 locale

I believe the 0xA0 in (2) should be replaced with a simple space 0x20.

For (1), a possible workaround is to add
    export LC_CTYPE=C
at the beginning of Config/installfns.sh.

It seems the C locale just works fine on Mac (and does no harm on Linux).

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Peter Stephenson]

On Mon, 25 Nov 2013 23:24:36 +0900
"Jun T." <takimoto-j@kba.biglobe.ne.jp> wrote:
> The following two files under Completion/ contain non-UTF-8 character:
> 
> (1) Completion/Unix/Command/_systemd: line 3, name of the author.
> The character is an 'e' with accent (in latin1 encoding, i.e., ISO8859-1).

Hmmm.... might be safer to fudge it as "e'".  There's not really any
need for non-ASCII characters in functions.  (Or simply convert to
UTF-8 but I think the simpler the better.)

(I don't think I'm going to try to post this patch...)

> (2) Completion/BSD/Command/_portaudit: line 9, just before the '\' at
> the end of line. The char is 0xA0 (non-breaking space in latin1 encoding?).
> 
> I believe the 0xA0 in (2) should be replaced with a simple space 0x20.

Indeed.

I'll commit directly.

pws

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Bart Schaefer]

On Nov 25,  3:49pm, Peter Stephenson wrote:
}
} > (1) Completion/Unix/Command/_systemd: line 3, name of the author.
} > The character is an 'e' with accent (in latin1 encoding, i.e., ISO8859-1).
} 
} Hmmm.... might be safer to fudge it as "e'".  There's not really any
} need for non-ASCII characters in functions.  (Or simply convert to
} UTF-8 but I think the simpler the better.)

Output of "file **/*(.) | egrep 'UTF|ISO' :

ChangeLog:                                       UTF-8 Unicode English text
Completion/BSD/Command/_portaudit:               ISO-8859 English text
Completion/Unix/Command/_cdrdao:                 UTF-8 Unicode English text
Completion/Unix/Command/_git:                    UTF-8 Unicode English text
Completion/Unix/Command/_growisofs:              UTF-8 Unicode English text
Etc/ChangeLog-3.0:                               ISO-8859 English text
Etc/ChangeLog-4.1:                               UTF-8 Unicode English text
Etc/ChangeLog-4.3:                               UTF-8 Unicode English text
Etc/CONTRIBUTORS:                                UTF-8 Unicode English text
LICENCE:                                         ISO-8859 English text
Src/module.c:                                    ISO-8859 C program text
Src/Modules/clone.c:                             ISO-8859 C program text
Src/Modules/example.c:                           ISO-8859 C program text
Test/A05execution.ztst:                          ISO-8859 text
Test/D02glob.ztst:                               ISO-8859 text
Test/D07multibyte.ztst:                          UTF-8 Unicode English text
Test/V07pcre.ztst:                               UTF-8 Unicode English text

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Peter Stephenson]

On Mon, 25 Nov 2013 08:56:31 -0800
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Nov 25,  3:49pm, Peter Stephenson wrote:
> }
> } > (1) Completion/Unix/Command/_systemd: line 3, name of the author.
> } > The character is an 'e' with accent (in latin1 encoding, i.e., ISO8859-1).
> } 
> } Hmmm.... might be safer to fudge it as "e'".  There's not really any
> } need for non-ASCII characters in functions.  (Or simply convert to
> } UTF-8 but I think the simpler the better.)
> 
> Output of "file **/*(.) | egrep 'UTF|ISO' :
> 
> ChangeLog:                                       UTF-8 Unicode English text

I think that's OK.  If we don't need to process a file using a tool
that's worried about locales we get away with it.  I don't see any point
in forbidding UTF-8 in text files.

> Completion/BSD/Command/_portaudit:               ISO-8859 English text
> Completion/Unix/Command/_cdrdao:                 UTF-8 Unicode English text
> Completion/Unix/Command/_git:                    UTF-8 Unicode English text
> Completion/Unix/Command/_growisofs:              UTF-8 Unicode English text

Probably safest to make these ASCII; none of the non-ASCII characters
are actually needed.  The big one here is _git which has a lot of
non-ASCII quotes and "...", but they're not actually necessary, even for
readability (they're all quotes / ellipses with an ASCII lookalike), so
I don't think that's a real loss.

> Etc/ChangeLog-3.0:                               ISO-8859 English text
> LICENCE:                                         ISO-8859 English text
> Src/module.c:                                    ISO-8859 C program text
> Src/Modules/clone.c:                             ISO-8859 C program text
> Src/Modules/example.c:                           ISO-8859 C program text

Should be UTF-8 for consistency, I think.  We'd know by now if a
compiler cared about non-ASCII characters.

> Test/A05execution.ztst:                          ISO-8859 text
> Test/D02glob.ztst:                               ISO-8859 text
> Test/D07multibyte.ztst:                          UTF-8 Unicode English text
> Test/V07pcre.ztst:                               UTF-8 Unicode English text

These are all deliberate: either multibyte tests, or single bytes with
the top bit set, so I think they're OK.  Certainly they're being
explicitly tested.

Shout if you're unhappy or I'll do it tomorrow (UK time).

pws

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Phil Pennock]

On 2013-11-25 at 08:56 -0800, Bart Schaefer wrote:
> Output of "file **/*(.) | egrep 'UTF|ISO' :
[...]
> Test/D07multibyte.ztst:                          UTF-8 Unicode English text

That seems reasonable. ;)

> Test/V07pcre.ztst:                               UTF-8 Unicode English text

This one was me, and is again reasonable: it's the PCRE multibyte tests
and the %prep step looks for a UTF-8 locale to run the tests under,
bailing appropriately if none could be found.

There's a chained dependency here on multibyte working at all, since if
é doesn't glob match against ? then the PCRE tests will never be run.
At least the PCRE tests don't depend upon PCRE matches before they'll
consider themselves candidates to run.  ;)

-Phil

Re: Helpfiles again (was Re: modify functions hierarchy (was: etc.))

[Peter Stephenson]

On Mon, 25 Nov 2013 15:14:49 -0500
Phil Pennock <zsh-workers+phil.pennock@spodhuis.org> wrote:
> There's a chained dependency here on multibyte working at all, since if
> é doesn't glob match against ? then the PCRE tests will never be run.

Yes, there are various things like that, since the test framework is
running the same scripting language as you're trying to test.  I don't
think that's intrinsically problematic from the point of view of the
tests themselves, it's just a potential debug headache if something goes
wrong.  But we're used to those.

pws

PATCH: Util/helpfiles failing on old-fashioned unix

[pda]

I may be the only one generating documentation on a system without colcrt,
so this isn't too important in the grand scheme of things.  The first
line I've changed has three things wrong with it on HP-UX -- no colcrt
here, man can't be used with ./manfile.1, but most importantly the check
of perl's open doesn't seem right for a pipe.  This pipeline looks like
it could have an undetected error where man fails, but colcrt doesn't.

My perl-fu isn't the strongest, so this could very well do with further
tweaking... was there any reason for the close I removed?


--- a/Util/helpfiles
+++ b/Util/helpfiles
@@ -65,13 +65,12 @@ $ENV{'LANG'} = 'C';
 $ENV{'MANWIDTH'} = '80';
 $ENV{'GROFF_NO_SGR'} = ''; # We need "classical" formatting of man pages.
 
-unless(open(MANPAGE, '-|', "man $manfile | colcrt -")) {
-    close(MANPAGE);
-    open(MANPAGE, '-|', "man $manfile | col -bx")
+unless( undef == open(MANPAGE, '-|', "man $manfile | colcrt -")) {
+      ( undef == open(MANPAGE, '-|', "nroff -man $manfile | col -bx"))
 # The x is necessary so that spaces don't turn into tabs, which messes
 # up the calculations of indentation on machines which randomly wrap lines
 # round to the previous line (so you see what we're up against).
-     || &Die('can run neither "man | colcrt -" nor "man | col -bx"');
+        && &Die('can run neither "man | colcrt -" nor "nroff -man | col -bx"');
 }
 
 unless($linkfile eq '') {

Re: PATCH: Util/helpfiles failing on old-fashioned unix

[Martin Vaeth]

pda@sdf.org <pda@sdf.org> wrote:
> This pipeline looks like
> it could have an undetected error where man fails, but colcrt doesn't.

This is right, but I do not know a clean way how to solve this
problem in the lack of PIPESTATUS in posix - using known posix
tricks like piping error status to >&3 also has compatbility issues
(IIRC on cygwin).

A natural way is to start the man/nroff and colcrt/col processes
separately. Unfortunately, I do not know a way in perl to redirect
stdin *and* stdout of a subprocess simultaneously.
Perhaps the only clean way is to use a temporary file first for the
man/nroff output and then for the colcrc/col output.

Anyway, your patch only breaks usage on other systems:
The semantics of your patch is just a changed second open call and
that the error status of the first call to open is negated which
is certainly not intended:

> +unless( undef == open(MANPAGE, '-|', "man $manfile | colcrt -")) {
> +      ( undef == open(MANPAGE, '-|', "nroff -man $manfile | col -bx"))

The second open is executed if (and only if) the first open *succeeeds*.
This is certainly not intended.

Moreover, a clean patch should somehow attempt all 4 possible combinations
of the tools.

Re: PATCH: Util/helpfiles failing on old-fashioned unix

[Martin Vaeth]

Martin Vaeth <vaeth@mathematik.uni-wuerzburg.de> wrote:
>
> Perhaps the only clean way is to use a temporary file first for the
> man/nroff output and then for the colcrc/col output.

The following patch does this (avoiding the problematic pipeline):

--- 1/Util/helpfiles
+++ 1/Util/helpfiles
@@ -65,13 +65,40 @@
 $ENV{'MANWIDTH'} = '80';
 $ENV{'GROFF_NO_SGR'} = ''; # We need "classical" formatting of man pages.
 
-unless(open(MANPAGE, '-|', "man $manfile | colcrt -")) {
-    close(MANPAGE);
-    open(MANPAGE, '-|', "man $manfile | col -bx")
+$mantmp = $destdir . '/man.tmp';
+$coltmpbase = 'col.tmp';
+$coltmp = $destdir . '/' . $coltmpbase;
+$args = "$manfile >$mantmp";
+unlink($mantmp);
+&Info('attempting man ', $args);
+if(system('man ' . $args) || !(-s $mantmp)) {
+    unlink($mantmp);
+    &Info('attempting nroff -man ', $args);
+    if(system('nroff -man ' . $args) || !(-s $mantmp)) {
+        unlink($mantmp);
+        &Die('man and nroff -man both failed for ', $manfile);
+    }
+}
+$args = "$mantmp >$coltmp";
+unlink($coltmp);
+&Info('attempting colcrt ', $args);
+if(system('colcrt ' . $args) || !(-s $coltmp)) {
+    unlink($coltmp);
+    &Info('attempting col -bx <', $args);
 # The x is necessary so that spaces don't turn into tabs, which messes
 # up the calculations of indentation on machines which randomly wrap lines
 # round to the previous line (so you see what we're up against).
-     || &Die('can run neither "man | colcrt -" nor "man | col -bx"');
+    if(system('col -bx <' . $args) || !(-s $coltmp)) {
+        unlink($mantmp);
+        unlink($coltmp);
+        &Die('colcrt and col -bx both failed');
+    }
+}
+unlink($mantmp) || &Die('cannot remove tempfile ', $mantmp);
+
+unless(open(MANPAGE, '<', $coltmp)) {
+    unlink($coltmp);
+    &Die('generated tmpfile cannot be read');
 }
 
 unless($linkfile eq '') {
@@ -198,7 +225,8 @@
 
 select STDOUT;
 close OUT;
-close(MANPAGE) || &Die('piping from man ', $manfile, ' failed');
+close(MANPAGE);
+unlink($coltmpbase) || &Die('cannot remove tempfile ', $coltmpbase);
 
 foreach $file (<*>) {
     open (IN, $file);

Re: PATCH: Util/helpfiles failing on old-fashioned unix

[Wayne Davison]

[-- Attachment #1: Type: text/plain, Size: 416 bytes --]

On Fri, Dec 13, 2013 at 1:08 AM, Martin Vaeth <
vaeth@mathematik.uni-wuerzburg.de> wrote:

> Unfortunately, I do not know a way in perl to redirect stdin *and* stdout
> of a subprocess simultaneously.
>

One easy way is to use IPC::Open3 <http://perldoc.perl.org/IPC/Open3.html>,
which is a default library that comes with perl.  This lets you start a
subprocess with any desired stdin & stdout & stderr.

..wayne..

Re: PATCH: Util/helpfiles failing on old-fashioned unix

[Martin Vaeth]

Wayne Davison <wayned@users.sourceforge.net> wrote:
> vaeth@mathematik.uni-wuerzburg.de> wrote:
>
>> Unfortunately, I do not know a way in perl to redirect stdin *and* stdout
>> of a subprocess simultaneously.
>>
>
> One easy way is to use IPC::Open3 <http://perldoc.perl.org/IPC/Open3.html>,
> which is a default library that comes with perl.  This lets you start a
> subprocess with any desired stdin & stdout & stderr.

Thanks. However, from the avoidance of the perl5 keyword "my" in
Util/helpfiles (and falling back to e.g. local) I guess that this
script is supposed to run even on historical perl{3,4} variants
where libraries can hardly be expected.
Looking at the implementation of IPC::Open3, I see that everything
is setup manually using e.g. "pipe" and "fork". The latter,
unsurprisingly, is not implemented on all systems.

Summarizing, I think that the posted solution with tempfiles is
more compatible, since its "system" call should work even in
non-multitasking environments (and does not need a cumbersome
implementation to make it work with historical perl variants).
It was intentional that the implementation used no library
(neither File::Temp::tempfile to produce the temporary file nor
File::Spec::catfile to concatenate directory and filename).
I even refrained from quoting the directories (hence, spaces
in the "help" directory name would cause trouble) to not rely
too much on the underlying shell used by perl.

Re: PATCH: Util/helpfiles failing on old-fashioned unix

[Bart Schaefer]

On Dec 14,  9:23am, Martin Vaeth wrote:
}
} Summarizing, I think that the posted solution with tempfiles is
} more compatible, since its "system" call should work even in
} non-multitasking environments (and does not need a cumbersome
} implementation to make it work with historical perl variants).

I've committed the patch from 32105.

Re: PATCH: Util/helpfiles failing on old-fashioned unix

[pda]

On Fri, Dec 13, 2013 at 09:08:04AM +0000, Martin Vaeth wrote:
> pda@sdf.org <pda@sdf.org> wrote:
> A natural way is to start the man/nroff and colcrt/col processes
> separately. Unfortunately, I do not know a way in perl to redirect
> stdin *and* stdout of a subprocess simultaneously.
> Perhaps the only clean way is to use a temporary file first for the
> man/nroff output and then for the colcrc/col output.

Yes... although I wasn't really proposing to go to that much trouble,
since this scenario seems uncommon enough not to be worth it.  I see
you've gone ahead and done that, so kudos.

> Anyway, your patch only breaks usage on other systems:
> The semantics of your patch is just a changed second open call and
> that the error status of the first call to open is negated which
> is certainly not intended:

D'oh!  That was certainly poorly tested.  It kind of worked for me,
without me fully understanding why at the time.  Looking more closely
at how perl opens a pipe, I see it seems to always succeed, even when
the pipe components fail.  Looks like testing for emptiness of the
file handle would do the trick.

Re: PATCH: Util/helpfiles failing on old-fashioned unix

[Martin Vaeth]

pda@sdf.org <pda@sdf.org> wrote:
> Looking more closely
> at how perl opens a pipe, I see it seems to always succeed, even when
> the pipe components fail.

I suppose perl just checks the call of the shell (and whether
the redirection succeeeded) which means that, usually, an open with
a pipe redirection succeeds fully (including the "close") if
the *last* command of that pipe succeeded.

> Looks like testing for emptiness of the file handle would do the trick.

I doubt that perl has any means to get this information after a call
to the shell. Moreover, filehandles in perl are very abstract objects
(even more in perl{3,4}); you probably cannot get a useful string
representation of it which has any meaning. I recall that, in another
context, even with perl5 and its filehandle *variables* I did not find
a proper way to find out the "succeess" from the handle, only (and I
had tried some tricks with "ref" and "defined" but perl's
autovivification seems to make the handle "well-behaved" just because
it was *used* in a call to open - even if that call failed).

However, with the code I had attached all these problems should
not occur since it uses "system" only instead of "open".