RE: autoconf 2.5 (Re: PATCH: terminfo horor)

RE: autoconf 2.5 (Re: PATCH: terminfo horor)

[Borzenkov Andrey]

> 
> Oliver Kiddle wrote:
> > The main goal I had in mind was just for the thing to actually compile
> > on my machine without messy fiddling.
> 
> This will do fine for now.  Otherwise it's probably never going to happen.
> 

OK, in this case your patch was the right thing. I am still curious as to
why this change was done in the first place.

Regards

-andrey

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Peter Stephenson]

Oliver Kiddle wrote:
> The main goal I had in mind was just for the thing to actually compile
> on my machine without messy fiddling.

This will do fine for now.  Otherwise it's probably never going to happen.

pws


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited. 
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Oliver Kiddle]

On 9 Jan, Andrey wrote:

> before starting to hack around configure we probably need to define the
> final goal. For module build some options are

The main goal I had in mind was just for the thing to actually compile
on my machine without messy fiddling.

> - module source location (in-tree vs. off-tree)
> - module detection (pre-configure, configure, post-configure)
> - module configuration (single configure possibly built from different
> sources vs. per-module configure)
> - build time (in-tree vs. off-tree)

I'd also add that a significant goal should be to keep the end result
as simple as possible because it could easily become very complex.

> Supporting them off-tree facilitates independent modules development but I
> have a feeling it is unlikely to be an issue in the near future :)

A lot more would have to be changed too allow off-tree modules so it is
probably best kept in-tree.

> Having separate per-module configure has one problem of passing results of
> main configure testing down because 2.5 disables caching by default. I could
> not find suitable solution to this problem unless recent autoconf has
> changed it. Besides to really utilize autoconf features list of
> sub-configure has to be statically included in top-level script. Advantage
> it has is that user can just drop in module source and does not need to
> worry about having suitable autoconf version

Can we have module-specific m4 fragments included into the main
configure script?

> Having module detection in configure (as is done currently) prevents adding
> per-module configure checks (at least, using autoconf language).

Why? Does it really matter if we have redundant configure checks?
Surely we need the result of some configure tests to determine which
modules to build anyway.

> So overall I am inclined to the build process that
> 
> - assumes all modules in-tree
> - builds initial modules list as part of preconfig and adds rule to Makefile
> to recheck and rebuild it later
> - (re-)builds configure from snippets optionally supplied by modules
> - provides --with-modules and --with-static-modules (with wildcard support)
> to control which modules are included and which are built statically into
> zsh, eliminating any auto-generated file editing.

Controlling what modules are linked with configure --with options would
probably be a lot nicer that the current file editing. Perhaps with
separate options, e.g. --with-module-pcre or --with-module-pcre=dynamic.

Oliver

RE: autoconf 2.5 (Re: PATCH: terminfo horor)

[Borzenkov Andrey]

> Andrey wrote:
> >
> > right; but to do it we need make configure puts into config.status (or
> as
> > init part of running config.modules.sh) information that modules may
> need to
> > make decision. Using cache variables was big mistake to start with :(
> 
> Why do the decisions modules make need to be re-made when config.status
> runs? Can't we just evaluate decisions once, in configure, and store the
> answers in config.status?
> 

well, that's what it always did actually :)

before starting to hack around configure we probably need to define the
final goal. For module build some options are

- module source location (in-tree vs. off-tree)
- module detection (pre-configure, configure, post-configure)
- module configuration (single configure possibly built from different
sources vs. per-module configure)
- build time (in-tree vs. off-tree)

having modules always in tree makes it possible to automate module detection
completely; i.e. Makfefile can just always rescan for changed modules and
rebuild necessary files if list has changed.

Supporting them off-tree facilitates independent modules development but I
have a feeling it is unlikely to be an issue in the near future :)

Having separate per-module configure has one problem of passing results of
main configure testing down because 2.5 disables caching by default. I could
not find suitable solution to this problem unless recent autoconf has
changed it. Besides to really utilize autoconf features list of
sub-configure has to be statically included in top-level script. Advantage
it has is that user can just drop in module source and does not need to
worry about having suitable autoconf version

Having module detection in configure (as is done currently) prevents adding
per-module configure checks (at least, using autoconf language).

So overall I am inclined to the build process that

- assumes all modules in-tree
- builds initial modules list as part of preconfig and adds rule to Makefile
to recheck and rebuild it later
- (re-)builds configure from snippets optionally supplied by modules
- provides --with-modules and --with-static-modules (with wildcard support)
to control which modules are included and which are built statically into
zsh, eliminating any auto-generated file editing.

Comments?

-andrey

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Oliver Kiddle]

Andrey wrote:
> 
> right; but to do it we need make configure puts into config.status (or as
> init part of running config.modules.sh) information that modules may need to
> make decision. Using cache variables was big mistake to start with :(

Why do the decisions modules make need to be re-made when config.status
runs? Can't we just evaluate decisions once, in configure, and store the
answers in config.status?

Oliver

RE: autoconf 2.5 (Re: PATCH: terminfo horor)

[Borzenkov Andrey]

> 
> > - autoconf 2.5 does not create anything in configure. Instead the sole
> point
> > of configure script is to create config.status - and it is config.status
> > that finally does (should do) the job of creating files, making
> > substitutions etc
> 
> Taking that, what we should probably do is have configure not create
> any config.modules files but have it do the work of config.modules.sh,
> putting the results directly into config.status in such a way that
> config.status will create the config.modules file.
> 

right; but to do it we need make configure puts into config.status (or as
init part of running config.modules.sh) information that modules may need to
make decision. Using cache variables was big mistake to start with :(

that was one reason I thought about those config.snippets.

doing that also allows to simply put commands for every module in it's own
configure section using init commands that *do* depend on cache variables,
like

AC_CONFIG_COMMANDS([zsh/terminfo], [
link='if test x$have_tigetstr = xyes; then
          if test x$shared_tigetstr = xyes; then
              echo either
          else
              echo static
          fi
      else
          echo either;
      fi
'
echo name=zsh/terminfo modfile=zsh/terminfo.mdd link=$link auto=yes load=no
>> ${CONFIG_MODULES}], [
have_tigetstr=$ac_cv_func_tigetstr
shared_tigetstr=$zsh_cv_shared_tigetstr
CONFIG_MODULES=$CONFIG_MODULES
])

where the last argument does initialization.

This can't be generated dynamically because tag (first argument) must be
static :(

So the options currently are to either put all needed variables from all
modules into config.modules.sh or run it out of configure.


-andrey

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Peter Stephenson]

Oliver Kiddle wrote:
> On 25 Dec, Andrey wrote:
> 
> > - autoconf 2.5 does not create anything in configure. Instead the sole poin
> t
> > of configure script is to create config.status - and it is config.status
> > that finally does (should do) the job of creating files, making
> > substitutions etc
> 
> Taking that, what we should probably do is have configure not create
> any config.modules files but have it do the work of config.modules.sh,
> putting the results directly into config.status in such a way that
> config.status will create the config.modules file.

That sounds reasonable.  The ac_cv_* values used in the .mdd tests
can only change if configure is rerun, so the result of the tests
is fixed.  config.modules already claims you need to rerun
`config.status --recheck' when you add a new module.

> I suggest you go ahead with the rest of the patch. Except there's a
> typo (arlier) and you could also merge zshconfig.ac into configure.ac -
> there's no need for a separate file.

I've committed it, so it's time for everyone to upgrade to 2.50 or later
(the latest is 2.59).  I have not altered the offending (or inoffensive,
depending on your point of view) three lines.

Things we should do to smooth out the use of 2.50, apart from the
config.modules problem:

- Replace acconfig.h by chunks defined within configure.ac.
- Remove AC_CYGWIN which causes 2.59 to complain.  I think it's that
  simple --- all it does is rerun AC_CANONICAL_HOST and define
  the variable CYGWIN which we don't use.  We use the host_os, which
  is the right thing to do.
- Fix problems reported with sys/ptem.h and term.h by 2.59.  These are
  relatively minor --- we are already checking for usability and
  prerequisites for these files, we just need to do it in an order
  that causes configure not to scream.  (The message doesn't turn
  up on all systems.  It says that the file was found but couldn't
  be compiled.)
- Replace the chunks in configure.ac and Src/mkmakemod.sh which handle this:

    # The standard config.status requires the pathname for the .in file to
    # be relative to the top of the source tree.  As we have it in the build
    # tree, this is a problem.  zsh's configure script edits config.status,
    # adding the feature that an input filename starting with "!" has the
    # "!" removed and is not mangled further.

  with some suitable 2.50 feature.  I think it was rumoured there was
  such a feature.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited. 
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Oliver Kiddle]

On 25 Dec, Andrey wrote:

> - autoconf 2.5 does not create anything in configure. Instead the sole point
> of configure script is to create config.status - and it is config.status
> that finally does (should do) the job of creating files, making
> substitutions etc

Taking that, what we should probably do is have configure not create
any config.modules files but have it do the work of config.modules.sh,
putting the results directly into config.status in such a way that
config.status will create the config.modules file.

> What I intended was to allow modules to supply "configure plugins" to make
> whatever checks are needed; then configure could simply emit list modules to
> build without generating (mostly identical) makefile fragments for every
> module. Hmm ... it does look suspiciously similar to linux kernel, does not
> it? :)
> 
> But it is near to impossible to create such plugins as shell scripts
> (autoconf simply does not support it).
> 
> What is possible is to provide configure snippets and make Util/preconfig to
> generate modules.m4 that includes them' modules.m4 would then be included by
> zshconfig.ac. Unfortunately it is not easy to handle dependencies (i.e.
> module added or removed after modules.m4 has been created). 

I don't know much about how the linux kernel handles things. The idea
of having configure snippets that are included as part of zshconfig.ac
sounds good. Would allow us to separate out configure tests that are
specific to a module.

Peter wrote:
> Can someone confirm this is a suitable patch for removing support for
> old versions of autoconf, together with deleting configure.in which I
> will commit at the same time?

That does solve the problem I had because config.status no longer
writes out a crap config.modules. Only trouble is that, as Bart points
out, those lines are needed when someone runs just config.status.

I suggest you go ahead with the rest of the patch. Except there's a
typo (arlier) and you could also merge zshconfig.ac into configure.ac -
there's no need for a separate file.

Oliver

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Peter Stephenson]

Borzenkov Andrey wrote:
> we could try it once more if 2.13 compatibility is not an issue.

I'd certainly like to drop 2.13 support.  If that's going to make the
problem Oliver saw go away as a side effect, so much the better.

Anyone who has a chance is welcome to have a go at this.  We should be
able to replace custom header prototypes by extra arguments to configure
macros, too.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

RE: autoconf 2.5 (Re: PATCH: terminfo horor)

[Borzenkov Andrey]

> >
> > The configure script is creating config.modules twice. Second time
> > round when evaluating $link using the code in terminfo.mdd,
> > $ac_cv_func_tigetstr was unset. It seems the configure script runs
> > config.status. By running configure with --no-create, it was happy to
> > build terminfo.so. Switching from autoconf 2.57 to 2.13 also solved the
> > issue.
> >
> 
> looking in current zshconfig.ac it should not do it unless something else
> is
> buggy.
> 
> > configure cats config.modules.sh into config.status using
> > AC_OUTPUT_COMMANDS which is obsolete in autoconf 2.5. Using the new
> > AC_CONFIG_COMMANDS doesn't help because variables like
> > $ac_cv_func_tigetstr are still not set.
> >
> > Does anyone understand the real point of config.status? Why does
> > configure need to run it? I get the impression that it should contain
> > just a duplicate of what is in config.modules as created and recreate
> > it as opposed to rerunning scripts to derive its contents.
> >
> 
> I do not remember if it was me who put it in. Anyway
> 

it was Bart in 17659,17661; if config.modules.sh is really called twice it
means we have 2.5 that does define those variables it is not supposed to.

The problem is cache variables are not defined in config.status nor is
config.cache garanteed to exist (2.5 does not cache by default). Meaning we
cannot rely on them and have to add some status variable that is set by
configure.

What I intended was to allow modules to supply "configure plugins" to make
whatever checks are needed; then configure could simply emit list modules to
build without generating (mostly identical) makefile fragments for every
module. Hmm ... it does look suspiciously similar to linux kernel, does not
it? :)

But it is near to impossible to create such plugins as shell scripts
(autoconf simply does not support it).

What is possible is to provide configure snippets and make Util/preconfig to
generate modules.m4 that includes them' modules.m4 would then be included by
zshconfig.ac. Unfortunately it is not easy to handle dependencies (i.e.
module added or removed after modules.m4 has been created). 

-andrey

> - autoconf 2.13 creates output files as the final step in running
> configure.
> It is using environment variables CONFIG_FILES et al to pass information
> which files are to be created further down
> 
> - autoconf 2.5 does not create anything in configure. Instead the sole
> point
> of configure script is to create config.status - and it is config.status
> that finally does (should do) the job of creating files, making
> substitutions etc
> 
> apparently zshconfig.ac  tries to detect 2.5 by checking if
> $CONFIG_FILES$CONFIG_HEADERS is empty; if it is it dumps config.modules.sh
> into config.status. Hmm ... and runs it once more after that ... that
> looks
> strange.
> 
> > We really ought to ditch support for autoconf 2.13 in the 4.1 branch.
> > Not having a mixture will just make things simpler.
> >
> 
> yes. 2.5 makes many things simpler and more consistent. OTOH I remember I
> have tried to switch over to 2.5 way - doing job in config.status - and
> failed for some reasons (do not remember exactly) - I guess I attempted to
> move creation of all makefiles into config.status instead of creating
> makefiles during make run. Also it definitely was not portable across
> 2.13/2.5
> 
> we could try it once more if 2.13 compatibility is not an issue.
> 
> -andrey

RE: autoconf 2.5 (Re: PATCH: terminfo horor)

[Borzenkov Andrey]

> 
> The configure script is creating config.modules twice. Second time
> round when evaluating $link using the code in terminfo.mdd,
> $ac_cv_func_tigetstr was unset. It seems the configure script runs
> config.status. By running configure with --no-create, it was happy to
> build terminfo.so. Switching from autoconf 2.57 to 2.13 also solved the
> issue.
> 

looking in current zshconfig.ac it should not do it unless something else is
buggy.

> configure cats config.modules.sh into config.status using
> AC_OUTPUT_COMMANDS which is obsolete in autoconf 2.5. Using the new
> AC_CONFIG_COMMANDS doesn't help because variables like
> $ac_cv_func_tigetstr are still not set.
> 
> Does anyone understand the real point of config.status? Why does
> configure need to run it? I get the impression that it should contain
> just a duplicate of what is in config.modules as created and recreate
> it as opposed to rerunning scripts to derive its contents.
> 

I do not remember if it was me who put it in. Anyway

- autoconf 2.13 creates output files as the final step in running configure.
It is using environment variables CONFIG_FILES et al to pass information
which files are to be created further down

- autoconf 2.5 does not create anything in configure. Instead the sole point
of configure script is to create config.status - and it is config.status
that finally does (should do) the job of creating files, making
substitutions etc

apparently zshconfig.ac  tries to detect 2.5 by checking if
$CONFIG_FILES$CONFIG_HEADERS is empty; if it is it dumps config.modules.sh
into config.status. Hmm ... and runs it once more after that ... that looks
strange.

> We really ought to ditch support for autoconf 2.13 in the 4.1 branch.
> Not having a mixture will just make things simpler.
>

yes. 2.5 makes many things simpler and more consistent. OTOH I remember I
have tried to switch over to 2.5 way - doing job in config.status - and
failed for some reasons (do not remember exactly) - I guess I attempted to
move creation of all makefiles into config.status instead of creating
makefiles during make run. Also it definitely was not portable across
2.13/2.5

we could try it once more if 2.13 compatibility is not an issue.

-andrey

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Mads Martin Joergensen]

* Peter Stephenson <pws@csr.com> [Dec 19. 2003 12:14]:
> > Ok, I've tweaked the buildenvironment to not having ncurses-devel, and
> > without it:
> > 
> > gcc  -s -rdynamic -o zsh main.o  `cat stamp-modobjs`   -ldl -lnsl -lm -lc
> > utils.o(.text+0x2aa9): In function `gettempname':
> > : the use of `mktemp' is dangerous, better use `mkstemp'
> > init.o(.text+0x1277): In function `init_term':
> > : undefined reference to `tgetent'
> 
> Is termcap still available?  I can't see it in the link line, which is
> suspicious.  What did configure say when it was looking for tgetent
> and tigetflag (from config.log)?

Well, termcap is available, but detection fails because termcap.h is in
/usr/include/termcap/termcap.h and the .so and .a libs are in
/usr/{lib,lib64}/termcap/

So the termcap.h test should use -I $INCLUDE/termcap and the
conftest programs should use -L $LIBDIR/termcap as well.

-- 
Mads Martin Joergensen, http://mmj.dk
"Why make things difficult, when it is possible to make them cryptic
 and totally illogical, with just a little bit more effort?"
                                -- A. P. J.

Re: autoconf 2.5 (Re: PATCH: terminfo horor)

[Peter Stephenson]

Oliver Kiddle wrote:
> This line of the patch looks suspicious:
> 
> +link='if test "x$ac_cv_func_tigetstr" = xyes -a "x$ac_cv_header_curses_h"; t
> hen
>
> Testing just "x$ac_cv_header_curses_h" will always be true. Did you
> intend an ` = xyes' or ` = xno' in there?

Aha, this may be why Mads was seeing problems.  Although it should still
compile the module without the curses stuff as a fallback, so I'm
not convinced...

Mads Martin Joergensen <mmj@suse.de> wrote:
> Ok, I've tweaked the buildenvironment to not having ncurses-devel, and
> without it:
> 
> gcc  -s -rdynamic -o zsh main.o  `cat stamp-modobjs`   -ldl -lnsl -lm -lc
> utils.o(.text+0x2aa9): In function `gettempname':
> : the use of `mktemp' is dangerous, better use `mkstemp'
> init.o(.text+0x1277): In function `init_term':
> : undefined reference to `tgetent'

Is termcap still available?  I can't see it in the link line, which is
suspicious.  What did configure say when it was looking for tgetent
and tigetflag (from config.log)?

Oliver:
> The configure script is creating config.modules twice. Second time
> round when evaluating $link using the code in terminfo.mdd,
> $ac_cv_func_tigetstr was unset. It seems the configure script runs
> config.status. By running configure with --no-create, it was happy to
> build terminfo.so. Switching from autoconf 2.57 to 2.13 also solved the
> issue.

I thought it stored the configure variables somewhere?  I haven't seen this
problem for some reason.  I've got 2.50 here and something later at home.

> We really ought to ditch support for autoconf 2.13 in the 4.1 branch.
> Not having a mixture will just make things simpler.

Yes, I was thinking that.

pws


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited. 
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

autoconf 2.5 (Re: PATCH: terminfo horor)

[Oliver Kiddle]

This line of the patch looks suspicious:

+link='if test "x$ac_cv_func_tigetstr" = xyes -a "x$ac_cv_header_curses_h"; then

Testing just "x$ac_cv_header_curses_h" will always be true. Did you
intend an ` = xyes' or ` = xno' in there?

With SuSE 7.3, the patch was fine.

On my Debian testing system it solves the ERR problem but I found that
it would never build the terminfo module regardless of whether or not I
used the --with-curses-terminfo option. This turned out to be a quite
separate and largely unrelated problem.

The configure script is creating config.modules twice. Second time
round when evaluating $link using the code in terminfo.mdd,
$ac_cv_func_tigetstr was unset. It seems the configure script runs
config.status. By running configure with --no-create, it was happy to
build terminfo.so. Switching from autoconf 2.57 to 2.13 also solved the
issue.

configure cats config.modules.sh into config.status using
AC_OUTPUT_COMMANDS which is obsolete in autoconf 2.5. Using the new
AC_CONFIG_COMMANDS doesn't help because variables like
$ac_cv_func_tigetstr are still not set.

Does anyone understand the real point of config.status? Why does
configure need to run it? I get the impression that it should contain
just a duplicate of what is in config.modules as created and recreate
it as opposed to rerunning scripts to derive its contents.

We really ought to ditch support for autoconf 2.13 in the 4.1 branch.
Not having a mixture will just make things simpler.

Oliver