zsh-workers
 help / color / mirror / code / Atom feed
* Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
@ 2008-02-26 13:18 Dagobert Michelsen
  2008-02-26 14:14 ` Peter Stephenson
  0 siblings, 1 reply; 18+ messages in thread
From: Dagobert Michelsen @ 2008-02-26 13:18 UTC (permalink / raw)
  To: zsh-workers

Hi,

I am building zsh 4.3.5 on Solaris 10 x86 with the
Sun Studio 12 compiler and I have some failed tests
in the teststuite:

  ==> Running make test in work/dam-thor.d/zsh-4.3.5
gmake[1]: Entering directory `/export/medusa/dam/trunk/devel/zsh/work/ 
dam-thor.d/zsh-4.3.5'
cd Test ; gmake check
gmake[2]: Entering directory `/export/medusa/dam/trunk/devel/zsh/work/ 
dam-thor.d/zsh-4.3.5/Test'
if test -n "cc"; then \
           cd .. && DESTDIR= \
           gmake MODDIR=`pwd`/Test/Modules install.modules > /dev/ 
null; \
         fi
if ZTST_testlist="`for f in ./*.ztst; \
            do echo $f; done`" \
          ZTST_srcdir="." \
          ZTST_exe=../Src/zsh \
          ../Src/zsh +Z -f ./runtests.zsh; then \
          stat=0; \
         else \
          stat=1; \
         fi; \
         rm -rf Modules .zcompdump; \
         exit $stat
./A01grammar.ztst: starting.
This test hangs the shell when it fails...
./A01grammar.ztst: all tests successful.
./A02alias.ztst: starting.
./A02alias.ztst: all tests successful.
./A03quoting.ztst: starting.
Test ./A03quoting.ztst failed: bad status 1, expected 0 from:
   print '<\u0041>'
   printf '%s\n' $'<\u0042>'
   print '<\u0043>'
   printf '%s\n' $'<\u0044>'
Error output:
(eval):1: cannot do charset conversion
Was testing: \u in both print and printf
./A03quoting.ztst: test failed.
./A04redirect.ztst: starting.
./A04redirect.ztst: all tests successful.
./A05execution.ztst: starting.
./A05execution.ztst: all tests successful.
./A06assign.ztst: starting.
./A06assign.ztst: all tests successful.
./B01cd.ztst: starting.
./B01cd.ztst: all tests successful.
./B02typeset.ztst: starting.
./B02typeset.ztst: all tests successful.
./B03print.ztst: starting.
./B03print.ztst: all tests successful.
./B04read.ztst: starting.
./B04read.ztst: all tests successful.
./C01arith.ztst: starting.
./C01arith.ztst: all tests successful.
./C02cond.ztst: starting.
Warning: not testing [[ -N file ]] (not supported with NFS)
Test ./C02cond.ztst failed: bad status 1, expected 0 from:
   [[ newnewnew -nt zerolength && ! (unmodified -nt zerolength) ]]
Was testing: -nt cond
./C02cond.ztst: test failed.
./C03traps.ztst: starting.
This test takes at least three seconds...
This test, too, takes at least three seconds...
Another test that takes three seconds
./C03traps.ztst: all tests successful.
./C04funcdef.ztst: starting.
./C04funcdef.ztst: all tests successful.
./D01prompt.ztst: starting.
./D01prompt.ztst: all tests successful.
./D02glob.ztst: starting.
./D02glob.ztst: all tests successful.
./D03procsubst.ztst: starting.
./D03procsubst.ztst: all tests successful.
./D04parameter.ztst: starting.
./D04parameter.ztst: all tests successful.
./D05array.ztst: starting.
./D05array.ztst: all tests successful.
./D06subscript.ztst: starting.
./D06subscript.ztst: all tests successful.
./D07multibyte.ztst: starting.
Testing multibyte with locale en_US.UTF-8
*** /tmp/zsh.ztst.out.19793     Tue Feb 26 08:06:28 2008
--- /tmp/zsh.ztst.tout.19793    Tue Feb 26 08:06:28 2008
***************
*** 1,4 ****
! OK
! OK
! OK
! OK
--- 1,4 ----
! Failed: no error message and no question mark
! Failed: no error message and no question mark
! Failed: no error message and no question mark
! Failed: no error message and no question mark
Test ./D07multibyte.ztst failed: output differs from expected as  
shown above for:
   testfn() { (LC_ALL=C; print $'\u00e9') }
   repeat 4 testfn 2>&1 | while read line; do
     if [[ $line = *"character not in range"* ]]; then
       print OK
     elif [[ $line = "?" ]]; then
       print OK
     else
       print Failed: no error message and no question mark
     fi
   done
   true
Was testing: error handling in Unicode quoting
./D07multibyte.ztst: test failed.
./D08cmdsubst.ztst: starting.
./D08cmdsubst.ztst: all tests successful.
./E01options.ztst: starting.
This test hangs the shell when it fails...
./E01options.ztst: all tests successful.
./E02xtrace.ztst: starting.
./E02xtrace.ztst: all tests successful.
./V01zmodload.ztst: starting.
./V01zmodload.ztst: all tests successful.
./V02zregexparse.ztst: starting.
./V02zregexparse.ztst: all tests successful.
./V03mathfunc.ztst: starting.
./V03mathfunc.ztst: all tests successful.
./V04features.ztst: starting.
./V04features.ztst: all tests successful.
./V05styles.ztst: starting.
./V05styles.ztst: all tests successful.
./Y01completion.ztst: starting.
./Y01completion.ztst: all tests successful.
./Y02compmatch.ztst: starting.
./Y02compmatch.ztst: all tests successful.
./Y03arguments.ztst: starting.
./Y03arguments.ztst: all tests successful.
**************************************
29 successful test scripts, 3 failures, 0 skipped
**************************************
gmake[2]: *** [check] Error 1
gmake[2]: Leaving directory `/export/medusa/dam/trunk/devel/zsh/work/ 
dam-thor.d/zsh-4.3.5/Test'
gmake[1]: *** [test] Error 2
gmake[1]: Leaving directory `/export/medusa/dam/trunk/devel/zsh/work/ 
dam-thor.d/zsh-4.3.5'
gmake: *** [test-work/dam-thor.d/zsh-4.3.5/Makefile] Error 2


Invocation was
   $ ./configure --prefix=/opt/csw --exec_prefix=/opt/csw --bindir=/ 
opt/csw/bin --sbindir=/opt/csw/sbin --libexecdir=/opt/csw/libexec -- 
datadir=/opt/csw/share --sysconfdir=/opt/csw/etc --sharedstatedir=/ 
opt/csw/share --localstatedir=/opt/csw/var --libdir=/opt/csw/lib -- 
infodir=/opt/csw/share/info --includedir=/opt/csw/include --mandir=/ 
opt/csw/share/man --enable-maildir-support --enable-fndir=/opt/csw/ 
share/zsh/functions --enable-pcre


Best regards

   -- Dagobert Michelsen


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 13:18 Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC Dagobert Michelsen
@ 2008-02-26 14:14 ` Peter Stephenson
  2008-02-26 14:42   ` Dagobert Michelsen
  0 siblings, 1 reply; 18+ messages in thread
From: Peter Stephenson @ 2008-02-26 14:14 UTC (permalink / raw)
  To: Dagobert Michelsen; +Cc: zsh-workers

On Tue, 26 Feb 2008 14:18:14 +0100
Dagobert Michelsen <dam@blastwave.org> wrote:
> I am building zsh 4.3.5 on Solaris 10 x86 with the
> Sun Studio 12 compiler and I have some failed tests
> in the teststuite:
>...
> ./A03quoting.ztst: starting.
> Test ./A03quoting.ztst failed: bad status 1, expected 0 from:
>    print '<\u0041>'
>    printf '%s\n' $'<\u0042>'
>    print '<\u0043>'
>    printf '%s\n' $'<\u0044>'
> Error output:
> (eval):1: cannot do charset conversion
>...
> ./D07multibyte.ztst: starting.
> Testing multibyte with locale en_US.UTF-8
> *** /tmp/zsh.ztst.out.19793     Tue Feb 26 08:06:28 2008
> --- /tmp/zsh.ztst.tout.19793    Tue Feb 26 08:06:28 2008
> ***************
> *** 1,4 ****
> ! OK
> ! OK
> ! OK
> ! OK
> --- 1,4 ----
> ! Failed: no error message and no question mark
> ! Failed: no error message and no question mark
> ! Failed: no error message and no question mark
> ! Failed: no error message and no question mark
> Test ./D07multibyte.ztst failed: output differs from expected as  

This means the it hasn't found either iconv or any other library to do
character conversion.  (The error message is suppressed in the second case
but there's a good chance it's the same error.)  The shell has two basic
ways of doing this:

(i) compile environment directly supports ISO 10646
(ii) libraries support nl_langinfo(CODESET) and
     (a) the current locale uses UTF-8 so the shell
         can do a trivial conversion to UTF-8.
     (b) iconv() is available in -liconv.

(i) or (ii) are handled at compile time.  The choice between (ii)(a) and
(ii)(b) is handled at run time so long as nl_langinfo(CODESET) is
available.  I don't think (ii)(a) applies in these tests, so this
presumably means both that the environment doesn't support ISO 10646, and
that iconv wasn't found.

It might for developers' purposes (this doesn't affect you directly except
as far as bug reports go) be useful to distinguish what failed...

Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.176
diff -u -r1.176 utils.c
--- Src/utils.c	25 Jan 2008 16:48:23 -0000	1.176
+++ Src/utils.c	26 Feb 2008 14:09:43 -0000
@@ -4877,7 +4877,7 @@
 
     	    	    cd = iconv_open(nl_langinfo(CODESET), "UCS-4BE");
 		    if (cd == (iconv_t)-1) {
-			zerr("cannot do charset conversion");
+			zerr("cannot do charset conversion (iconv failed)");
 			CHARSET_FAILED();
 		    }
                     count = iconv(cd, &inptr, &inbytes, &t, &outbytes);
@@ -4889,12 +4889,12 @@
 		    if ((how & GETKEY_UPDATE_OFFSET) && s - sstart < *misc)
 			(*misc) += count;
 #   else
-                    zerr("cannot do charset conversion");
+                    zerr("cannot do charset conversion (iconv not available)");
 		    CHARSET_FAILED();
 #   endif
 		}
 #  else
-                zerr("cannot do charset conversion");
+                zerr("cannot do charset conversion (NLS not supported)");
 		CHARSET_FAILED();
 #  endif
 # endif


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 14:14 ` Peter Stephenson
@ 2008-02-26 14:42   ` Dagobert Michelsen
  2008-02-26 14:57     ` Peter Stephenson
  0 siblings, 1 reply; 18+ messages in thread
From: Dagobert Michelsen @ 2008-02-26 14:42 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: zsh-workers

Hi Peter,

first, thanks for the effort. I have applied the patch
and reran the testsuite.

Am 26.02.2008 um 15:14 schrieb Peter Stephenson:
> On Tue, 26 Feb 2008 14:18:14 +0100
> Dagobert Michelsen <dam@blastwave.org> wrote:
>> I am building zsh 4.3.5 on Solaris 10 x86 with the
>> Sun Studio 12 compiler and I have some failed tests
>> in the teststuite:
>> ...
>> ./A03quoting.ztst: starting.
>> Test ./A03quoting.ztst failed: bad status 1, expected 0 from:
>>    print '<\u0041>'
>>    printf '%s\n' $'<\u0042>'
>>    print '<\u0043>'
>>    printf '%s\n' $'<\u0044>'
>> Error output:
>> (eval):1: cannot do charset conversion

./A03quoting.ztst: starting.
Test ./A03quoting.ztst failed: bad status 1, expected 0 from:
   print '<\u0041>'
   printf '%s\n' $'<\u0042>'
   print '<\u0043>'
   printf '%s\n' $'<\u0044>'
Error output:
(eval):1: cannot do charset conversion (iconv failed)
Was testing: \u in both print and printf

>> ...
>> ./D07multibyte.ztst: starting.
>> Testing multibyte with locale en_US.UTF-8
>> *** /tmp/zsh.ztst.out.19793     Tue Feb 26 08:06:28 2008
>> --- /tmp/zsh.ztst.tout.19793    Tue Feb 26 08:06:28 2008
>> ***************
>> *** 1,4 ****
>> ! OK
>> ! OK
>> ! OK
>> ! OK
>> --- 1,4 ----
>> ! Failed: no error message and no question mark
>> ! Failed: no error message and no question mark
>> ! Failed: no error message and no question mark
>> ! Failed: no error message and no question mark
>> Test ./D07multibyte.ztst failed: output differs from expected as

./D07multibyte.ztst: starting.
Testing multibyte with locale en_US.UTF-8
*** /tmp/zsh.ztst.out.12712     Tue Feb 26 09:35:42 2008
--- /tmp/zsh.ztst.tout.12712    Tue Feb 26 09:35:42 2008
***************
*** 1,4 ****
! OK
! OK
! OK
! OK
--- 1,4 ----
! Failed: no error message and no question mark
! Failed: no error message and no question mark
! Failed: no error message and no question mark
! Failed: no error message and no question mark
Test ./D07multibyte.ztst failed: output differs from expected as  
shown above for:
   testfn() { (LC_ALL=C; print $'\u00e9') }
   repeat 4 testfn 2>&1 | while read line; do
     if [[ $line = *"character not in range"* ]]; then
       print OK
     elif [[ $line = "?" ]]; then
       print OK
     else
       print Failed: no error message and no question mark
     fi
   done
   true
Was testing: error handling in Unicode quoting


> This means the it hasn't found either iconv or any other library to do
> character conversion.  (The error message is suppressed in the  
> second case
> but there's a good chance it's the same error.)  The shell has two  
> basic
> ways of doing this:
>
> (i) compile environment directly supports ISO 10646
> (ii) libraries support nl_langinfo(CODESET) and
>      (a) the current locale uses UTF-8 so the shell
>          can do a trivial conversion to UTF-8.
>      (b) iconv() is available in -liconv.
>
> (i) or (ii) are handled at compile time.  The choice between (ii) 
> (a) and
> (ii)(b) is handled at run time so long as nl_langinfo(CODESET) is
> available.  I don't think (ii)(a) applies in these tests, so this
> presumably means both that the environment doesn't support ISO  
> 10646, and
> that iconv wasn't found.

At least the dynamic binding is working:

thor% ldd work/dam-thor.d/zsh-4.3.5/Src/zsh
         libpcre.so.0 =>  /opt/csw/lib/i386/libpcre.so.0
         libiconv.so.2 =>         /opt/csw/lib/i386/libiconv.so.2
         libsocket.so.1 =>        /usr/lib/libsocket.so.1
         libdl.so.1 =>    /usr/lib/libdl.so.1
         libncurses.so.5 =>       /opt/csw/lib/i386/libncurses.so.5
         libm.so.1 =>     /usr/lib/libm.so.1
         libc.so.1 =>     /usr/lib/libc.so.1
         libnsl.so.1 =>   /usr/lib/libnsl.so.1
         libmp.so.2 =>    /usr/lib/libmp.so.2

It may be of interest that I am doing a "package build" with
DESTDIR installation as following:

cd work/dam-thor.d/zsh-4.3.5 && prefix="/opt/csw" exec_prefix="/opt/ 
csw" bindir="/opt/csw/bin" optbindir="/opt/csw/bin/i386" sbindir="/ 
opt/csw/sbin" libexecdir="/opt/csw/libexec" datadir="/opt/csw/share"  
sysconfdir="/opt/csw/etc" sharedstatedir="/opt/csw/share"  
localstatedir="/opt/csw/var" libdir="/opt/csw/lib" optlibdir="/opt/ 
csw/lib/i386" infodir="/opt/csw/share/info" lispdir="/opt/csw/share/ 
emacs/site-lisp" includedir="/opt/csw/include" mandir="/opt/csw/share/ 
man" docdir="/opt/csw/share/doc" sourcedir="/opt/csw/src" CPPFLAGS=" - 
I/export/medusa/dam/build/build.5.8-i386/opt/csw/include -I/opt/csw/ 
include" CFLAGS="-I/opt/csw/include/ncurses -xO3 -xtarget=generic - 
xarch=generic -I/export/medusa/dam/build/build.5.8-i386/opt/csw/ 
include -I/opt/csw/include " CXXFLAGS="-xO3 -xtarget=generic - 
xarch=generic -I/export/medusa/dam/build/build.5.8-i386/opt/csw/ 
include -I/opt/csw/include" LDFLAGS="  -L/export/medusa/dam/build/ 
build.5.8-i386/opt/csw/lib -L/opt/csw/lib" ASFLAGS="" OPTFLAGS="-xO3 - 
xtarget=generic -xarch=generic" CC="cc" CXX="CC" LD_OPTIONS="-R/opt/ 
csw/lib/\$ISALIST -R/opt/csw/lib" CC_HOME="/opt/studio/SOS11/ 
SUNWspro" CC_VERSION="Sun C 5.8 Patch 121016-07 2007/10/03"  
CXX_VERSION="Sun C++ 5.8 Patch 121018-11 2007/05/02" VENDORNAME=""  
VENDORSTAMP="" GARCH="i386" GAROSREL="5.8" GARPACKAGE="zsh"  
PKG_CONFIG_PATH="/export/medusa/dam/build/build.5.8-i386/opt/csw/lib/ 
pkgconfig:/opt/csw/lib/pkgconfig:" DESTDIR="/export/medusa/dam/build/ 
build.5.8-i386" ./configure --prefix=/opt/csw --exec_prefix=/opt/csw  
--bindir=/opt/csw/bin --sbindir=/opt/csw/sbin --libexecdir=/opt/csw/ 
libexec --datadir=/opt/csw/share --sysconfdir=/opt/csw/etc -- 
sharedstatedir=/opt/csw/share --localstatedir=/opt/csw/var --libdir=/ 
opt/csw/lib --infodir=/opt/csw/share/info --includedir=/opt/csw/ 
include --mandir=/opt/csw/share/man --enable-maildir-support --enable- 
fndir=/opt/csw/share/zsh/functions --enable-pcre

zsh configuration
-----------------
zsh version               : 4.3.5
host operating system     : i386-pc-solaris2.8
source code location      : .
compiler                  : cc
preprocessor flags        :  -I/export/medusa/dam/build/build.5.8- 
i386/opt/csw/include -I/opt/csw/include -I/opt/csw/include
executable compiler flags : -I/opt/csw/include/ncurses -xO3 - 
xtarget=generic -xarch=generic -I/export/medusa/dam/build/build.5.8- 
i386/opt/csw/include -I/opt/csw/include
module compiler flags     : -I/opt/csw/include/ncurses -xO3 - 
xtarget=generic -xarch=generic -I/export/medusa/dam/build/build.5.8- 
i386/opt/csw/include -I/opt/csw/include  -KPIC
executable linker flags   :   -L/export/medusa/dam/build/build.5.8- 
i386/opt/csw/lib -L/opt/csw/lib
module linker flags       :   -L/export/medusa/dam/build/build.5.8- 
i386/opt/csw/lib -L/opt/csw/lib  -G
library flags             : -L/opt/csw/lib -lpcre -liconv -lsocket - 
ldl -lncurses -lm  -lc
installation basename     : zsh
binary install path       : /opt/csw/bin
man page install path     : /opt/csw/share/man
info install path         : /opt/csw/share/info
functions install path    : /opt/csw/share/zsh/functions
See config.modules for installed modules and functions.


If I can do anything to help resolving these testing issues let me know.

Thanks!

   -- Dagobert Michelsen


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 14:42   ` Dagobert Michelsen
@ 2008-02-26 14:57     ` Peter Stephenson
  2008-02-26 15:18       ` Dagobert Michelsen
  0 siblings, 1 reply; 18+ messages in thread
From: Peter Stephenson @ 2008-02-26 14:57 UTC (permalink / raw)
  To: Dagobert Michelsen; +Cc: zsh-workers

On Tue, 26 Feb 2008 15:42:19 +0100
Dagobert Michelsen <dam@blastwave.org> wrote:
> Hi Peter,
> 
> first, thanks for the effort. I have applied the patch
> and reran the testsuite.
> ./A03quoting.ztst: starting.
> Test ./A03quoting.ztst failed: bad status 1, expected 0 from:
>    print '<\u0041>'
>    printf '%s\n' $'<\u0042>'
>    print '<\u0043>'
>    printf '%s\n' $'<\u0044>'
> Error output:
> (eval):1: cannot do charset conversion (iconv failed)
> Was testing: \u in both print and printf

Weird... that's the variant I wasn't expecting since it implies
a run time failure in iconv, and not a dynamical link failure, either.
That means this call failed (Src/utils.c:4878 in the current source):

    	    	    cd = iconv_open(nl_langinfo(CODESET), "UCS-4BE");
		    if (cd == (iconv_t)-1) {
			zerr("cannot do charset conversion (iconv failed)");
			CHARSET_FAILED();
		    }

Any idea if your iconv supports UCS-4BE?  GNU iconv will tell you with
"iconv --list", but if you have a Solaris variant it's likely to be
different.  Otherwise, could nl_langinfo(CODESET) not be returning
anything?

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 14:57     ` Peter Stephenson
@ 2008-02-26 15:18       ` Dagobert Michelsen
  2008-02-26 16:10         ` * " Peter Stephenson
  0 siblings, 1 reply; 18+ messages in thread
From: Dagobert Michelsen @ 2008-02-26 15:18 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: zsh-workers

Hi Peter,

Am 26.02.2008 um 15:57 schrieb Peter Stephenson:
> On Tue, 26 Feb 2008 15:42:19 +0100
> Dagobert Michelsen <dam@blastwave.org> wrote:
>> first, thanks for the effort. I have applied the patch
>> and reran the testsuite.
>> ./A03quoting.ztst: starting.
>> Test ./A03quoting.ztst failed: bad status 1, expected 0 from:
>>    print '<\u0041>'
>>    printf '%s\n' $'<\u0042>'
>>    print '<\u0043>'
>>    printf '%s\n' $'<\u0044>'
>> Error output:
>> (eval):1: cannot do charset conversion (iconv failed)
>> Was testing: \u in both print and printf
>
> Weird... that's the variant I wasn't expecting since it implies
> a run time failure in iconv, and not a dynamical link failure, either.
> That means this call failed (Src/utils.c:4878 in the current source):
>
>     	    	    cd = iconv_open(nl_langinfo(CODESET), "UCS-4BE");
> 		    if (cd == (iconv_t)-1) {
> 			zerr("cannot do charset conversion (iconv failed)");
> 			CHARSET_FAILED();
> 		    }
>
> Any idea if your iconv supports UCS-4BE?  GNU iconv will tell you with
> "iconv --list", but if you have a Solaris variant it's likely to be
> different.  Otherwise, could nl_langinfo(CODESET) not be returning
> anything?

In fact I am linking against libiconv from Blastwave.org at
/opt/csw/lib. However, the charset seems to be supported:

thor% /opt/csw/bin/iconv -l | grep UCS-4BE
UCS-4BE

The system trace looks like this:

16896:  sigprocmask(SIG_BLOCK, 0x080464E4, 0x080464C0)  = 0
16896:  sigprocmask(SIG_BLOCK, 0x080460B4, 0x08046090)  = 0
16896:  -> libiconv:libiconv_open(0xdf986e04, 0x80c1e90)

...and...

thor% mdb -p 16896
Loading modules: [ ]
 > 0xdf986e04/S
0xdf986e04:     646
 > 0x80c1e90/S
0x80c1e90:      UCS-4BE

Looks like there is no 646:

thor% /opt/csw/bin/iconv -l | grep 646
ANSI_X3.4-1968 ANSI_X3.4-1986 ASCII CP367 IBM367 ISO-IR-6 ISO646-US  
ISO_646.IRV:1991 US US-ASCII CSASCII
ISO-10646-UCS-2 UCS-2 CSUNICODE
ISO-10646-UCS-4 UCS-4 CSUCS4
ISO-IR-14 ISO646-JP JIS_C6220-1969-RO JP CSISO14JISC6220RO
CN GB_1988-80 ISO-IR-57 ISO646-CN CSISO57GB1988

Is this an error from zsh or is this a bug in libiconv?


Thanks!

   -- Dago


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 15:18       ` Dagobert Michelsen
@ 2008-02-26 16:10         ` Peter Stephenson
  2008-02-26 17:37           ` Oliver Kiddle
  2008-02-26 18:08           ` Peter Stephenson
  0 siblings, 2 replies; 18+ messages in thread
From: Peter Stephenson @ 2008-02-26 16:10 UTC (permalink / raw)
  To: Dagobert Michelsen; +Cc: zsh-workers

Dagobert Michelsen wrote:
> 16896:  -> libiconv:libiconv_open(0xdf986e04, 0x80c1e90)
> 
> ...and...
> 
> thor% mdb -p 16896
> Loading modules: [ ]
>  > 0xdf986e04/S
> 0xdf986e04:     646
>  > 0x80c1e90/S
> 0x80c1e90:      UCS-4BE
> 
> Looks like there is no 646:
> 
> thor% /opt/csw/bin/iconv -l | grep 646
> ANSI_X3.4-1968 ANSI_X3.4-1986 ASCII CP367 IBM367 ISO-IR-6 ISO646-US  
> ISO_646.IRV:1991 US US-ASCII CSASCII
> ISO-10646-UCS-2 UCS-2 CSUNICODE
> ISO-10646-UCS-4 UCS-4 CSUCS4
> ISO-IR-14 ISO646-JP JIS_C6220-1969-RO JP CSISO14JISC6220RO
> CN GB_1988-80 ISO-IR-57 ISO646-CN CSISO57GB1988
> 
> Is this an error from zsh or is this a bug in libiconv?

The first argument comes directly from what nl_langinfo(CODESET) is
returning.  I've tried it on our rather old Solaris 8 system and I get
the same as you, 646---and my iconv doesn't handle that either.

I asked the nice man from Mountain View about "nl_langinfo CODESET 646"
and it seems Solaris just sort of does that if doesn't think much of the
current codeset (e.g. not installed).

So I think we just need to massage it to some reasonable default, which
probably means "US-ASCII" (I'm not sure why that's better than "ASCII",
which I thought was unique, but that's what people seem to use---POSIX
rather self-centredly suggests "POSIX" but iconv doesn't seem to support
that).  I don't think 646 means anything, so I haven't made this
dependent on system.  Assuming ASCII shouldn't be a problem:  if the
7-bit subset isn't ASCII the shell is likely to have disappeared in a
puff of smoke by now.

I suppose trapping "" is a good thing, too, since iconv_open() doesn't
appear to have useful defaults (and there's no reason why it should
have).

As they say in US police dramas, "I'm denying your 646".  Or something.

Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.177
diff -u -r1.177 utils.c
--- Src/utils.c	26 Feb 2008 14:50:05 -0000	1.177
+++ Src/utils.c	26 Feb 2008 16:02:07 -0000
@@ -4867,6 +4867,7 @@
 		} else {
 #   ifdef HAVE_ICONV
 		    ICONV_CONST char *inptr = inbuf;
+		    const char *codesetstr = nl_langinfo(CODESET);
     	    	    inbytes = 4;
 		    outbytes = 6;
 		    /* store value in big endian form */
@@ -4875,6 +4876,18 @@
 			wval >>= 8;
 		    }
 
+		    /*
+		     * If the code set isn't handled, we'd better
+		     * assume it's US-ASCII rather than just failing
+		     * hopelessly.  Solaris has a weird habit of
+		     * returning 646.
+		     *
+		     * It shouldn't ever be NULL, but while we're
+		     * being paranoid...
+		     */
+		    if (!codessetstr || !*codsetstr ||
+			!strcmp(codesetstr, "646"))
+			codesetstr == "US-ASCII";
     	    	    cd = iconv_open(nl_langinfo(CODESET), "UCS-4BE");
 		    if (cd == (iconv_t)-1) {
 			zerr("cannot do charset conversion (iconv failed)");


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 16:10         ` * " Peter Stephenson
@ 2008-02-26 17:37           ` Oliver Kiddle
  2008-02-27 11:29             ` Peter Stephenson
  2008-02-26 18:08           ` Peter Stephenson
  1 sibling, 1 reply; 18+ messages in thread
From: Oliver Kiddle @ 2008-02-26 17:37 UTC (permalink / raw)
  To: Dagobert Michelsen, zsh-workers

Peter wrote:
> > Is this an error from zsh or is this a bug in libiconv?

It's actually better to avoid using libiconv and stick to the builtin
iconv on Solaris. It handles 646 correctly. I always make a point of
making sure it doesn't find GNU libiconv when compiling.

> +		    if (!codessetstr || !*codsetstr ||
> +			!strcmp(codesetstr, "646"))
> +			codesetstr == "US-ASCII";

Is the last line of this correct? Isn't it meant to be an assignment?

I would be inclined to wrap the whole thing inside #ifdef
_libiconv_version because the problem only occurs when using GNU iconv
instead of the Solaris builtin one.

Oliver


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 16:10         ` * " Peter Stephenson
  2008-02-26 17:37           ` Oliver Kiddle
@ 2008-02-26 18:08           ` Peter Stephenson
  2008-02-26 18:12             ` İsmail Dönmez
  2008-02-26 19:19             ` Dagobert Michelsen
  1 sibling, 2 replies; 18+ messages in thread
From: Peter Stephenson @ 2008-02-26 18:08 UTC (permalink / raw)
  To: zsh-workers; +Cc: Dagobert Michelsen

On Tue, 26 Feb 2008 16:10:58 +0000
Peter Stephenson <pws@csr.com> wrote:
> +		    if (!codessetstr || !*codsetstr ||
> +			!strcmp(codesetstr, "646"))
> +			codesetstr == "US-ASCII";

Er, except that there are three mistakes in those three lines of code, as
Geoff pointed out.  I was lulled into a false sense of security because
this doesn't get compiled on my machine.  I tried it specially. 

Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.178
diff -u -r1.178 utils.c
--- Src/utils.c	26 Feb 2008 16:19:34 -0000	1.178
+++ Src/utils.c	26 Feb 2008 18:05:59 -0000
@@ -4885,9 +4885,9 @@
 		     * It shouldn't ever be NULL, but while we're
 		     * being paranoid...
 		     */
-		    if (!codessetstr || !*codsetstr ||
+		    if (!codesetstr || !*codesetstr ||
 			!strcmp(codesetstr, "646"))
-			codesetstr == "US-ASCII";
+			codesetstr = "US-ASCII";
     	    	    cd = iconv_open(nl_langinfo(CODESET), "UCS-4BE");
 		    if (cd == (iconv_t)-1) {
 			zerr("cannot do charset conversion (iconv failed)");


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 18:08           ` Peter Stephenson
@ 2008-02-26 18:12             ` İsmail Dönmez
  2008-02-26 19:19             ` Dagobert Michelsen
  1 sibling, 0 replies; 18+ messages in thread
From: İsmail Dönmez @ 2008-02-26 18:12 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: zsh-workers, Dagobert Michelsen

Hi,

On Tue, Feb 26, 2008 at 8:08 PM, Peter Stephenson <pws@csr.com> wrote:
> On Tue, 26 Feb 2008 16:10:58 +0000
>  Peter Stephenson <pws@csr.com> wrote:
>  > +                 if (!codessetstr || !*codsetstr ||
>  > +                     !strcmp(codesetstr, "646"))
>  > +                     codesetstr == "US-ASCII";
>
>  Er, except that there are three mistakes in those three lines of code, as
>  Geoff pointed out.  I was lulled into a false sense of security because
>  this doesn't get compiled on my machine.  I tried it specially.
>
>  Index: Src/utils.c
>  ===================================================================
>  RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
>  retrieving revision 1.178
>  diff -u -r1.178 utils.c
>  --- Src/utils.c 26 Feb 2008 16:19:34 -0000      1.178
>  +++ Src/utils.c 26 Feb 2008 18:05:59 -0000
>  @@ -4885,9 +4885,9 @@
>                      * It shouldn't ever be NULL, but while we're
>                      * being paranoid...
>                      */
>  -                   if (!codessetstr || !*codsetstr ||
>  +                   if (!codesetstr || !*codesetstr ||
>                         !strcmp(codesetstr, "646"))
>  -                       codesetstr == "US-ASCII";
>  +                       codesetstr = "US-ASCII";
>                     cd = iconv_open(nl_langinfo(CODESET), "UCS-4BE");
>                     if (cd == (iconv_t)-1) {
>                         zerr("cannot do charset conversion (iconv failed)");

Yeah my patch did the same. Thanks.

-- 
Never learn by your mistakes, if you do you may never dare to try again


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 18:08           ` Peter Stephenson
  2008-02-26 18:12             ` İsmail Dönmez
@ 2008-02-26 19:19             ` Dagobert Michelsen
  2008-02-26 19:47               ` Dagobert Michelsen
  1 sibling, 1 reply; 18+ messages in thread
From: Dagobert Michelsen @ 2008-02-26 19:19 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: zsh-workers

Hi Peter,

Am 26.02.2008 um 19:08 schrieb Peter Stephenson:
> Er, except that there are three

Four ;-)

> mistakes in those three lines of code, as
> Geoff pointed out.  I was lulled into a false sense of security  
> because
> this doesn't get compiled on my machine.  I tried it specially.
>
> Index: Src/utils.c
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
> retrieving revision 1.178
> diff -u -r1.178 utils.c
> --- Src/utils.c	26 Feb 2008 16:19:34 -0000	1.178
> +++ Src/utils.c	26 Feb 2008 18:05:59 -0000
> @@ -4885,9 +4885,9 @@
>  		     * It shouldn't ever be NULL, but while we're
>  		     * being paranoid...
>  		     */
> -		    if (!codessetstr || !*codsetstr ||
> +		    if (!codesetstr || !*codesetstr ||
>  			!strcmp(codesetstr, "646"))
> -			codesetstr == "US-ASCII";
> +			codesetstr = "US-ASCII";
>      	    	    cd = iconv_open(nl_langinfo(CODESET), "UCS-4BE");

      	    	    cd = iconv_open(codesetstr, "UCS-4BE");

>  		    if (cd == (iconv_t)-1) {
>  			zerr("cannot do charset conversion (iconv failed)");

Do you have a clue about the other failing tests?


Best regards

   -- Dago


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 19:19             ` Dagobert Michelsen
@ 2008-02-26 19:47               ` Dagobert Michelsen
  2008-02-26 20:42                 ` Peter Stephenson
  0 siblings, 1 reply; 18+ messages in thread
From: Dagobert Michelsen @ 2008-02-26 19:47 UTC (permalink / raw)
  To: zsh-workers

Hi all,

Am 26.02.2008 um 20:19 schrieb Dagobert Michelsen:
> Do you have a clue about the other failing tests?

The testsuite now runs fine with my fix for the last fix
with the exception of one remaining failing test:

thor% ZTST_verbose=1 gmake TESTNUM=C02 check
if test -n "cc"; then \
           cd .. && DESTDIR= \
           gmake MODDIR=`pwd`/Test/Modules install.modules > /dev/ 
null; \
         fi
if ZTST_testlist="`for f in ./C02*.ztst; \
            do echo $f; done`" \
          ZTST_srcdir="." \
          ZTST_exe=../Src/zsh \
          ../Src/zsh +Z -f ./runtests.zsh; then \
          stat=0; \
         else \
          stat=1; \
         fi; \
         rm -rf Modules .zcompdump; \
         exit $stat
./C02cond.ztst: starting.
Running test: -a cond
Test successful.
Running test: -b cond
Test successful.
Running test: -c cond
Test successful.
Running test: -d cond
Test successful.
Running test: -e cond
Test successful.
Running test: -f cond
Test successful.
Running test: -g cond
Test successful.
Running test: -h cond
Test successful.
Running test: -k cond
Test successful.
Running test: -n cond
Test successful.
Running test: -o cond
Test successful.
Running test: -p cond
Test successful.
Running test: -r cond
Test successful.
Running test: -s cond
Test successful.
Running test: -u cond
Test successful.
Running test: -x cond
Test successful.
Running test: -z cond
Test successful.
Running test: -L cond
Test successful.
Running test: -O cond
Test successful.
Running test: -G cond
Test successful.
Running test: -N cond
Warning: not testing [[ -N file ]] (not supported with NFS)
Test successful.
Running test: -nt cond
Test ./C02cond.ztst failed: bad status 1, expected 0 from:
   [[ newnewnew -nt zerolength && ! (unmodified -nt zerolength) ]]
Was testing: -nt cond
./C02cond.ztst: test failed.
**************************************
0 successful test scripts, 1 failure, 0 skipped
**************************************
gmake: *** [check] Error 1


I suspect this part here from C02cond.ztst:

> # can't be bothered with -S
>
>   if [[ $OSTYPE == "cygwin" ]]; then
>     print -u$ZTST_fd "Warning: not testing [[ -N file ]] (not  
> supported on Cygwin)"
>     true
>   elif [[ "$(find . -prune -fstype nfs 2>/dev/null)" == "." ]]; then
>     print -u$ZTST_fd "Warning: not testing [[ -N file ]] (not  
> supported with NFS)"
>     true
>   else
>     print -u $ZTST_fd 'This test takes two seconds...'
>     sleep 2
>     cat unmodified
>     touch newnewnew
>     [[ -N newnewnew && ! -N unmodified ]]
>   fi
> 0:-N cond
> F:This test can fail on NFS-mounted filesystems as the access and
> F:modification times are not updated separately.  The test will fail
> F:on HFS+ (Apple Mac OS X default) filesystems because access times
> F:are not recorded.  Also, Linux ext3 filesystems may be mounted
> F:with the noatime option which does not update access times.
> F:Failures in these cases do not indicate a problem in the shell.
>
>   [[ newnewnew -nt zerolength && ! (unmodified -nt zerolength) ]]
> 0:-nt cond


If this is on NFS (which it is in my build environment) the
file 'newnewnew' is not generated and the following tests fail
instead of being skipped on NFS.


Best regards

   -- Dagobert


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 19:47               ` Dagobert Michelsen
@ 2008-02-26 20:42                 ` Peter Stephenson
  0 siblings, 0 replies; 18+ messages in thread
From: Peter Stephenson @ 2008-02-26 20:42 UTC (permalink / raw)
  To: Dagobert Michelsen, zsh-workers

Dagobert Michelsen wrote:
> If this is on NFS (which it is in my build environment) the
> file 'newnewnew' is not generated and the following tests fail
> instead of being skipped on NFS.

I can see how that might be problematic.  Here's the simple way out.

Index: Test/C02cond.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/C02cond.ztst,v
retrieving revision 1.21
diff -u -r1.21 C02cond.ztst
--- Test/C02cond.ztst	10 Jan 2008 18:53:50 -0000	1.21
+++ Test/C02cond.ztst	26 Feb 2008 20:42:20 -0000
@@ -125,6 +125,10 @@
 
 # can't be bothered with -S
 
+  print -u $ZTST_fd 'This test takes two seconds...'
+  sleep 2
+  cat unmodified
+  touch newnewnew
   if [[ $OSTYPE == "cygwin" ]]; then
     print -u$ZTST_fd "Warning: not testing [[ -N file ]] (not supported on Cygwin)"
     true
@@ -132,10 +136,6 @@
     print -u$ZTST_fd "Warning: not testing [[ -N file ]] (not supported with NFS)"
     true
   else
-    print -u $ZTST_fd 'This test takes two seconds...'
-    sleep 2
-    cat unmodified
-    touch newnewnew
     [[ -N newnewnew && ! -N unmodified ]]
   fi
 0:-N cond



-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-26 17:37           ` Oliver Kiddle
@ 2008-02-27 11:29             ` Peter Stephenson
  2008-02-27 16:16               ` Bart Schaefer
  2008-02-29  9:53               ` Oliver Kiddle
  0 siblings, 2 replies; 18+ messages in thread
From: Peter Stephenson @ 2008-02-27 11:29 UTC (permalink / raw)
  To: Dagobert Michelsen, zsh-workers

Oliver Kiddle wrote:
> Peter wrote:
> > > Is this an error from zsh or is this a bug in libiconv?
> 
> It's actually better to avoid using libiconv and stick to the builtin
> iconv on Solaris. It handles 646 correctly. I always make a point of
> making sure it doesn't find GNU libiconv when compiling.

I've tried to take account of that below...

> > +		    if (!codessetstr || !*codsetstr ||
> > +			!strcmp(codesetstr, "646"))
> > +			codesetstr == "US-ASCII";
> 
> Is the last line of this correct? Isn't it meant to be an assignment?

Yes, that was fixed by Dagobert.

> I would be inclined to wrap the whole thing inside #ifdef
> _libiconv_version because the problem only occurs when using GNU iconv
> instead of the Solaris builtin one.

And there's the reciprocal problem: some native Solaris iconv's (such as
the one I just tried in Solaris 8) don't support US-ASCII and other
obvious forms, they only support 646 etc, so we definitely need more
work.

The following is the best I've got:  it tests for _libiconv_version
(which should be an external variable and hence tell us whether we're
actually linking against libiconv, whatever the header is) in configure
and if that's set compiles in some fairly defensive code.  I tried
this under Cygwin which uses libiconv.

Note this doesn't trigger if iconv is part of glibc, as under (most?)
Linux distributions.

Index: MACHINES
===================================================================
RCS file: /cvsroot/zsh/zsh/MACHINES,v
retrieving revision 1.5
diff -u -r1.5 MACHINES
--- MACHINES	21 May 2007 09:52:30 -0000	1.5
+++ MACHINES	27 Feb 2008 10:20:16 -0000
@@ -212,7 +212,11 @@
 	This may work, but the first username completion will be _very_
 	slow (as slow as in tcsh).
 
-Sun: Solaris 2.x, 8, 9
+Sun: Solaris 2.x, 8, 9, ...
+	It is recommended that the system library version of iconv()
+	be used rather than libiconv since there are incompatibilities
+	in the way codesets are named.
+
 	The UCB versions of the routines for reading directories are not
 	usable (the struct definitions are incompatible with the ones
 	assumed by zsh).  The symptom of this is that globbed filenames in
Index: configure.ac
===================================================================
RCS file: /cvsroot/zsh/zsh/configure.ac,v
retrieving revision 1.92
diff -u -r1.92 configure.ac
--- configure.ac	3 Feb 2008 18:17:13 -0000	1.92
+++ configure.ac	27 Feb 2008 10:20:17 -0000
@@ -824,8 +824,13 @@
       [ #include <iconv.h> ])
   fi
 fi
+AH_TEMPLATE([ICONV_FROM_LIBICONV],
+[Define to 1 if iconv() is linked from libiconv])
 if test "x$ac_found_iconv" = xyes; then
   AC_DEFINE(HAVE_ICONV, 1, [Define if you have the iconv() function.])
+  AC_TRY_LINK([#include <iconv.h>],
+    [int myversion = _libiconv_version],
+    AC_DEFINE(ICONV_FROM_LIBICONV), )
 fi
 
 dnl Check if iconv uses const in prototype declaration
Index: Src/utils.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/utils.c,v
retrieving revision 1.181
diff -u -r1.181 utils.c
--- Src/utils.c	26 Feb 2008 20:50:12 -0000	1.181
+++ Src/utils.c	27 Feb 2008 10:20:20 -0000
@@ -4880,15 +4880,32 @@
 		     * If the code set isn't handled, we'd better
 		     * assume it's US-ASCII rather than just failing
 		     * hopelessly.  Solaris has a weird habit of
-		     * returning 646.
+		     * returning 646.  This is handled by the
+		     * native iconv(), but not by GNU iconv; what's
+		     * more, some versions of the native iconv don't
+		     * handle standard names like ASCII.
+		     *
+		     * This should only be a problem if there's a
+		     * mismatch between the NLS and the iconv in use,
+		     * which probably only means if libiconv is in use.
+		     * We checked at configure time if our libraries
+		     * pulled in _libiconv_version, which should be
+		     * a good test.
 		     *
 		     * It shouldn't ever be NULL, but while we're
 		     * being paranoid...
 		     */
-		    if (!codesetstr || !*codesetstr ||
-			!strcmp(codesetstr, "646"))
+#ifdef ICONV_FROM_LIBICONV
+		    if (!codesetstr || !*codesetstr)
 			codesetstr = "US-ASCII";
+#endif
     	    	    cd = iconv_open(codesetstr, "UCS-4BE");
+#ifdef ICONV_FROM_LIBICONV
+		    if (cd == (iconv_t)-1 &&  !strcmp(codesetstr, "646")) {
+			codesetstr = "US-ASCII";
+			cd = iconv_open(codesetstr, "UCS-4BE");
+		    }
+#endif
 		    if (cd == (iconv_t)-1) {
 			zerr("cannot do charset conversion (iconv failed)");
 			CHARSET_FAILED();


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-27 11:29             ` Peter Stephenson
@ 2008-02-27 16:16               ` Bart Schaefer
  2008-02-29  9:53               ` Oliver Kiddle
  1 sibling, 0 replies; 18+ messages in thread
From: Bart Schaefer @ 2008-02-27 16:16 UTC (permalink / raw)
  To: zsh-workers

On Feb 27, 11:29am, Peter Stephenson wrote:
}
} > I would be inclined to wrap the whole thing inside #ifdef
} > _libiconv_version because the problem only occurs when using GNU iconv
} > instead of the Solaris builtin one.
} 
} And there's the reciprocal problem: some native Solaris iconv's (such as
} the one I just tried in Solaris 8) don't support US-ASCII and other
} obvious forms, they only support 646 etc, so we definitely need more
} work.

It's beginning to sound as if it might be worthwhile making a configure
test for this.


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-27 11:29             ` Peter Stephenson
  2008-02-27 16:16               ` Bart Schaefer
@ 2008-02-29  9:53               ` Oliver Kiddle
  2008-03-03 12:08                 ` Peter Stephenson
  1 sibling, 1 reply; 18+ messages in thread
From: Oliver Kiddle @ 2008-02-29  9:53 UTC (permalink / raw)
  To: zsh-workers

Peter wrote:
> The following is the best I've got:  it tests for _libiconv_version
> (which should be an external variable and hence tell us whether we're
> actually linking against libiconv, whatever the header is) in configure

I'd have thought the AC_CHECK_DECL for _libiconv_version as used already
earlier in configure.ac is a sufficient test. If you try to compile zsh
with the iconv header file not matching what it links against, you get
bigger problems and it doesn't compile at all. In fact, there is
_LIBICONV_VERSION which is a #define so you could #ifdef that.

Oliver


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-02-29  9:53               ` Oliver Kiddle
@ 2008-03-03 12:08                 ` Peter Stephenson
  2008-03-03 12:43                   ` Dagobert Michelsen
  0 siblings, 1 reply; 18+ messages in thread
From: Peter Stephenson @ 2008-03-03 12:08 UTC (permalink / raw)
  To: Zsh Hackers' List

On Fri, 29 Feb 2008 10:53:24 +0100
Oliver Kiddle <okiddle@yahoo.co.uk> wrote:
> Peter wrote:
> > The following is the best I've got:  it tests for _libiconv_version
> > (which should be an external variable and hence tell us whether we're
> > actually linking against libiconv, whatever the header is) in configure
> 
> I'd have thought the AC_CHECK_DECL for _libiconv_version as used already
> earlier in configure.ac is a sufficient test. If you try to compile zsh
> with the iconv header file not matching what it links against, you get
> bigger problems and it doesn't compile at all. In fact, there is
> _LIBICONV_VERSION which is a #define so you could #ifdef that.

I wanted to be safe about that and as far as I can tell checking if we get
_libiconv_version linked in should do that, right?

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-03-03 12:08                 ` Peter Stephenson
@ 2008-03-03 12:43                   ` Dagobert Michelsen
  2008-03-03 12:59                     ` Peter Stephenson
  0 siblings, 1 reply; 18+ messages in thread
From: Dagobert Michelsen @ 2008-03-03 12:43 UTC (permalink / raw)
  To: Peter Stephenson; +Cc: Zsh Hackers' List

Hi,

Am 03.03.2008 um 13:08 schrieb Peter Stephenson:
> On Fri, 29 Feb 2008 10:53:24 +0100
> Oliver Kiddle <okiddle@yahoo.co.uk> wrote:
>> Peter wrote:
>>> The following is the best I've got:  it tests for _libiconv_version
>>> (which should be an external variable and hence tell us whether  
>>> we're
>>> actually linking against libiconv, whatever the header is) in  
>>> configure
>>
>> I'd have thought the AC_CHECK_DECL for _libiconv_version as used  
>> already
>> earlier in configure.ac is a sufficient test. If you try to  
>> compile zsh
>> with the iconv header file not matching what it links against, you  
>> get
>> bigger problems and it doesn't compile at all. In fact, there is
>> _LIBICONV_VERSION which is a #define so you could #ifdef that.
>
> I wanted to be safe about that and as far as I can tell checking if  
> we get
> _libiconv_version linked in should do that, right?

I posted the libiconv-issues on the libiconv-list and got the
following reply for reference:

> 	Von: 	  bruno@clisp.org
> 	Betreff: 	Re: [bug-gnu-libiconv] Solaris libiconv vs. GNU libiconv  
> in terms of 646
> 	Datum: 	28. Februar 2008 01:37:21 MEZ
> 	An: 	  dam@blastwave.org, bug-gnu-libiconv@gnu.org
>
> Hello,
>
> Dagobert Michelsen wrote:
>
>> I am currently packaging zsh for Solaris when I noticed
>> some failing tests due to problems with libiconv.
>> It looks like Solaris uses 646 as standard which is
>> known to Solaris libiconv but not GNU libiconv.
>>
>
> Yes, this is a known problem. Solaris uses "646" to denote the ASCII
> encoding, and GNU libiconv does not.
>
> What is standard in the realm of character encodings, is defined by  
> IANA,
> here:
>   http://www.iana.org/assignments/character-sets
>
>
>> I already found this in the notes:
>>
>>    Q: Support encodings mentioned in RFC 1345 ?
>>    A: No, they are not in use any more. Supporting ISO-646 variants
>> is pointless
>>       since ISO-8859-* have been adopted.
>>
>
> The issue is actually slightly different. The encoding itself is well
> supported by both Solaris iconv and GNU libiconv. Only the name used
> by Solaris is a fantasy name.
>
>
>> The troubles from the zsh-people is best described at
>>     <http://www.zsh.org/mla/workers/2008/msg00271.html>
>> The ZSH-people are currently building a workaround, but
>> this whole situation should be better addresses in libiconv.
>>
>
> This situation has been addressed in full generality - there is not  
> only
> Solaris and "646", there is also HP-UX and "hp15CN", and many others -
> in the gnulib module 'iconv_open', here:
>
>    http://www.gnu.org/software/gnulib/MODULES.html#module%3Diconv_open
>
> Regarding the workaround that you are doing:
>
>   - The idea to use "US-ASCII" for GNU libiconv but "646" for  
> Solaris iconv
>     is right.
>
>   - Instead of a configure-time test you can also use a simpler  
> compile-time
>     test:
>
>       #ifdef _LIBICONV_VERSION
>         /* using GNU libiconv */
>       #else
>         /* using the native system iconv */
>         #ifdef __sun
>           /* using the Solaris iconv */
>         #else
>           /* other systems */
>         #endif
>       #endif
>
>   - Regarding the recommendation to use Solaris iconv, I recommend the
>     opposite: Solaris iconv is known to crash in some situations.
>
> Bruno
>
> PS: This mail should appear as
> <http://lists.gnu.org/archive/html/bug-gnu-libiconv/2008-02/ 
> msg00003.html>



Best regards

   -- Dagobert


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: * Re: Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC
  2008-03-03 12:43                   ` Dagobert Michelsen
@ 2008-03-03 12:59                     ` Peter Stephenson
  0 siblings, 0 replies; 18+ messages in thread
From: Peter Stephenson @ 2008-03-03 12:59 UTC (permalink / raw)
  To: Zsh Hackers' List

Dagobert Michelsen wrote:
> > I wanted to be safe about that and as far as I can tell checking if  
> > we get
> > _libiconv_version linked in should do that, right?
> 
> I posted the libiconv-issues on the libiconv-list and got the
> following reply for reference:

Thanks, I think that means that what we've got is OK, but as things
stand we could get away with something simpler; unless someone has a
killer argument for the simpler #ifdef test I'll leave it as it is.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2008-03-03 13:00 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-02-26 13:18 Failed tests of zsh 4.3.5 in Solaris 10 w/Sun Studio 12 CC Dagobert Michelsen
2008-02-26 14:14 ` Peter Stephenson
2008-02-26 14:42   ` Dagobert Michelsen
2008-02-26 14:57     ` Peter Stephenson
2008-02-26 15:18       ` Dagobert Michelsen
2008-02-26 16:10         ` * " Peter Stephenson
2008-02-26 17:37           ` Oliver Kiddle
2008-02-27 11:29             ` Peter Stephenson
2008-02-27 16:16               ` Bart Schaefer
2008-02-29  9:53               ` Oliver Kiddle
2008-03-03 12:08                 ` Peter Stephenson
2008-03-03 12:43                   ` Dagobert Michelsen
2008-03-03 12:59                     ` Peter Stephenson
2008-02-26 18:08           ` Peter Stephenson
2008-02-26 18:12             ` İsmail Dönmez
2008-02-26 19:19             ` Dagobert Michelsen
2008-02-26 19:47               ` Dagobert Michelsen
2008-02-26 20:42                 ` Peter Stephenson

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).